CN112702576A - Data acquisition plug-flow display method for vehicle-mounted video - Google Patents
Data acquisition plug-flow display method for vehicle-mounted video Download PDFInfo
- Publication number
- CN112702576A CN112702576A CN202011544980.3A CN202011544980A CN112702576A CN 112702576 A CN112702576 A CN 112702576A CN 202011544980 A CN202011544980 A CN 202011544980A CN 112702576 A CN112702576 A CN 112702576A
- Authority
- CN
- China
- Prior art keywords
- data
- vehicle
- video
- thread
- stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000004458 analytical method Methods 0.000 claims abstract description 42
- 230000005540 biological transmission Effects 0.000 claims abstract description 36
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000004891 communication Methods 0.000 claims abstract description 9
- 239000000872 buffer Substances 0.000 claims description 25
- 230000008569 process Effects 0.000 claims description 10
- 238000005516 engineering process Methods 0.000 description 7
- 206010047571 Visual impairment Diseases 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000446 fuel Substances 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012858 packaging process Methods 0.000 description 1
- 239000003208 petroleum Substances 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0841—Registering performance data
- G07C5/085—Registering performance data using electronic data carriers
- G07C5/0866—Registering performance data using electronic data carriers the electronic data carrier being a digital video recorder in combination with video camera
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
Abstract
The invention provides a data acquisition plug flow display method of a vehicle-mounted video, which comprises the following steps: acquiring a video stream data acquisition signal, and executing and creating a capturer, a resolver and a plug flow device for capturing data from a preset interface, wherein the resolver and the plug flow device adopt asynchronous communication and asynchronous operation. The capturer comprises a capturing thread, and the resolver is created by the capturing thread; the analyzer comprises an analysis thread, the analyzer calls the interface to send data to the analysis thread, and the analyzer asynchronously executes analysis operation in the analysis thread. By adopting asynchronous communication, asynchronous processing and chained creating operation, the processing efficiency of data can be improved, time is saved, and the acquisition of video data and the plug-flow transmission to a remote server can be finished before the cached video data is automatically cleared in a video cache space.
Description
Technical Field
The invention relates to the field of data acquisition plug flow of vehicle-mounted videos, in particular to a data acquisition plug flow display method of a vehicle-mounted video.
Background
With the improvement of the living standard of people, an automobile becomes a basic consumer product, but with the development of the automobile towards the direction of intellectualization, safety and individuation, the combination of the vehicle-mounted camera and the automobile is more and more close, the application of the vehicle-mounted camera is more and more extensive, the vehicle-mounted camera becomes one of indispensable devices for traffic safety, a more scientific video and audio basis is provided for traffic accidents and positioning, and the guarantee is provided for property and life safety. However, in the conventional automobile all-round viewing system, a plurality of cameras are distributed on the front, the rear, the left and the right of the automobile body to capture image information around the automobile body, and images of the plurality of cameras at the same time are spliced to form a panoramic image. The camera mostly adopts traditional interfaces such as lvds, the output of the look-around system is a display screen, and no external data output interface is provided. Because the cameras mostly use traditional interfaces such as lvds and the like, the data volume is large, and the storage space of the look-around system as an embedded system is limited, a small amount of data caching can be carried out in the look-around system only due to the requirement of a look-around splicing algorithm, long-time data acquisition and storage cannot be carried out, and remote storage and online real-time playing cannot be carried out more easily without an output channel of original data of the cameras, which undoubtedly causes huge waste to video resources. Therefore, under the condition that the existing local video cache space at the vehicle end is not changed, the collection of the vehicle-mounted camera is completed before the video data which is cached and shot by the vehicle-mounted camera is automatically cleared in the video cache space, and the collected data is transmitted to a remote server for storage, which is a great difficulty. In addition, although the traditional camera adopting the LVDS interface has high transmission rate, the traditional camera does not have a time synchronization function and cannot be applied to a low-delay scene in a vehicle, so in order to meet the intelligent requirement of automobile development, the invention provides a technical scheme for acquiring and storing videos shot by a vehicle-mounted Ethernet camera with a time synchronization function and low-delay transmission so as to meet the requirements of current and future video application scenes.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a data acquisition plug flow display method of a vehicle-mounted video, which specifically comprises the following steps:
creating a capturer for capturing video stream data from a preset network interface;
creating an analyzer for performing asynchronous analysis operation on the acquired data;
creating a stream pusher for asynchronously pushing and displaying the decoded video frames by using a rtmp protocol;
the resolver and the impeller adopt asynchronous communication.
A data acquisition plug flow display method of a vehicle-mounted video is characterized in that a capturer comprises a capturing thread, and an analyzer is created by the capturing thread; the analyzer comprises an analysis thread, the analyzer calls the interface to send data to the analysis thread, and the analyzer asynchronously executes analysis operation in the analysis thread.
A data acquisition plug-flow display method for vehicle-mounted videos further comprises the steps that one analyzer analyzes video data shot by one vehicle-mounted camera, and when data shot by a plurality of vehicle-mounted cameras exist, the plurality of analyzers simultaneously analyze data of shooting strokes of the vehicle-mounted cameras with different IDs.
A data acquisition plug-flow display method of a vehicle-mounted video is characterized in that video stream data come from a cache space shot by 1 or more vehicle-mounted cameras and temporarily stored in a preset position of a vehicle end, and the video data shot by the cameras are added with camera identification IDs and timestamps to be packaged by a preset protocol and then transmitted to the cache space of the preset position of the vehicle end; the timestamp comprises a calibrated master time and a local time;
the predetermined protocol for video data transmission includes IEEE 1722.
A data acquisition plug-flow display method for a vehicle-mounted video further comprises the steps that a capturer acquires video stream data, then identifies a vehicle-mounted camera ID of a message source according to a message field, distinguishes the video stream data shot by different vehicle-mounted cameras according to the ID and sends the video stream data to 1 or more corresponding resolvers for processing.
A data acquisition plug flow display method of a vehicle-mounted video is characterized in that a parser creates an annular queue, the annular queue creates a plurality of frame buffer areas and specifies the number and the size of the frame buffer areas, and the annular queue is used for caching the parsed data.
A data acquisition plug flow display method of a vehicle-mounted video, further, a plug flow device comprises 1 or more plug flow threads, the plug flow device calls an interface of the plug flow device to send data to the plug flow threads, and the plug flow device asynchronously executes plug flow operation in the plug flow threads; and the multiple push flow threads simultaneously process the data in the video stream data addresses received by the multiple push flow threads and carry out push flow operation.
A data acquisition plug-flow display method for a vehicle-mounted video further comprises the steps that a capturing thread searches a network card interface for capturing from a network card information list, and after the captured network card interface is found, the capturing thread executes corresponding connection, disconnection or quitting according to state identification switching of a state machine.
A data acquisition plug-flow display method for a vehicle-mounted video is further provided, and state switching of a state machine specifically comprises the following steps: and acquiring a current state identifier, if the current state identifier is represented as STAT _ QUIT, exiting the capturing thread, if the current state identifier is represented as STAT _ CONNET, connecting the capturing thread with a network card interface to acquire data, and if the current state identifier is represented as STAT _ DISCONNET, disconnecting the capturing thread.
A data acquisition plug flow display method of a vehicle-mounted video further comprises the steps of setting a corresponding number of resolvers according to the number of vehicle-mounted Ethernet cameras, simultaneously processing received vehicle-mounted Ethernet camera messages by the plurality of resolvers, filling resolved video stream data into a frame buffer area, and sending an address of the frame of video stream data to a plug flow thread corresponding to an ID (identity) of the vehicle-mounted Ethernet camera for processing when the frame of video stream data is resolved.
A data acquisition plug-flow display method of a vehicle-mounted video further comprises the steps that video flow data shot by a camera of a vehicle-mounted camera is transmitted by an IEEE1722 protocol, each frame of video is divided into a plurality of sections of messages to be transmitted, and a frame head mark, a frame tail mark, a check mark and a frame mark are added in the messages;
when the video data is analyzed, the analyzer judges whether the received message data is the same frame data according to the frame head identifier, the frame tail identifier, the check identifier and the frame identifier.
A data acquisition plug-flow display method for a vehicle-mounted video further comprises the steps that a plug-flow thread sends plug-flow data packets to a preset server for storage, in the process of plug-flow data transmission, a UDP protocol is adopted by the plug-flow data packets on a transmission layer, 1 check packet is sent after every N data packets are sent, and the check packets comprise associated information of the N data packets.
The beneficial technical effects are as follows:
1. according to the invention, by adopting asynchronous communication and asynchronous processing modes in a plurality of threads, the processing efficiency of data can be improved, the time is saved, and the problems that after the vehicle-mounted camera adopts IEEE1722 high-rate transmission to cause that continuous large-capacity video data is cached in a smaller video storage space of a vehicle end, the analysis of the vehicle-mounted camera is completed before the cached video data shot by the vehicle-mounted camera is automatically cleared in the video cache space under the condition of not changing the existing smaller cache space of local video of the vehicle end, and the video data is pushed to be transmitted to a remote server are solved.
2. The capturer, the resolver and the plug flow device are created in a chained mode, and compared with the prior art, the capturer, the resolver and the plug flow device can save system operation resources and reduce system operation load.
3. The video stream data comprises main time and local time, and after images shot by a plurality of vehicle-mounted cameras are collected at the same time and finally spliced into a panoramic image, the splicing synchronism in the video playback process can be ensured.
4. The invention transmits the video stream data shot by the camera of the vehicle-mounted camera by an IEEE1722 protocol to divide each frame of video into a plurality of sections for transmission, and adds a frame head identifier, a frame tail identifier, a check identifier and a frame identifier in a message; after the frame head identification, the frame tail identification, the check identification and the frame identification are adopted, the subsequent push stream data analysis is used for identifying and merging the frame video stream, and the technical problem that after the vehicle-mounted Ethernet camera is introduced, the single frame message cannot be transmitted by one frame due to the fact that the size of the single frame message is limited by a protocol is solved.
5. In order to solve the problems of time consumption and high bandwidth occupation caused by TCP protocol retransmission after data packet loss caused by congestion in network transmission, the plug flow data packet adopts a UDP protocol in a transmission layer, 1 more check packet is sent after N data packets are sent in data transmission, and the check packet comprises the associated information of the N data packets. After the lost data packet is obtained through the check packet, the lost data packet only needs to be retransmitted without retransmitting all data packets of the frame picture as required by adopting a TCP protocol in the prior art.
Drawings
The following drawings are only schematic illustrations and explanations of the present invention, and do not limit the scope of the present invention.
Fig. 1 is a logic diagram for creating modules and sub-threads for acquiring vehicle-mounted video data according to an embodiment of the present invention.
FIG. 2 is a logic diagram illustrating the state machine changes in the flow chart of vehicle video data acquisition in accordance with an embodiment of the present invention.
Fig. 3 is a flowchart illustrating a start-up of a packet capture state of vehicle-mounted video data according to an embodiment of the present invention.
Fig. 4 is a flowchart of vehicle-mounted video data acquisition according to an embodiment of the present invention.
Detailed Description
For a more clear understanding of the technical features, objects, and effects herein, embodiments of the present invention will now be described with reference to the accompanying drawings, in which like reference numerals refer to like parts throughout. For the sake of simplicity, the drawings are schematic representations of relevant parts of the invention and are not intended to represent actual structures as products. In addition, for simplicity and clarity of understanding, only one of the components having the same structure or function is schematically illustrated or labeled in some of the drawings.
The term "connected" in the present invention may include direct connection, indirect connection, communication connection, and electrical connection, unless otherwise specified.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, values, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, values, steps, operations, elements, components, and/or groups thereof. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items
It should be understood that the term "vehicle" or "vehicular" or other similar terms as used herein generally includes motor vehicles such as passenger automobiles including Sport Utility Vehicles (SUVs), buses, trucks, various commercial vehicles, watercraft including a variety of boats, ships, aircraft, and the like, and includes hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other alternative fuel vehicles (e.g., fuels derived from non-petroleum sources). As referred to herein, a hybrid vehicle is a vehicle having two or more power sources, such as both gasoline-powered and electric-powered vehicles.
The implementation provides a data acquisition plug flow display method of a vehicle-mounted video, which comprises the following steps:
creating a capturer for capturing video stream data from a preset network interface;
creating an analyzer for performing asynchronous analysis operation on the acquired data;
creating a stream pusher used for pushing the analyzed video frame to a server specified by a user by using an rtmp protocol;
the capturer comprises 1 capturing thread and creates a parser corresponding to each video stream according to the source of the captured camera video stream.
The analyzer comprises an analysis thread, the capturer calls an interface of the analyzer to send the message data to the analyzer corresponding to the video stream, and the analyzer asynchronously executes analysis operation in the analysis thread; when data shot by a plurality of vehicle-mounted cameras exist, the plurality of analysis analyzers simultaneously analyze data of vehicle-mounted camera shooting strokes with different IDs in an analysis thread of the analysis analyzers.
Specifically, the vehicle-mounted cameras comprise a plurality of cameras which can be arranged on the roof of the vehicle and on the front, back, left and right of the vehicle body, wherein pictures shot by the four cameras on the front, back, left and right of the vehicle body can form 360-degree all-around pictures after being spliced;
the transmission of the vehicle-mounted camera adopts a vehicle-mounted Ethernet bus, and a transmission interface is a 100base-T1 interface;
the method comprises the following steps that 1 or more vehicle-mounted Ethernet cameras can be collected according to requirements by collecting video streams;
the video stream data are shot by 1 or more vehicle-mounted cameras and temporarily stored in a cache space at a preset position of a vehicle end, and the video data shot by the cameras are added with camera identification IDs and timestamps, packaged by a preset protocol and transmitted to the cache space at the preset position of the vehicle end;
the cameras at different positions are identified through IDs, and the capturer acquires video stream data IDs and timestamps from the message data according to the received message data; the capturer distinguishes which camera comes from the video stream data according to the ID and distributes the video stream data to the analyzer;
the time stamp includes a calibrated master time and a local time.
The main time is defined as the time for time synchronization of each ECU in the in-vehicle network, the transmission delay and clock drift compensation are carried out on the time of each ECU, the local time is the video streaming time of the vehicle-mounted Ethernet camera, and the main time and the local time are in one-to-one correspondence;
because the storage space of the existing vehicle end is very limited, the data of the video frames which can be cached is very small, and if the storage space needs to be increased, the cost is undoubtedly increased. On the other hand, a large data capacity at the time of invocation also causes a slowness of the system at the time of invocation of an ECU whose configuration is not high. Therefore, the minimum data acquisition rate without changing the existing storage space is to ensure that the system is automatically cleared after the acquisition of each piece of video stream data is finished, and if the minimum data acquisition rate is lower than the existing storage space, the video rate is lost. Therefore, in order to provide special requirements of the vehicle-mounted Ethernet for shooting the data collected by the video stream, an asynchronous communication mechanism is adopted in communication, and the number of resolvers is the same as that of the video stream shot by the vehicle-mounted camera needing to be resolved. The parse thread is created by the parser, and the parser is created by the capture thread. Therefore, the number of analysis threads can be flexibly established according to the number of the cameras which need to be acquired, the analysis threads are in one-to-one correspondence, the analysis efficiency of the video stream is improved, and the resource load of the system is reduced.
Specifically, as shown in fig. 1, a parser is created by a capture thread, the parser includes a parsing thread and a circular queue, the parsing thread creates a pusher, the pusher includes a pushing thread, the parser calls an interface of the pusher to send data to the pusher, and the pusher asynchronously executes pushing operation in the pushing thread.
The ring queue is formed by establishing a plurality of frame buffer areas, and the number and the size of the frame buffer areas can be specified;
specifically, it should be noted that, because the parser, the parsing thread, the circular queue, the stream pusher, and the stream pushing thread are all in one system, in order to save resources, in this embodiment, data transmission of the parsed video stream frames in the same operating system does not adopt a data value copy manner, but transmits addresses of data storage in the circular queue corresponding to frame data storage, and the stream pusher acquires data according to the received addresses. By this design, space and data transfer time can be saved.
In this embodiment, in order to improve efficiency, the capture thread, the parsing thread, and the stream pushing thread all work simultaneously and are executed asynchronously. Such as: there are 4 vehicle-mounted cameras, and correspondingly, there are 4 analysis threads, 4 plug-flow threads and 4 annular queues. In order to improve the efficiency, asynchronous execution processing is adopted, when video stream data messages shot by 4 cameras are captured, 4 analysis threads can simultaneously analyze, and 4 plug-flow threads can simultaneously plug-flow; parsing and plug flow can be done simultaneously without waiting. Compared with a single-thread mode (namely, stream pushing is carried out after one frame is analyzed, and the next frame is analyzed and stream pushing is carried out after the current frame is waited to be analyzed), the analysis and stream pushing multithreading mode is adopted, after one frame of data is analyzed by the analysis thread, the next frame of data can be analyzed without waiting for the completion of the stream pushing operation of the frame of data, and the processing efficiency can be improved.
Each flow pusher comprises 1 flow pushing thread, the analysis thread calls an interface of the flow pusher to send data to the flow pusher, and the flow pusher asynchronously executes flow pushing operation in the flow pushing thread; and the multiple push flow threads simultaneously process the data in the video stream data addresses received by the multiple push flow threads and carry out push flow operation.
The capturing thread acquires video stream data, then identifies the ID of the vehicle-mounted camera of the message source according to the message field, processes the video stream data from different vehicle-mounted cameras according to the ID and sends the video stream data to the corresponding 1 or more resolvers for processing, the capturing thread is responsible for the creation and destruction of each video stream resolver, searches for a network card interface which can be used for capturing, establishes connection and disconnection with a specified interface, captures 1722 protocol data packets, distinguishes video streams according to the ID and distributes the video streams to each video stream resolver.
The method specifically comprises the following steps: the capturing thread searches a network card interface for capturing from the network card information list, and after the captured network card interface is found, the capturing thread executes corresponding connection, disconnection or exit according to the state identification switching of the state machine;
capturing the original data packet, whether it is destined for a machine or exchanged on another device (shared medium);
before the data packet is sent to a certain application program, filtering the data packet according to a rule specified by a user;
sending out the original data packet through a network;
collecting and counting network flow information;
as shown in fig. 2, the state switching of the state machine specifically includes: acquiring a state identifier of a current state machine, if the current state identifier is represented as STAT _ QUIT, exiting the capturing thread, if the current state identifier is represented as STAT _ CONNET, connecting the capturing thread with a network card interface to acquire data, and if the current state identifier is represented as STAT _ DISCONNET, disconnecting the capturing thread.
Specifically, referring to fig. 3, the packet capture state initiation of the vehicle-mounted video data includes the following methods:
step S1: creating an analyzer, wherein the analyzer creates an analysis thread;
step S2: acquiring a state identifier of the current state machine, if the current state identifier is STAT _ CONNET, opening an interface, and executing the step S3; if the current state is marked as STAT _ DISCCONNET, judging whether the interface is closed, if the interface is not closed, continuing to execute the step S4, and if the interface is closed, executing the step S5; if the current state identification is STAR _ QUIT, ending;
step S3: starting packet capturing, after capturing the data packet, if the state identifier M _ state is STAT _ CONNECT, distributing the data packet, otherwise, executing step S4;
step S4: closing the interface;
step S5: the jump returns to execute step S2.
Setting a corresponding number of analysis sub-threads according to the number of the vehicle-mounted Ethernet cameras, simultaneously processing the vehicle-mounted Ethernet camera messages received by the analysis sub-threads, filling the analyzed video stream data into a frame buffer area, and sending the address of the frame of video stream data to a stream pushing thread corresponding to the ID of the vehicle-mounted Ethernet camera for processing when one frame of video stream data is analyzed;
and adding a corresponding timestamp and the ID of the vehicle-mounted Ethernet camera to each frame of analyzed video stream data.
The annular queue comprises a plurality of video stream frame buffers for caching the analyzed video stream;
specifically, the interface is designed: construction function int id of video stream
Analyzing an interface: sending a data updating signal for driving the analysis processing function to execute;
QByteArray & block: a byte stream to be parsed;
data update signal, qbyterarray block: a byte stream to be parsed;
an analysis processing function for performing data analysis in the analysis sub-thread; and copying the data to a frame buffer area, and calling a stream pushing interface of the stream pusher for processing after one frame is analyzed. QByteArray block: a byte stream to be parsed.
The resolver creates a ring queue and controls and manages the ring queue;
the ring queue is composed of a plurality of frame buffers, and the number and the size of the frame buffers are required to be specified when the ring queue is created.
The ring queue creates and destroys frame buffers.
The ring queue manages write operations of the ring queue.
Construction function int nCount, number of frame buffer area, int nSize, size of each frame buffer area
GetBufferToWrite: obtaining the next writable Frame buffer object address, and returning the value Frame: a frame address;
m _ nSize: number of created frame buffer objects
m _ nWrPos: location in queue of currently writing frame buffer object
m _ list: pointer arrays for frame buffer objects;
and a multithreading technology is used for processing the data of the multiple paths of cameras, so that the data processing efficiency is improved.
And capturing the multi-channel mixed camera message by 1 capturing thread on the whole, distinguishing a message source camera ID according to the field, and distributing the message source camera ID to an analysis thread corresponding to the camera ID for processing.
In the prior art, an on-vehicle camera is based on an LDVS interface, an LVDS (Low-Voltage Differential Signaling) Low-Voltage Differential signal is a Differential signal technology with Low power consumption, Low error rate, Low crosstalk and Low radiation, the transmission technology can reach more than 155Mbps, the core of the LVDS technology is to adopt an extremely Low Voltage swing high-speed Differential transmission data, so that point-to-point or point-to-multipoint connection can be realized, and a transmission medium of the LVDS technology is a copper PCB connecting line or a balance cable. When LVDS transmission is adopted, the LVDS transmission can be carried out at a high speed, but the LVDS transmission does not have a time synchronization function, after images shot by a plurality of vehicle-mounted cameras are collected simultaneously and finally spliced into a panoramic image, if time synchronization is lacked, the splicing synchronization is difficult to ensure in the video playback process after data collection, therefore, in order to solve the technical problem, the vehicle-mounted Ethernet cameras are adopted in the embodiment to shoot images and are packaged with IEEE1722 in the data transmission process, and the main time and the local time are added in the packaging process.
In addition, after the vehicle-mounted Ethernet camera is introduced, the data acquisition mode of the vehicle-mounted Ethernet camera is different from that of a camera with an LVDS interface in the aspect of data acquisition. However, after the vehicle-mounted ethernet camera is transmitted based on the IEEE1722 protocol, the size of each frame of the transmitted frame is limited to 1500 bytes at most, but the size of each frame of the picture far exceeds 1500 bytes, the stream pushing operation cannot be realized according to the existing acquisition scheme, and the technical problem and the solution are not disclosed in the prior art. Based on this, the present implementation therefore provides a solution;
referring to fig. 4, the improved method for collecting vehicle-mounted video data is as follows:
the specific implementation mode is as follows:
transmitting video stream data shot by a camera of a vehicle-mounted camera by an IEEE1722 protocol to divide each frame of video into a plurality of sections for transmission, and adding a frame head identifier, a frame tail identifier, a check identifier and a frame identifier in a message;
the frame head mark indicates that each frame of video is divided into the first section of multi-section transmission, the frame tail mark indicates that each frame of video is divided into the last section of multi-section transmission, and the frame mark indicates that the load data transmitted by the current message belongs to the same frame;
step S101: the analyzer judges whether the received message data belongs to the segmented data of the same frame, and if the received message data belongs to the segmented data of the same frame, the received message data is put into a buffer area belonging to a frame queue;
judging whether the frame identification is a frame identification or not, and if the frame identification is the same, determining the segmented data belonging to the same frame;
step S102: judging whether the accumulation of the received frame segment data is satisfied with one frame data, if so, sending, and if not, continuing to wait for the subsequent frame segment data to be filled into a buffer queue;
each frame of data is judged according to the frame head identification, the frame tail identification and the check bit;
step S103: after the resolver sends a frame of data, a new address is obtained from the circular queue again for storing the next frame of data.
In order to process multiple paths of camera data simultaneously, 4 paths of analysis threads are used for processing the received camera messages simultaneously, the analyzed video stream data are filled into a frame buffer area, and when enough messages are received, and one frame of video stream data is analyzed, the address of the frame of video stream data is sent to the plug flow thread processing corresponding to the camera ID.
Because the analysis thread needs to analyze a plurality of messages to analyze a frame of video stream, the operation of pushing the current frame and the operation of analyzing the next frame can be carried out simultaneously, and the pushing operation is also carried out in an independent thread. And the 4-path stream pushing threads simultaneously process the data in the video stream data addresses received by the 4-path stream pushing threads respectively to perform stream pushing operation.
Asynchronous communication is adopted among multiple threads, namely the capture thread calls the analysis thread to execute analysis operation in an asynchronous mode, and the analysis thread calls the plug flow thread to execute plug flow operation in an asynchronous mode.
The embodiment can be implemented by adopting a third-party plug-in, specifically, capture of the data packet by the capture thread can adopt Winpcap, ffmpeg is adopted by the stream pusher, and the h264 frame is displayed in a rtmp protocol stream pushing mode and is saved to a designated server at the same time.
Specifically, in the prior art, online display is performed by pushing the stream to the internet, because there is a difference in the environment of internet transmission, at a certain moment, when a large amount of transmission occupies a bandwidth, network congestion may occur, and at this moment, because there is a time delay and packet loss in unstable stream transmission of the network, a situation that TCP response acknowledgement cannot be received occurs in transmission using a TCP protocol, so that the transmission speed is reduced, and the stream transmission quality cannot be guaranteed.
In order to ensure transmission quality and reduce packet loss rate, the implementation provides a stream pushing transmission method, where a stream pushing data packet uses a UDP protocol in a transmission layer, and during data transmission, 1 more check packet is sent after each N data packets are sent, where the check packet includes associated information of the N data packets, and the associated information may be other related information such as a frame header, a frame address, and a category to which the frame address belongs of the N data packets, but does not include load data (payload) of the N data packets. In this way, a packet is lost in N +1 packets, and the lost packet can be calculated by checking the packets. Compared with the existing push flow technology, the push flow technology is based on the TCP protocol, all data packets need to be retransmitted after packet loss, and time and bandwidth are saved.
What has been described above is only a preferred embodiment of the present invention, and the present invention is not limited to the above examples. It is clear to those skilled in the art that the form in this embodiment is not limited thereto, and the adjustable manner is not limited thereto. It is to be understood that other modifications and variations, which may be directly derived or suggested to one skilled in the art without departing from the basic concept of the invention, are to be considered as included within the scope of the invention.
Claims (10)
1. A data acquisition plug-flow display method of vehicle-mounted video is characterized in that,
creating a capturer for capturing video stream data from a preset network interface;
creating an analyzer for performing asynchronous analysis operation on the acquired data;
creating a plug flow device for asynchronously plug-flow displaying the analyzed video frame by a rtmp protocol;
the resolver and the impeller adopt asynchronous communication.
2. The data acquisition plug-flow display method of the vehicle-mounted video according to claim 1, wherein the capturer comprises a capturing thread, and the parser is created by the capturing thread; the analyzer comprises an analysis thread, the analyzer calls the interface to send data to the analysis thread, and the analyzer asynchronously executes analysis operation in the analysis thread.
3. The data acquisition and plug-flow display method of the vehicle-mounted video as claimed in claim 2, wherein a parser parses video data shot by a vehicle-mounted camera, and when there are multiple data shot by vehicle-mounted cameras, multiple parsers perform parsing operation on data of shooting strokes of vehicle-mounted cameras with different IDs at the same time.
4. The data acquisition and stream pushing display method of the vehicle-mounted video as claimed in claim 1, wherein the video stream data is from a buffer space shot by 1 or more vehicle-mounted cameras and temporarily stored in a preset position of the vehicle end, and the video data shot by the cameras is encapsulated by a preset protocol with a camera identification ID and a timestamp and then transmitted to the buffer space in the preset position of the vehicle end; the timestamp comprises a calibrated master time and a local time;
the predetermined protocol for video data transmission includes IEEE 1722.
5. The data acquisition plug-flow display method of the vehicle-mounted video as claimed in claim 2, wherein the capturer identifies the vehicle-mounted camera ID of the message source according to the message field after acquiring the video stream data, distinguishes the video stream data shot by different vehicle-mounted cameras according to the ID and sends the video stream data to the corresponding 1 or more resolvers for processing.
6. The data acquisition and stream pushing display method of the vehicle-mounted video as claimed in claim 2, wherein the parser creates a ring queue, the ring queue creates a plurality of frame buffers and specifies the number and the size of the frame buffers, and the ring queue is used for caching the parsed data.
7. The data acquisition, stream pushing and displaying method of the vehicle-mounted video as claimed in claim 1, wherein the stream pusher comprises a stream pushing thread, the stream pusher calls an interface of the stream pusher to send data to the stream pushing thread, and the stream pusher asynchronously executes stream pushing operation in the stream pushing thread; when the stream pusher comprises a plurality of paths, the plurality of paths of stream pushing threads simultaneously process the data in the video stream data addresses received by the plurality of paths of stream pushing threads and carry out stream pushing operation.
8. The data acquisition plug-flow display method of the vehicle-mounted video according to claim 1, wherein the capturing thread searches a network card interface for capturing from the network card information list, and executes corresponding connection, disconnection or exit according to state identification switching of the state machine after the captured network card interface is found.
9. The data acquisition plug-flow display method of the vehicle-mounted video according to claim 9, wherein the state switching of the state machine specifically comprises: and acquiring a current state identifier, if the current state identifier is represented as STAT _ QUIT, exiting the capturing thread, if the current state identifier is represented as STAT _ CONNET, connecting the capturing thread with a network card interface to acquire data, and if the current state identifier is represented as STAT _ DISCONNET, disconnecting the capturing thread.
10. The method as claimed in claim 1, wherein a corresponding number of parsers are provided according to the number of the vehicle-mounted ethernet cameras, the parsers process the received vehicle-mounted ethernet camera messages simultaneously, the parsed video stream data are filled into a frame buffer, and when a frame of video stream data is parsed, an address of the frame of video stream data is sent to a stream pushing thread corresponding to the ID of the vehicle-mounted ethernet camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011544980.3A CN112702576A (en) | 2020-12-24 | 2020-12-24 | Data acquisition plug-flow display method for vehicle-mounted video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011544980.3A CN112702576A (en) | 2020-12-24 | 2020-12-24 | Data acquisition plug-flow display method for vehicle-mounted video |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112702576A true CN112702576A (en) | 2021-04-23 |
Family
ID=75509560
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011544980.3A Pending CN112702576A (en) | 2020-12-24 | 2020-12-24 | Data acquisition plug-flow display method for vehicle-mounted video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112702576A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113286175A (en) * | 2021-04-27 | 2021-08-20 | 金卯新能源集团有限公司 | Video stream processing method, device and storage medium |
| CN113395489A (en) * | 2021-06-09 | 2021-09-14 | 哈尔滨低空探索智能科技有限责任公司 | Video switching method and video switching system suitable for network camera |
| CN113824973A (en) * | 2021-08-04 | 2021-12-21 | 杭州星犀科技有限公司 | Multi-platform direct-push plug flow method, system, electronic device and storage medium |
| CN114827491A (en) * | 2022-04-18 | 2022-07-29 | 鹰驾科技（深圳）有限公司 | Wireless transmission panoramic all-around splicing technology |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105743597A (en) * | 2014-12-25 | 2016-07-06 | 瑞萨电子株式会社 | Semiconductor device, electronic device module and network system |
| CN105843735A (en) * | 2016-03-18 | 2016-08-10 | 广东欧珀移动通信有限公司 | Terminal memory consumption method and apparatus |
| CN107027042A (en) * | 2017-04-19 | 2017-08-08 | 中国电子科技集团公司电子科学研究院 | A kind of panorama live video stream processing method and processing device based on many GPU |
| CN109640120A (en) * | 2018-12-18 | 2019-04-16 | 浩云科技股份有限公司 | A kind of high concurrent audio-video frequency media data network transmission method and device |
-
2020
- 2020-12-24 CN CN202011544980.3A patent/CN112702576A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105743597A (en) * | 2014-12-25 | 2016-07-06 | 瑞萨电子株式会社 | Semiconductor device, electronic device module and network system |
| CN105843735A (en) * | 2016-03-18 | 2016-08-10 | 广东欧珀移动通信有限公司 | Terminal memory consumption method and apparatus |
| CN107027042A (en) * | 2017-04-19 | 2017-08-08 | 中国电子科技集团公司电子科学研究院 | A kind of panorama live video stream processing method and processing device based on many GPU |
| CN109640120A (en) * | 2018-12-18 | 2019-04-16 | 浩云科技股份有限公司 | A kind of high concurrent audio-video frequency media data network transmission method and device |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113286175A (en) * | 2021-04-27 | 2021-08-20 | 金卯新能源集团有限公司 | Video stream processing method, device and storage medium |
| CN113395489A (en) * | 2021-06-09 | 2021-09-14 | 哈尔滨低空探索智能科技有限责任公司 | Video switching method and video switching system suitable for network camera |
| CN113824973A (en) * | 2021-08-04 | 2021-12-21 | 杭州星犀科技有限公司 | Multi-platform direct-push plug flow method, system, electronic device and storage medium |
| CN114827491A (en) * | 2022-04-18 | 2022-07-29 | 鹰驾科技（深圳）有限公司 | Wireless transmission panoramic all-around splicing technology |
| CN114827491B (en) * | 2022-04-18 | 2023-02-14 | 鹰驾科技（深圳）有限公司 | Wireless transmission panoramic view splicing technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112702576A (en) | Data acquisition plug-flow display method for vehicle-mounted video | |
| CN108881927B (en) | Video data synthesis method and device | |
| CN110022297B (en) | High-definition video live broadcast system | |
| KR101278632B1 (en) | Method for performing data transport over a serial bus using internet protocol and apparatus for use in the method | |
| CN112383452B (en) | DPDK frame-based DDS data transmission diagnosis method and system | |
| CN113132552B (en) | Video stream processing method and device | |
| CN112104918A (en) | Image transmission method and device based on satellite network | |
| CN114598843A (en) | Image processing system and method applied to multi-path cameras of large automobile | |
| CN112104893A (en) | Video stream management method and device for realizing plug-in-free playing of webpage end | |
| EP2099193A1 (en) | Data transport container for transferring data in a high speed internet protocol network | |
| Tuohy et al. | Hybrid testbed for simulating in-vehicle automotive networks | |
| CN101316161B (en) | Synchronous indication method and system for distributed video | |
| CN111147175B (en) | Time-triggered Ethernet data frame capturing and storing device and method | |
| CN112822722A (en) | Data message transmission method and device | |
| CN112738417A (en) | Data acquisition, storage and display method for vehicle-mounted video | |
| CN114501091B (en) | Method and device for generating remote driving picture and electronic equipment | |
| CN108650160B (en) | Bus bridge based on chain type port and working method thereof | |
| CN110798688A (en) | High-definition video compression coding system based on real-time transmission | |
| CN102438182A (en) | Method and device for analyzing video stream | |
| CN115550350A (en) | Intelligent driving automobile data uploading method and system | |
| CN114979793A (en) | Live broadcast data transmission method, device, system, equipment and medium | |
| CN116668415A (en) | Streaming media data processing method and system | |
| CN110474934B (en) | Data processing method and video networking monitoring platform | |
| CN112399381A (en) | Data transmission method and vehicle-mounted Ethernet transmission system | |
| CN113950825A (en) | Data transmission method, data processing method, movable platform, terminal equipment and chip |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210423 |
|
| RJ01 | Rejection of invention patent application after publication |