CN114727070A - Camera real-time video data processing method and device - Google Patents

Camera real-time video data processing method and device Download PDF

Info

Publication number
CN114727070A
CN114727070A CN202210504413.8A CN202210504413A CN114727070A CN 114727070 A CN114727070 A CN 114727070A CN 202210504413 A CN202210504413 A CN 202210504413A CN 114727070 A CN114727070 A CN 114727070A
Authority
CN
China
Prior art keywords
data
video data
camera
real
component
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210504413.8A
Other languages
Chinese (zh)
Inventor
黄云
黄劲
徐文
覃念芳
龙伊雯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Electronic Technology Co Ltd
Original Assignee
Guangzhou Electronic Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Electronic Technology Co Ltd filed Critical Guangzhou Electronic Technology Co Ltd
Priority to CN202210504413.8A priority Critical patent/CN114727070A/en
Publication of CN114727070A publication Critical patent/CN114727070A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/01Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

The invention discloses a method and a device for processing real-time video data of a camera, which are characterized in that a streaming pipeline design is carried out by a gstreamer streaming media frame, the real-time video data of the camera is collected, a connection shared memory is created, the video data format is converted into a format required by output by configuring a corresponding plug-in according to the streaming mode format output by the camera, finally, an application program interaction assembly is connected to store a video data frame to a shared memory annular buffer area, other application processes acquire the real-time video data from the shared memory, the multi-process interaction of the real-time video data is realized, the problem of resource exclusivity of starting the camera is solved, compared with the method that an opencv library function is directly called to carry out video collection processing, and the collection efficiency of a system is also improved to a certain extent.

Description

Camera real-time video data processing method and device
Technical Field
The invention relates to the technical field of video processing, in particular to a method and a device for processing real-time video data of a camera.
Background
The MIPI-CSI interface camera is widely applied to a plurality of embedded products such as a vehicle traveling data recorder, a law enforcement instrument and a evidence obtaining system, multi-process video image acquisition processing is needed in some application scenes, but the MIPI-CSI interface camera cannot be started by different application programs at the same time, namely a certain camera is started in one process, the same camera cannot be started in the other process, and camera resources are monopolized.
The Gstreamer multimedia framework can be used for engineering development through a code direct encapsulation command, adopts a plug-in implementation mode, has good reusability of program codes, can flexibly cut plug-ins according to business requirements, has good expansibility, and is suitable for application fields such as embedded type and mobile terminals. The Gstreamer can realize a series of media solutions such as collection, encoding, decoding, rendering, filtering and the like. In an embedded image processing device system, sometimes, multiple processes are required to process real-time video data, one process starts a camera (for example, a camera of an MIPI-CSI2 interface adopted in this patent), resources are monopolized, multiple application processes that need to acquire video data cannot acquire video data directly, an udp network is usually adopted to send an RTP stream, or a server stream pushing mode (rtsp stream, rtmp stream) is adopted to realize real-time video data interaction of multiple processes, an existing patent document CN202011554999.6 discloses a high-concurrency video processing method and system for sharing data by multiple processes, and an existing patent document CN110891178A discloses a method and device for video real-time rendering, which all need to process through a coding and decoding process. However, for the embedded image processing device with low power consumption, the above methods all involve the encoding and decoding process and the micro server push stream loading, which increases the power consumption of the system, and especially causes the system to be overloaded and run slowly when multiple videos are processed.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a method and a device for processing real-time video data of a camera.
In order to achieve the purpose, the invention adopts the following technical scheme:
a camera real-time video data processing method comprises the following steps:
s1, according to the number of the configured cameras, starting corresponding number of threads to manage each path of real-time video;
s2, constructing a gstreamer pipeline in each thread to be responsible for the conversion processing of each path of video data;
s3, at least one video source component is included in the gstreamer pipeline, and the video data output by the camera is acquired through the video source component;
s4, the gstreamer pipeline at least comprises a reasoning component, and the reasoning component converts the video data into a video frame format required to be processed to obtain video frame data;
s5, the gstreamer pipeline comprises an application program interaction component, and the application program interaction component puts the video data frame into a ring-shaped shared memory area;
and S6, connecting a plurality of application processes needing to acquire video data to the annular shared memory area, and reading the video frame data from the annular shared memory area for processing.
Further, the step S2 further includes: each component attribute in the gstreamer pipeline is implemented by a standard GObject object method.
Further, the video source component is V4L2 src.
Further, the application program interaction component is an applink component.
Further, the step of transferring the sampled data of the appink component to the ring-shaped shared memory area includes:
s01, setting the emit-scales attribute of the appsink component in each gstreamer pipeline as "true";
s02, connecting the new-sample sampling signal of the applink component;
s03, storing the received video frame data in a buffer area of the constructed annular shared memory area in a message callback function, and setting the segment name of the video frame data;
and S04, acquiring real-time video frame data for processing in a plurality of application processes needing to acquire video data according to the corresponding annular shared memory area connected by the segment name.
The utility model provides a real-time video data processing apparatus of camera, real-time video data processing apparatus of camera is used for realizing the hardware device of real-time video data processing method of camera, includes camera group, data main control unit and display control screen, camera group with data main control unit connects, data main control unit with display control screen connects, main control unit includes data processing module, communication module and portable storage medium, data main control unit passes through communication module is connected with the customer end that needs to acquire video data, the external portable storage medium that removes of data processing module, communication module pass through usb with data processing module carries out data transmission.
Further, the camera group is an MIPI camera group.
Furthermore, the communication module is a 4G/WIFI module.
Furthermore, the cameras in the camera group are connected with the data processing module through usb, the communication module is fixedly connected with the data processing module through cylindrical firmware, the communication module is located above the data processing module, and the data processing module is an Nvdia Jeson series development board.
The invention has the beneficial effects that:
the invention relates to a method and a device for processing real-time video data of a camera, which are characterized in that a streaming pipeline design is carried out by a gstreamer streaming media frame, the real-time video data of the camera is collected, a connection shared memory is created, the video data format is converted into a format required by output by configuring a corresponding plug-in according to the streaming mode format output by the camera, finally, an application program interaction assembly is connected to store a video data frame to a shared memory annular buffer area, and a plurality of application processes needing to acquire the video data acquire the real-time video data from the shared memory, thereby realizing the multi-process interaction of the real-time video data and solving the problem of resource exclusivity of the opening of the camera. Meanwhile, compared with directly calling opencv library functions to carry out video acquisition processing, the acquisition efficiency of a system formed on the basis of the method and the device is improved to a certain extent.
Drawings
FIG. 1 is a schematic diagram illustrating a design of a real-time video multiprocess data interaction interface based on a gstreamer framework according to the present invention;
FIG. 2 is a schematic flow chart illustrating the transfer of sampled data from an appink device to the ring-shaped shared memory region according to the present invention;
FIG. 3 is a schematic diagram illustrating a connection mode of modules in a camera real-time video data processing apparatus according to the present invention;
FIG. 4 is a schematic structural diagram of a camera real-time video data processing apparatus according to an embodiment of the present invention;
reference numerals: 1. a data processing module; 2. a camera group; 3. and the 4G/wifi module.
Detailed Description
The invention will be further described with reference to the accompanying drawings and the detailed description below:
as shown in fig. 1, a method for processing real-time video data of a camera includes the following steps:
s1, according to the number of the configured cameras, starting a corresponding number of threads to manage each path of real-time video;
s2, constructing a streamer Pipeline (Pipeline) in each thread to be responsible for the conversion processing of each path of video data; each component attribute in the gstreamer pipeline is realized by a standard GObject object method, and the method of using GObject can inquire, set and acquire the value of the attribute for the gstreamer.
S3, at least one video source component is included in the gstreamer pipeline, and the video data output by the camera is acquired through the video source component; the video source component is V4L2 src.
S4, the gstreamer pipeline at least comprises a reasoning component, and the reasoning component converts the video data into a video frame format required to be processed to obtain video frame data;
s5, the gstreamer pipeline comprises an application program interaction component, and the application program interaction component puts the video data frame into a ring-shaped shared memory area; the application program interaction component is an applink component.
And S6, connecting a plurality of application processes needing to acquire video data to the annular shared memory area, and reading the video frame data from the annular shared memory area for processing.
Further, as shown in fig. 2, an interactive implementation method of video data and a shared memory of a gstreamer pipeline (taking an applink component (application program interaction component) as an example), where the step of transferring the sampled data of the applink component to the ring-shaped shared memory area includes:
s01, setting the emit-scales attribute of the appsink component in each gstreamer pipeline as "true";
s02, connecting the new-sample sampling signal of the applink component;
s03, storing the received video frame data in a buffer area of the constructed annular shared memory area in a message callback function, and setting the segment name of the video frame data;
and S04, acquiring real-time video frame data for processing in a plurality of application processes needing to acquire video data according to the corresponding annular shared memory area connected by the segment name.
As shown in fig. 3, a camera real-time video data processing device is a hardware device for implementing a camera real-time video data processing method, and includes a camera group 2, a data main control unit and a display control screen, where the camera group 2 is connected to the data main control unit, the data main control unit is connected to the display control screen, and the main control unit includes a data processing module 1, a communication module and a removable storage medium. The data master control unit is connected with a client end needing to acquire video data through the communication module, the data processing module is externally connected with a mobile storage medium, and the communication module performs data transmission with the data processing module through the usb.
The camera group 2 is used for outputting a real-time video, and the camera group 2 at least comprises one camera; the data processing module 1 is configured to obtain video data, convert the video data into a video frame format of a video to be processed, obtain video frame data, and further place the video frame data in the ring-shaped shared memory area. The removable storage medium is used for storing video frame data; the communication module can transmit the video frame processing data to an application process of the client; the communication module can transmit the video frame processing data to one or more application processes; the display control screen is used as a display and control terminal of the data processing module and can be configured or not.
Further, the camera group 2 is an MIPI camera group.
Further, the communication module is a 4G/WIFI module 3.
Further, the cameras in the camera group 2 are connected with the data processing module through usb, the communication module is fixedly connected with the data processing module 1 through cylindrical firmware, the communication module is located above the data processing module 1, and the data processing module is an Nvdia Jeson series development board.
As shown in FIG. 4, the multi-channel camera 2 is connected to the data processing module 1 through the usb, the 4G/wifi module 3 is fixed with the data processing module 1 through the cylindrical firmware, data are transmitted through the usb interface, and the data processing module 1 can be externally connected with a mobile storage medium.
The technical scheme of the invention is further explained by taking the double-process interactive application of the local video and the target object detection application as an example:
in the embodiment, the data processing module 1 adopts an Nvdia Jeson series Jetson Nano development board, is matched with an Ubuntu 18.04.2linux operating system, is configured with two VEYE-MIPI-327 model cameras, outputs an image resolution of 1920 x 1080, and realizes startup of a local video and target object detection application, and the target object detection adopts a deep learning framework based on Darknet.
Two threads are started, a gstreamer pipeline is constructed in each thread and is responsible for conversion processing of each video data, a V4L2src component is adopted in the pipeline to obtain UYVY mode data output by an MIPI-CSI2 camera, an nvvidconv video converter component is adopted to convert video frames into a 420I mode, pipeline stream data are distributed through a tee component, and two src pad distribution threads are created through a queue 2:
and one path of the video stream is encoded into h264 format through an encoder component omxh264enc, a matriskamuxc component encapsulates and converts the received data into a matriskal stream, and finally the pipeline stream is connected with a fileink component to write the pipeline stream into a file for video storage. And the other path continues to load a video format conversion component nvvidconv, converts video stream data from a 420I format into a BGRX format, loads a video converter video color space conversion component, converts received BGRX format data into a BGR format, finally connects an applink component to place video data frames in the BGR format into a ring-shaped shared memory area, and connects a darknet target detection application process to the created ring-shaped shared memory area to acquire image data from the ring-shaped shared memory area for vehicle target detection.
In an open-source Darknet target detection original frame, video data are obtained by calling a camera video acquisition function through an opencv software library, the two paths of video target detection processing frame rates are about 6 frames, a gstreamer pipeline is constructed to store the video data to a shared memory, a target detection process obtains video frame data from the shared memory, the two paths of video target detection processing frame rates can reach about 8 frames, and the system processing efficiency is improved to a certain extent.
In the embodiment, real-time video data of the camera is transmitted to the annular shared memory area through constructing a pipeline design based on a gstreamer, and is interacted with other application processes, so that the problem of exclusivity of the camera is solved, local video of a video in an independent process and target identification of a vehicle object are realized, and in the application process of vehicle target identification, compared with the method for directly calling an opencv video function to obtain an image frame, the method provided by the invention has the advantages that the video acquisition efficiency is improved to a certain extent, and the operation efficiency of the target identification process is improved.
Various other modifications and changes may be made by those skilled in the art based on the above-described technical solutions and concepts, and all such modifications and changes should fall within the scope of the claims of the present invention.

Claims (10)

1. A camera real-time video data processing method is characterized by comprising the following steps:
s1, according to the number of the configured cameras, starting a corresponding number of threads to manage each path of real-time video;
s2, constructing a gstreamer pipeline in each thread to be responsible for the conversion processing of each path of video data;
s3, at least one video source component is included in the gstreamer pipeline, and the video data output by the camera is acquired through the video source component;
s4, the gstreamer pipeline at least comprises a reasoning component, and the reasoning component converts the video data into a video frame format required to be processed to obtain video frame data;
s5, the gstreamer pipeline comprises an application program interaction component, and the application program interaction component puts the video data frame into a ring-shaped shared memory area;
and S6, connecting a plurality of application processes needing to acquire video data to the annular shared memory area, and reading the video frame data from the annular shared memory area for processing.
2. The method as claimed in claim 1, wherein the video data output from the camera is in a stream mode format, and the video frame format is in a BGR format.
3. The method for processing the real-time video data of the camera according to claim 1, wherein the step S2 further includes: each component attribute in the gstreamer pipeline is implemented by a standard GObject object method.
4. The method of claim 1, wherein the video source component is V4L2 src.
5. The method for processing real-time video data of a camera according to claim 1, wherein the application program interaction component is an applink component.
6. The method of claim 5, wherein the step of transferring the sampled data of the applink component to the ring-shaped shared memory area comprises:
s01, setting the emit-scales attribute of the appsink component in each gstreamer pipeline as "true";
s02, connecting the new-sample sampling signal of the applink component;
s03, storing the received video frame data in a buffer area of the constructed annular shared memory area in a message callback function, and setting the segment name of the video frame data;
and S04, acquiring real-time video frame data for processing in a plurality of application processes needing to acquire video data according to the corresponding annular shared memory area connected by the segment name.
7. The camera real-time video data processing device is characterized in that the camera real-time video data processing device is used for realizing a hardware device of the camera real-time video data processing method in claim 1, and comprises a camera group, a data main control unit and a display control screen, wherein the camera group is connected with the data main control unit, the data main control unit is connected with the display control screen, the main control unit comprises a data processing module, a communication module and a movable storage medium, the data main control unit is connected with a client side needing to acquire video data through the communication module, the data processing module is externally connected with the movable storage medium, and the communication module carries out data transmission with the data processing module through usb.
8. The device of claim 7, wherein the camera group is a MIPI camera group.
9. The device for processing real-time video data of a camera according to claim 7, wherein the communication module is a 4G/WIFI module.
10. The device for processing the real-time video data of the cameras according to claim 7, wherein the cameras in the camera group are connected with the data processing module through usb, the communication module is fixedly connected with the data processing module through cylindrical firmware, the communication module is located above the data processing module, and the data processing module is an Nvdia Jeson series development board.
CN202210504413.8A 2022-05-10 2022-05-10 Camera real-time video data processing method and device Pending CN114727070A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210504413.8A CN114727070A (en) 2022-05-10 2022-05-10 Camera real-time video data processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210504413.8A CN114727070A (en) 2022-05-10 2022-05-10 Camera real-time video data processing method and device

Publications (1)

Publication Number Publication Date
CN114727070A true CN114727070A (en) 2022-07-08

Family

ID=82231528

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210504413.8A Pending CN114727070A (en) 2022-05-10 2022-05-10 Camera real-time video data processing method and device

Country Status (1)

Country Link
CN (1) CN114727070A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115858081A (en) * 2023-02-24 2023-03-28 深圳市东微智能科技股份有限公司 UI (user interface) media resource display method, device and equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9387813B1 (en) * 2012-03-21 2016-07-12 Road-Iq, Llc Device, system and method for aggregating networks and serving data from those networks to computers
CN109194698A (en) * 2018-11-01 2019-01-11 浩云科技股份有限公司 A kind of Real-time Video Processing System and method based on GStreamer frame
CN110493626A (en) * 2019-09-10 2019-11-22 海信集团有限公司 Video data handling procedure and device
CN112099970A (en) * 2020-09-04 2020-12-18 中国第一汽车股份有限公司 Video data processing method, device, equipment and storage medium
CN112528961A (en) * 2020-12-28 2021-03-19 山东巍然智能科技有限公司 Video analysis method based on Jetson Nano
CN112804410A (en) * 2019-11-14 2021-05-14 西安诺瓦星云科技股份有限公司 Multi-display-screen synchronous display method and device, video processing equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9387813B1 (en) * 2012-03-21 2016-07-12 Road-Iq, Llc Device, system and method for aggregating networks and serving data from those networks to computers
CN109194698A (en) * 2018-11-01 2019-01-11 浩云科技股份有限公司 A kind of Real-time Video Processing System and method based on GStreamer frame
CN110493626A (en) * 2019-09-10 2019-11-22 海信集团有限公司 Video data handling procedure and device
CN112804410A (en) * 2019-11-14 2021-05-14 西安诺瓦星云科技股份有限公司 Multi-display-screen synchronous display method and device, video processing equipment and storage medium
CN112099970A (en) * 2020-09-04 2020-12-18 中国第一汽车股份有限公司 Video data processing method, device, equipment and storage medium
CN112528961A (en) * 2020-12-28 2021-03-19 山东巍然智能科技有限公司 Video analysis method based on Jetson Nano

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115858081A (en) * 2023-02-24 2023-03-28 深圳市东微智能科技股份有限公司 UI (user interface) media resource display method, device and equipment and storage medium
CN115858081B (en) * 2023-02-24 2023-06-16 深圳市东微智能科技股份有限公司 UI (user interface) media resource display method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN101500128B (en) Method and apparatus for loading additional information on display image of network camera device terminal
CN107155093B (en) Video preview method, device and equipment
US20090228620A1 (en) Digital Adapter
US20150296173A1 (en) Method and device for implementing analog high-definition image capturing
CN111935531A (en) Integrated display system graph processing method based on embedded platform
CN110366005A (en) Sending method and device, the display methods and device of analog composite video signal
CN114727070A (en) Camera real-time video data processing method and device
CN110958431A (en) Multi-channel video compression post-transmission system and method
CN102802022A (en) Coding and decoding system and method
CN206117878U (en) Intelligent video analysis device, equipment and video monitor system
CN112995611A (en) Lossless image acquisition and transmission method, device and system
EP3013025A1 (en) Multimedia data transmission method, and apparatus
CN203279065U (en) Multi-screen display NVR system
CN111654674A (en) Method, software system and terminal for processing multiple video sources in parallel based on single ffmpeg process
CN104883518A (en) Low-cost small-sized thermal infrared imager
CN114501091B (en) Method and device for generating remote driving picture and electronic equipment
US20210037208A1 (en) Compositing video signals and stripping composite video signal
CN113438460A (en) Lossless AR video acquisition and transmission method, device and system
CN101184177A (en) Digital television picture catching method and system
CN210381033U (en) Low-power-consumption video imaging device
CN104010221B (en) Digital billboard playing system, instant monitoring system and instant monitoring method thereof
CN113612938A (en) Multi-type adaptive resolution image conversion method and device
CN113038086A (en) Highway operation management monitoring system and method
CN114037646A (en) Intelligent image detection method, system, readable medium and equipment based on Internet of things
CN213213677U (en) Vehicle-mounted camera and motor vehicle video acquisition system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination