CN116828262A - Video signal splitting method and device - Google Patents
Video signal splitting method and device Download PDFInfo
- Publication number
- CN116828262A CN116828262A CN202310001791.9A CN202310001791A CN116828262A CN 116828262 A CN116828262 A CN 116828262A CN 202310001791 A CN202310001791 A CN 202310001791A CN 116828262 A CN116828262 A CN 116828262A
- Authority
- CN
- China
- Prior art keywords
- video signal
- real
- communication link
- video
- data stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000004891 communication Methods 0.000 claims abstract description 167
- 238000012545 processing Methods 0.000 claims description 26
- 238000000605 extraction Methods 0.000 claims description 12
- 230000002596 correlated effect Effects 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 abstract description 30
- 230000008447 perception Effects 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 description 11
- 238000003860 storage Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 239000000284 extract Substances 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000000875 corresponding effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The embodiment of the invention provides a video signal splitting method and device. The video signal splitting method is applied to image acquisition equipment, and the image acquisition equipment is established with a first communication link and a second communication link. In the method, when the image acquisition equipment senses that the network delay exists in the first communication link, video detail data in the real-time video signal data stream can be extracted when the video transmission is possibly unstable, the video detail data is sent to a video receiving end, namely the rear end, through the second communication link, and the extracted real-time video signal data stream occupies smaller bandwidth when being transmitted, has higher transmission speed and can be smoothly transmitted through the first communication link. Therefore, even if the network has delay, the image acquisition equipment can adapt to the network delay and transmit real-time video data to the rear end, so that the stability of video signal data transmission can be ensured when the network delay is higher, and the rear end can have good perception experience.
Description
Technical Field
The invention relates to the field of data processing, in particular to a video signal splitting method and device.
Background
With the development of 5G, big data, cloud platforms and autopilot technologies, autopilot technologies have tended to mature. Currently, many cities have laid out an autopilot demonstration travel route and developed point-to-point autopilot network taxi service. With the continuous development of artificial intelligence technology, big data and other technologies, the barrier of unmanned technology is broken through continuously, and unmanned technology is gradually applied. In order to ensure the safety of unmanned operation, the video stream acquired in the unmanned operation process needs to be transmitted to the rear end in real time, so that the rear end can sense the running state of the vehicle in real time, and the vehicle is controlled when necessary to ensure the safety.
However, the current video transmission technology is not reliable enough, so that the video stream is unstable when the back end receives the video stream, and the perceived experience of the back end is not good.
Disclosure of Invention
Therefore, the present invention aims to provide a video signal splitting method and apparatus, which can ensure stability of video signal data transmission and ensure good perceived experience at the back end even when network delay is high.
In order to achieve the above object, the technical scheme adopted by the embodiment of the invention is as follows:
In a first aspect, the present invention provides a video signal splitting method applied to an image acquisition apparatus, where the image acquisition apparatus establishes a first communication link and a second communication link. The video signal splitting method of the first aspect includes: acquiring network delay of the first communication link and a real-time video signal data stream; when the network delay of the first communication link is larger than a first preset value, extracting video detail data in the real-time video signal data stream to obtain a real-time video signal data stream after detail extraction; transmitting the real-time video signal data stream after the details are extracted to a video receiving end through the first communication link; transmitting the video detail data to the video receiving end through the second communication link; transmitting video merging indication information to the video receiving end through the second communication link; the video merging indication information is used for indicating: and recovering the real-time video signal data stream after extracting details according to the video detail data and playing the recovered real-time video signal data stream.
In an alternative embodiment of the present invention, the step of extracting video detail data in the real-time video signal data stream when the network delay of the first communication link is greater than a first preset value includes: extracting N pixel information from a key frame of the real-time video signal data stream if the network delay of the first communication link is greater than the first preset value; wherein N is an integer greater than 1; the N pieces of pixel information are determined as the video detail data.
In an alternative embodiment of the invention N is positively correlated with the network delay of said first communication link.
In an alternative embodiment of the invention, N is determined by the following formula:the method comprises the steps of carrying out a first treatment on the surface of the Where k is a constant and PING is the network delay of the first communication link.
In an alternative embodiment of the present invention, the step of extracting N pixel information in a key frame of the real-time video signal data stream includes: detecting the type of the real-time video according to the real-time video signal data stream; when the type of the real-time video is character video, extracting N pieces of pixel information except characters from key frames of the real-time video signal data stream; and when the type of the real-time video is the object video, extracting N pieces of pixel information except for the object in the center of the picture from the key frame of the real-time video signal data stream.
In an optional embodiment of the invention, the video signal splitting method according to the first aspect further comprises: and when the type of the real-time video is the type except the character video and the object video, uniformly extracting N pixel information in the key frame of the real-time video signal data stream.
In an optional embodiment of the invention, the video signal splitting method according to the first aspect further comprises: acquiring network delay of the second communication link; when the network delay of the second communication link is larger than a second preset value, sending low-image-quality playing information to the video receiving end through the first communication link; wherein the low-quality playing information is used for indicating: and playing according to the real-time video signal data stream after the details are extracted.
In an optional embodiment of the present invention, the image acquisition device is disposed on an unmanned automobile, the image acquisition device establishes the second communication link with a router on the unmanned automobile that provides Wi-Fi signals, and the router is further connected with a terminal device of a user; the video signal splitting method according to the first aspect further includes: when the network delay of the second communication link is larger than a second preset value, sending speed limit indication information to the router; the speed limit indication information is used for indicating: reducing the communication rate with the user's terminal device.
In a second aspect, the present invention provides a video signal splitting apparatus applied to an image capturing device, where the image capturing device establishes a first communication link and a second communication link. The video signal splitting device of the second aspect comprises a transceiver module and a processing module. The receiving and transmitting module is used for acquiring network delay of the first communication link and real-time video signal data flow; the processing module is configured to extract video detail data in the real-time video signal data stream when the network delay of the first communication link is greater than a first preset value, so as to obtain a real-time video signal data stream after the details are extracted; the receiving-transmitting module is further configured to send the real-time video signal data stream after the details are extracted to a video receiving end through the first communication link; the transceiver module is further configured to send the video detail data to the video receiving end through the second communication link; the transceiver module is further configured to send video merging indication information to the video receiving end through the second communication link; the video merging indication information is used for indicating: and recovering the real-time video signal data stream after extracting details according to the video detail data and playing the recovered real-time video signal data stream.
In an alternative embodiment of the present invention, the processing module is further configured to extract N pixel information in a key frame of the real-time video signal data stream if the network delay of the first communication link is greater than the first preset value; wherein N is an integer greater than 1; the processing module is further configured to determine the N pixel information as the video detail data.
In an alternative embodiment of the invention N is positively correlated with the network delay of said first communication link.
In an alternative embodiment of the invention, N is determined by the following formula:the method comprises the steps of carrying out a first treatment on the surface of the Where k is a constant and PING is the network delay of the first communication link.
In an alternative embodiment of the present invention, the processing module is further configured to detect a type of real-time video according to the real-time video signal data stream; the processing module is further configured to extract N pieces of pixel information other than the person in a key frame of the real-time video signal data stream when the type of the real-time video is a person video; and the processing module is also used for extracting N pixel information except for the object in the center of the picture from the key frame of the real-time video signal data stream when the type of the real-time video is the object video.
In an optional embodiment of the present invention, the processing module is further configured to uniformly extract N pixel information in a key frame of the real-time video signal data stream when the type of the real-time video is a type other than the character video and the object video.
In an alternative embodiment of the present invention, the processing module is further configured to obtain a network delay of the second communication link; the transceiver module is further configured to send low-quality playing information to the video receiving end through the first communication link when the network delay of the second communication link is greater than a second preset value; wherein the low-quality playing information is used for indicating: and playing according to the real-time video signal data stream after the details are extracted.
In an optional embodiment of the present invention, the image acquisition device is disposed on an unmanned automobile, the image acquisition device establishes the second communication link with a router on the unmanned automobile that provides Wi-Fi signals, and the router is further connected with a terminal device of a user; the receiving-transmitting module is further configured to send speed limit indication information to the router when the network delay of the second communication link is greater than a second preset value; the speed limit indication information is used for indicating: reducing the communication rate with the user's terminal device.
In a third aspect, the present application provides an image acquisition apparatus, comprising a processor and a memory, the memory storing machine-readable instructions, the processor being configured to execute the machine-readable instructions to implement the video signal splitting method according to any of the alternative embodiments of the first aspect.
In a fourth aspect, the present application provides a computer readable storage medium comprising a computer program, which when run on a computer implements the video signal splitting method according to any of the alternative embodiments of the first aspect.
It will be appreciated that, based on the embodiments provided in the above aspects, the image capturing device establishes a first communication link and a second communication link, and then the image capturing device passes: acquiring network delay of a first communication link and a real-time video signal data stream; when the network delay of the first communication link is larger than a first preset value, extracting video detail data in the real-time video signal data stream to obtain the real-time video signal data stream after detail extraction; transmitting the real-time video signal data stream after the details are extracted to a video receiving end through a first communication link; transmitting video detail data to a video receiving end through a second communication link; transmitting video merging indication information to a video receiving end through a second communication link; the video merging indication information is used for indicating: and recovering the real-time video signal data stream after extracting details according to the video detail data and playing the video signal data stream. In other words, in the scheme provided by the embodiment of the application, when the image acquisition device senses that the network delay exists in the first communication link, the image acquisition device can extract video detail data in the real-time video signal data stream and send the video detail data to the video receiving end, namely the rear end, through the second communication link, and the extracted real-time video signal data stream occupies smaller bandwidth during transmission, has faster transmission speed and can be smoothly transmitted through the first communication link. Therefore, even if the network has delay, the image acquisition equipment can adapt to the network delay and transmit the real-time video data to the video receiving end, and when the network delay is higher, the embodiment of the application can also ensure the stability of video signal data transmission and ensure that the rear end can have good perception experience. In addition, the video receiving end can restore and play the real-time video signal data stream after extracting details according to the video detail data, so that the scheme provided by the application can also ensure the definition degree of the real-time video while improving the transmission smoothness of the real-time video.
In order to make the above objects, features and advantages of the present invention more comprehensible, embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments of the present invention will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and should not be considered as limiting the scope, and other related drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a block diagram of an image capturing apparatus according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a video signal splitting method according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an application scenario of a video signal splitting method according to an embodiment of the present invention;
fig. 4 is a functional block diagram of a video signal splitting apparatus according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
Furthermore, the terms "first," "second," and the like, if any, are used merely for distinguishing between descriptions and not for indicating or implying a relative importance.
It should be noted that the features of the embodiments of the present invention may be combined with each other without conflict.
In order to solve the problems in the prior art, the embodiment of the invention provides a technical scheme, which comprises a video signal splitting method and a video signal splitting device. In this scheme, the image acquisition device establishes a first communication link and a second communication link. When sensing that the network delay exists in the first communication link, the image acquisition equipment can extract video detail data in the real-time video signal data stream and send the video detail data to the video receiving end, namely the rear end, through the second communication link, and the real-time video signal data stream after the details are extracted can be smoothly transmitted through the first communication link due to the fact that the bandwidth occupied by the real-time video signal data stream after the details are extracted is smaller during transmission, the transmission speed is higher, and the video transmission is unstable. Therefore, even if the network has delay, the image acquisition equipment can adapt to the network delay and transmit the real-time video data to the video receiving end, and when the network delay is higher, the embodiment of the invention can also ensure the stability of video signal data transmission and ensure that the rear end can have good perception experience. In addition, the video receiving end can restore and play the real-time video signal data stream after extracting details according to the video detail data, so that the scheme provided by the invention can also ensure the definition degree of the real-time video while improving the transmission smoothness of the real-time video.
First, the embodiment of the invention provides an image acquisition device. Referring to fig. 1, a block diagram of an image capturing device according to an embodiment of the present invention is shown. The image capturing apparatus 100 includes: the memory 110, the processor 120, the memory 110, the processor 120 may be electrically connected directly or indirectly to the communication interface 130 to enable transmission and interaction of data. For example, the components may be electrically connected to each other via buses and/or signal lines.
In one embodiment, the image capture device 100 is a camera mounted with a mobile network communication chip (e.g., 4G communication chip/5G communication chip) and a Wi-Fi communication chip. The image capturing device 100 may be installed on an unmanned vehicle, and is configured to capture real-time video around the unmanned vehicle, and transmit a real-time video signal data stream including the real-time video to a video receiving end, that is, a back end, through a network. The backend (also referred to as a background server) may be used to collect data of the unmanned vehicle. Such as a real-time video signal data stream in the present scheme.
Processor 120 may process information and/or data related to the video signal splitting method to perform one or more of the functions described herein. For example, the processor 120 may: acquiring network delay of a first communication link and a real-time video signal data stream; when the network delay of the first communication link is larger than a first preset value, extracting video detail data in the real-time video signal data stream to obtain the real-time video signal data stream after detail extraction; transmitting the real-time video signal data stream after the details are extracted to a video receiving end through a first communication link; transmitting video detail data to a video receiving end through a second communication link; transmitting video merging indication information to a video receiving end through a second communication link; the video merging indication information is used for indicating: and recovering the real-time video signal data stream after extracting details according to the video detail data and playing the video signal data stream. In this way, the image capturing device 100 can ensure stability of video signal data transmission when the network delay is high, and ensure that the back end has good sensing experience.
The memory 110 may include, but is not limited to: solid State Disk (SSD), mechanical Hard Disk (Hard Disk Drive), read Only Memory (ROM), programmable Read Only Memory (Programmable Read-Only Memory, PROM), erasable Read Only Memory (Erasable Programmable Read-Only Memory, EPROM), random access Memory (Random AccessMemory, RAM), electrically erasable Read Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM), and the like.
The processor 120 described above may include, but is not limited to: a central processor (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but are also not limited to: an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a Digital signal processor (Digital SignalProcessing, DSP), a Field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. Thus, the processor 120 may be an integrated circuit chip with signal processing capabilities.
It is to be understood that the configuration of the image capturing device 100 shown in fig. 1 is only one schematic configuration, and that the image capturing device 100 may also include more or fewer components or modules than the configuration shown in fig. 1, or have a different configuration or construction than the configuration shown in fig. 1. Also, the components shown in FIG. 1 may be implemented in hardware, software, or a combination of both.
In addition, it should be further understood that the image capturing device 100 provided by the present invention may be configured or constructed differently according to the needs of the actual application. For example, the image capturing device 100 provided by the present invention may be an electronic device having communication, computing and storage functions, such as a mobile phone, a portable smart device, etc.
When the image capturing apparatus 100 provided by the present invention is an electronic apparatus having communication, calculation, and storage functions, these electronic apparatuses may also: acquiring network delay of a first communication link and a real-time video signal data stream; when the network delay of the first communication link is larger than a first preset value, extracting video detail data in the real-time video signal data stream to obtain the real-time video signal data stream after detail extraction; transmitting the real-time video signal data stream after the details are extracted to a video receiving end through a first communication link; transmitting video detail data to a video receiving end through a second communication link; transmitting video merging indication information to a video receiving end through a second communication link; the video merging indication information is used for indicating the real-time video signal data stream after the details are extracted to be played after the video detail data are restored. Therefore, the image acquisition device 100 can also ensure the stability of video signal data transmission when the network delay is high, and ensure that the rear end can have good perception experience.
In the following, for ease of understanding, the following embodiments of the present invention will take the image capturing apparatus 100 shown in fig. 1 as an example, and the video signal splitting method provided by the embodiments of the present invention will be described with reference to the accompanying drawings.
Referring to fig. 2, fig. 2 is a flowchart illustrating a video signal splitting method according to an embodiment of the invention. The video signal splitting method may be applied to the image capturing apparatus 100 described above, where the image capturing apparatus 100 may establish a first communication link and a second communication link, and the image capturing apparatus 100 may transmit real-time video signals to one or more video receiving ends through the first communication link and the second communication link. The video receiving end may be a background server, that is, a backend.
The video signal splitting method may include the following steps S110 to S150, which are described below.
And S110, acquiring network delay of the first communication link and real-time video signal data flow.
And S120, extracting video detail data in the real-time video signal data stream when the network delay of the first communication link is larger than a first preset value, and obtaining the real-time video signal data stream after detail extraction.
And S130, transmitting the real-time video signal data stream with the extracted details to a video receiving end through a first communication link.
And S140, sending the video detail data to a video receiving end through a second communication link.
And S150, sending video merging indication information to a video receiving end through a second communication link. The video merging indication information is used for indicating: and recovering the real-time video signal data stream after extracting details according to the video detail data and playing the video signal data stream.
The following describes the expressions in the respective steps described above.
The first communication link and the second communication link described above may be communication links established by the image capturing apparatus 100 with network apparatuses (including base stations, routers, switches, network servers, and the like).
For example, referring to the application scenario shown in fig. 3, the image capturing device 100 may establish a first communication link with a base station, and specifically, the image capturing device 100 may be communicatively connected to the base station through a fourth generation mobile communication technology (4G) or a fifth generation mobile communication technology (5G), so as to establish the first communication link. After establishing the first communication link, the image capture device 100 may transmit real-time data (e.g., a real-time video signal data stream) to the video receiving end over the first communication link so that the video receiving end can acquire the real-time video signal data stream. The video receiving end can access to a network and be connected with a network server to receive real-time data transmitted to the network by the image acquisition device 100. In addition, the image capturing device 100 may also be communicatively connected to the router through Wi-Fi, thereby establishing a second communication link. Similarly, after establishing the second communication link, the image capture device 100 may transmit real-time data (e.g., a real-time video signal data stream) to the video receiving end over the second communication link.
In one embodiment, the first communication link is a 4G or 5G communication link. The second communication link is a Wi-Fi communication link.
In S110, the network delay of the first communication link may be acquired by the real-time application installed on the image capturing apparatus 100 detecting PING values of the first communication link and the second communication link through the internet packet explorer (Packet Internet Groper, PING), where the PING values indicate whether the communication links are normally connected and the network delay of the communication links. The smaller the PING value, the smaller the network delay representing the communication link.
The real-time video signal data stream comprises real-time video.
The first preset value may be an empirical value, and may be preconfigured and modified halfway. Alternatively, the first preset value may be 150-200. It will be appreciated that a network delay of the first communication link greater than the first preset value indicates that the network delay of the first communication link is high, which may cause a packet drop or loss of the real-time video signal data stream transmitted through the first communication link, resulting in a real-time video clip, buffer or even interruption.
In an alternative embodiment, the real-time video signal data stream refers to audio/video data recorded by the camera of the image capturing device 100, that is, includes real-time video. The real-time video signal data stream can be transmitted through a network after video encoding, and can be played after video decoding at a video receiving end. The encoding mode of the real-time video signal data stream may include h.261, h.263, h.264, H265, etc., which is not limited thereto.
In the foregoing S120, when the network delay of the first communication link is greater than the first preset value, the method for extracting the video detail data in the real-time video signal data stream may include: the extraction of the key frames (i.e. I frames) in the real-time video signal data stream may specifically be performed on some pixel information in the key frames, for example, modifying the values of some pixels in the key frames to be consistent with the values of surrounding pixels, or modifying the values of some pixels in the key frames to be less than a preset threshold. In this way, the amount of information included in the key frame can be reduced, thereby reducing the amount of data occupied by the predicted frame (P frame) and the bi-directional predicted frame (B frame) generated when video encoding is performed. In other words, in S120, the bandwidth occupied by the extracted real-time video signal data stream is smaller and the transmission speed is faster, and even when the network delay of the first communication link increases due to the network fluctuation, the extracted real-time video signal data stream can be smoothly transmitted through the first communication link. In addition, compared with the mode of changing a network line and reestablishing a new communication link to transmit video data when the real-time video data transmission is blocked due to network fluctuation, the technical scheme provided by the embodiment of the invention can avoid the situation that the communication link is switched when the network is delayed, so that the back-end operation and maintenance is obviously blocked when the real-time video is watched, thereby improving the user experience.
Continuing with the example of the real-time video signal data stream by the h.264 encoding technique, S150 may further include: when receiving the extracted real-time video signal data stream, the video detail data and the video merging indication information, the video receiving end can recover the extracted pixel points in the key frame obtained by decoding by utilizing the video detail data when decoding the extracted real-time video signal data stream by the H.264 coding technology, so as to obtain the recovered real-time video signal data stream and play the recovered real-time video signal data stream. Therefore, the scheme provided by the invention can also improve the real-time fluency and ensure the definition of the real-time video.
It can be appreciated that according to the above-mentioned method embodiment, when the image capturing device 100 senses that the first communication link has a network delay and may cause unstable video transmission, the video detail data in the real-time video signal data stream can be extracted, and the video detail data is sent to the video receiving end, i.e. the back end, through the second communication link, and the real-time video signal data stream after the details are extracted can be smoothly transmitted through the first communication link due to the smaller bandwidth occupied by the extracted real-time video signal data stream during transmission, and the transmission speed is faster. Thus, even if the network has delay, the image acquisition device 100 can adapt to the network delay and transmit real-time video data to the video receiving end, so that the embodiment of the invention can ensure the stability of video signal data transmission and ensure that the rear end can have good perception experience when the network delay is higher. In addition, the video receiving end can restore and play the real-time video signal data stream after extracting details according to the video detail data, so that the scheme provided by the invention can also ensure the definition degree of the real-time video while improving the transmission smoothness of the real-time video.
In an alternative embodiment, S120, when the network delay of the first communication link is greater than a first preset value, extracts video detail data in the real-time video signal data stream, including the following steps 1.1 and 1.2:
step 1.1, extracting N pixel information in a key frame of the real-time video signal data stream in case the network delay of the first communication link is greater than a first preset value. Wherein N is an integer greater than 1. In particular, N pieces of pixel information in all or part of the key frames may be extracted. The pixel information herein may refer to pixel points in a key frame.
And step 1.2, determining N pieces of pixel information as video detail data.
In an alternative embodiment, N is positively correlated with the network delay of the first communication link. That is, when the network delay of the first communication link is larger, N is larger, so that the more pixel information is extracted in the key frame, the smaller the bandwidth occupied by the real-time video signal data stream after extracting the details is, and the faster the transmission speed is. In other words, the image capturing apparatus 100 may adaptively adjust the size of the real-time video signal data stream after extracting the details according to the size of the network delay, so as to adapt to the transmission under different network delays, so that the real-time video signal data stream after extracting the details can be smoothly transmitted under various network environments.
In an alternative embodiment of the invention, N is determined by the following formula:. Where k is a constant and PING is the network delay of the first communication link. It will be appreciated that in the above formula, as PING increases, N becomes larger and exhibits an S-type increase. Therefore, when the video receiving end does not successfully receive the video detail data and directly plays the video detail data according to the real-time video signal data stream after the detail extraction, the extraction degree of the real-time video signal data stream can be increased along with the network delay, so that the real-time video signal data stream can be ensured to be smoothly transmitted in various network environments, and the real-time video signal data stream after the detail extraction can be as clear as possible, and the user experience is improved.
In an alternative embodiment of the present invention, the step 1.1 of extracting N pixel information in a key frame of a real-time video signal data stream includes the following steps 1.11, 1.12 and 1.13:
step 1.11, detecting the type of real-time video included in the real-time video signal data stream according to the real-time video signal data stream. For example, the image capture device 100 may utilize a trained neural network model to identify whether people and items are present in key frames of a real-time video signal data stream. When people exist in the key frames of the real-time video signal data stream, the type of the real-time video can be determined to be the man-made video. When no person is present in the key frames of the real-time video signal data stream and an item is present, then the type of real-time video may be determined to be an item video, which may also be referred to as a video that does not include a person.
And step 1.12, when the type of the real-time video is character video, extracting N pieces of pixel information except the character from the key frame of the real-time video signal data stream. For example, N pieces of pixel information in a pixel region other than the pixel region occupied by the distance person in the key frame are extracted.
And step 1.13, when the type of the real-time video is the object video, extracting N pieces of pixel information except for the object in the center of the picture from the key frame of the real-time video signal data stream. For example, N pieces of pixel information in a pixel region other than the pixel region occupied by an item located at the center of the screen in the key frame are extracted.
It can be understood that in the above steps 1.11-1.13, by identifying the type of the real-time video, the real-time video signal data streams of different real-time video types are extracted in a targeted manner, so that the pixel information of the operation and maintenance attention area is reserved as much as possible (for example, the pixel information of the person is reserved in the pixel information of the person except the person for extracting the person video), and when the video receiving end does not successfully receive the video detail data and directly plays according to the real-time video signal data stream after the detail extraction, the real-time video signal data stream after the detail extraction is as clear as possible, thereby improving the user experience.
In an alternative embodiment of the present invention, the video signal splitting method shown in fig. 2 further includes the steps of: when the type of the real-time video is other than the type of the human video and the article video, N pieces of pixel information are uniformly extracted from the key frames of the real-time video signal data stream.
In an alternative embodiment of the present invention, the video signal splitting method shown in fig. 2 further includes the following steps 2.1 to 2.2:
step 2.1, obtaining network delay of a second communication link;
and 2.2, when the network delay of the second communication link is larger than a second preset value, sending low-image-quality playing information to the video receiving end through the first communication link. Wherein the low-quality playing information is used for indicating: and playing according to the real-time video signal data stream after the details are extracted.
It can be understood that, in step 2.1 and step 2.2, when the network delay of the second communication link for transmitting the video detail data is also large, which may cause that the video receiving end cannot receive the video detail data and cannot clearly recover the real-time video signal data stream, the image capturing device 100 may further instruct the video receiving end to play directly according to the extracted real-time video signal data stream through the low-image-quality playing information, so that although a certain image quality loss is caused, smooth playing of the video can be ensured, and user experience is ensured.
In an alternative embodiment of the present invention, the image capturing device 100 is disposed on an unmanned vehicle, the image capturing device 100 establishes the second communication link with a router that provides Wi-Fi signals on the unmanned vehicle, and the router is further connected with a terminal device (e.g., a mobile phone, a notebook computer, etc.) of the user. The video signal splitting method shown in fig. 2 further includes: and when the network delay of the second communication link is larger than a second preset value, sending speed limit indication information to the router. Wherein, the speed limit indication information is used for indicating: reducing the communication rate with the user's terminal device.
It will be appreciated that when capturing real-time video with the image capturing device 100, there is often also a user's terminal device around the image capturing device 100 accessing a router to enable network communication. In other words, the image capturing apparatus 100 may have a network competition relationship with terminal apparatuses of surrounding users. By sending the speed limit indication information to the router to instruct the router to reduce the communication rate with the terminal device of the user, more transmission bandwidth can be reserved for the second communication link (i.e., wi-Fi link) of the image capturing device 100, so that the smoothness of video transmission can be further ensured.
In order to perform the corresponding steps in the foregoing embodiments and the various possible manners, an implementation manner of a video signal splitting apparatus is given below, and referring to fig. 4, fig. 4 is a functional block diagram of a video signal splitting apparatus 300 according to an embodiment of the present invention. The video signal splitting apparatus 300 may be applied to the image capturing device 100 shown in fig. 1 and may be used to perform the steps that can be performed by the image capturing device 100 in the method embodiment. It should be noted that, the basic principle and the technical effects of the video signal splitting apparatus 300 provided in this embodiment are the same as those of the above embodiment, and for brevity, the corresponding contents of the above embodiment may be standardized. The video signal splitting apparatus 300 may include: a transceiver module 310 and a processing module 320.
Alternatively, the above modules may be stored in a memory in the form of software or Firmware (Firmware) or cured in an Operating System (OS) of the image capturing apparatus 100 shown in fig. 1 provided in the present invention, and may be executed by a processor in the image capturing apparatus 100 shown in fig. 1. Meanwhile, data, codes of programs, and the like required to execute the above-described modules may be stored in the memory.
It will be appreciated that the transceiver module 310, the processing module 320 may be configured to support the image capturing device 100 shown in fig. 1 to perform the steps associated with the method embodiments described above, and/or other processes for the techniques described herein, such as the method embodiments shown in fig. 2 and the various method embodiments described above, without limitation.
The transceiver module 310 is configured to obtain a network delay of the first communication link and a real-time video signal data stream; the processing module 320 is configured to extract video detail data in the real-time video signal data stream when the network delay of the first communication link is greater than a first preset value, so as to obtain a real-time video signal data stream after the details are extracted; the transceiver module 310 is further configured to send the real-time video signal data stream after the details are extracted to the video receiving end through the first communication link; the transceiver module 310 is further configured to send video detail data to the video receiving end through a second communication link; the transceiver module 310 is further configured to send video merging indication information to the video receiving end through a second communication link; the video merging indication information is used for indicating: and recovering the real-time video signal data stream after extracting details according to the video detail data and playing the video signal data stream.
In an alternative embodiment of the present invention, the processing module 320 is further configured to extract N pixel information from a key frame of the real-time video signal data stream if the network delay of the first communication link is greater than a first preset value; wherein N is an integer greater than 1; the processing module 320 is further configured to determine the N pieces of pixel information as video detail data.
In an alternative embodiment of the invention, N is positively correlated with the network delay of the first communication link.
In an alternative embodiment of the invention, N is determined by the following formula:the method comprises the steps of carrying out a first treatment on the surface of the Where k is a constant and PING is the network delay of the first communication link.
In an alternative embodiment of the present invention, the processing module 320 is further configured to detect a type of real-time video according to the real-time video signal data stream; the processing module 320 is further configured to extract N pieces of pixel information other than the person in the key frame of the real-time video signal data stream when the type of the real-time video is the person video; the processing module 320 is further configured to extract N pieces of pixel information other than the object in the center of the picture in the key frame of the real-time video signal data stream when the type of the real-time video is the object video.
In an alternative embodiment of the present invention, the processing module 320 is further configured to uniformly extract N pixel information in a key frame of the real-time video signal data stream when the type of the real-time video is other than the type of the human video and the article video.
In an alternative embodiment of the present invention, the processing module 320 is further configured to obtain a network delay of the second communication link; the transceiver module 310 is further configured to send low-quality playback information to the video receiving end through the first communication link when the network delay of the second communication link is greater than a second preset value; wherein, the low image quality playing information is used for indicating: and playing according to the real-time video signal data stream after the details are extracted.
In an alternative embodiment of the present invention, the image capturing device 100 is disposed on an unmanned automobile, the image capturing device 100 establishes a second communication link with a router that provides Wi-Fi signals on the unmanned automobile, and the router is further connected with a terminal device of a user; the transceiver module 310 is further configured to send speed limit indication information to the router when the network delay of the second communication link is greater than a second preset value; the speed limit indication information is used for indicating: reducing the communication rate with the user's terminal device.
Based on the above method embodiments, the present invention further provides a computer readable storage medium, where a computer program is stored, the computer program executing the above video signal splitting method when executed by a processor.
Specifically, the storage medium may be a general storage medium, such as a mobile disk, a hard disk, and the like, and when a computer program on the storage medium is executed, the video signal splitting method in the above embodiment can be executed, so as to solve the problem that the current video transmission technology is not reliable enough, resulting in unstable video stream when the back end receives, and poor perceived experience of the back end, and can ensure stability of video signal data transmission and good perceived experience of the back end when the network delay is high.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
Further, the units described as separate units may or may not be physically separate, and units displayed as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Furthermore, functional modules in various embodiments of the present invention may be integrated together to form a single portion, or each module may exist alone, or two or more modules may be integrated to form a single portion.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The above description is only an example of the present invention and is not intended to limit the scope of the present invention, and various modifications and variations will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A video signal splitting method applied to an image acquisition apparatus, the image acquisition apparatus having a first communication link and a second communication link established, the video signal splitting method comprising:
acquiring network delay of the first communication link and a real-time video signal data stream;
when the network delay of the first communication link is larger than a first preset value, extracting video detail data in the real-time video signal data stream to obtain a real-time video signal data stream after detail extraction;
transmitting the real-time video signal data stream after the details are extracted to a video receiving end through the first communication link;
transmitting the video detail data to the video receiving end through the second communication link;
transmitting video merging indication information to the video receiving end through the second communication link; the video merging indication information is used for indicating: and recovering the real-time video signal data stream after extracting details according to the video detail data and playing the recovered real-time video signal data stream.
2. The video signal splitting method of claim 1, wherein the step of extracting video detail data in the real-time video signal data stream when the network delay of the first communication link is greater than a first preset value comprises:
Extracting N pixel information from a key frame of the real-time video signal data stream if the network delay of the first communication link is greater than the first preset value; wherein N is an integer greater than 1;
the N pieces of pixel information are determined as the video detail data.
3. The video signal splitting method of claim 2, wherein N is positively correlated with a network delay of the first communication link.
4. A video signal splitting method according to claim 3, wherein N is determined by the formula:
；
where k is a constant and PING is the network delay of the first communication link.
5. The video signal splitting method of claim 2, wherein the step of extracting N pixel information in a key frame of the real-time video signal data stream comprises:
detecting the type of the real-time video according to the real-time video signal data stream;
when the type of the real-time video is character video, extracting N pieces of pixel information except characters from key frames of the real-time video signal data stream;
and when the type of the real-time video is the object video, extracting N pieces of pixel information except for the object in the center of the picture from the key frame of the real-time video signal data stream.
6. The video signal splitting method of claim 5, further comprising:
and when the type of the real-time video is the type except the character video and the object video, uniformly extracting N pixel information in the key frame of the real-time video signal data stream.
7. The video signal splitting method of claim 1, further comprising:
acquiring network delay of the second communication link;
when the network delay of the second communication link is larger than a second preset value, sending low-image-quality playing information to the video receiving end through the first communication link; wherein the low-quality playing information is used for indicating: and playing according to the real-time video signal data stream after the details are extracted.
8. The video signal splitting method according to claim 7, wherein the image acquisition device is disposed on an unmanned vehicle, the image acquisition device establishes the second communication link with a router on the unmanned vehicle that provides Wi-Fi signals, the router further being connected with a terminal device of a user;
wherein the method further comprises:
When the network delay of the second communication link is larger than the second preset value, sending speed limit indication information to the router; the speed limit indication information is used for indicating: reducing the communication rate with the user's terminal device.
9. The video signal splitting device is characterized by being applied to image acquisition equipment, wherein the image acquisition equipment is provided with a first communication link and a second communication link; the video signal splitting device comprises a receiving and transmitting module and a processing module; wherein,,
the receiving and transmitting module is used for acquiring network delay of the first communication link and real-time video signal data flow;
the processing module is configured to extract video detail data in the real-time video signal data stream when the network delay of the first communication link is greater than a first preset value, so as to obtain a real-time video signal data stream after the details are extracted;
the receiving-transmitting module is further configured to send the real-time video signal data stream after the details are extracted to a video receiving end through the first communication link;
the transceiver module is further configured to send the video detail data to the video receiving end through the second communication link;
The transceiver module is further configured to send video merging indication information to the video receiving end through the second communication link; the video merging indication information is used for indicating: and recovering the real-time video signal data stream after extracting details according to the video detail data and playing the recovered real-time video signal data stream.
10. The video signal splitting device of claim 9, wherein the processing module is further configured to extract N pixel information in a key frame of the real-time video signal data stream if the network delay of the first communication link is greater than the first preset value; wherein N is an integer greater than 1;
the processing module is further configured to determine the N pixel information as the video detail data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310001791.9A CN116828262B (en) | 2023-01-03 | 2023-01-03 | Video signal splitting method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310001791.9A CN116828262B (en) | 2023-01-03 | 2023-01-03 | Video signal splitting method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116828262A true CN116828262A (en) | 2023-09-29 |
| CN116828262B CN116828262B (en) | 2024-04-19 |
Family
ID=88126380
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310001791.9A Active CN116828262B (en) | 2023-01-03 | 2023-01-03 | Video signal splitting method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116828262B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117880605A (en) * | 2024-03-12 | 2024-04-12 | 深圳市诚立业科技发展有限公司 | Short message video quick playing method and system based on 5G technology |
| CN118118709A (en) * | 2024-01-18 | 2024-05-31 | 常州恒安智能科技有限公司 | Digital video information transmission method based on IP network |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5983263A (en) * | 1998-01-02 | 1999-11-09 | Intel Corporation | Method and apparatus for transmitting images during a multimedia teleconference |
| WO2007109091A2 (en) * | 2006-03-16 | 2007-09-27 | Smartdrive Systems, Inc. | Vehicle event recorded systems and networks having parallel communication links |
| CN103580773A (en) * | 2012-07-18 | 2014-02-12 | 中兴通讯股份有限公司 | Method and device for transmitting data frame |
| CN104796731A (en) * | 2015-04-16 | 2015-07-22 | 无锡天脉聚源传媒科技有限公司 | Video processing method and device |
| CN105392020A (en) * | 2015-11-19 | 2016-03-09 | 广州华多网络科技有限公司 | Internet video direct broadcasting method and system |
| CN111935201A (en) * | 2019-05-13 | 2020-11-13 | 华为技术有限公司 | In-vehicle communication system, in-vehicle communication method and device |
-
2023
- 2023-01-03 CN CN202310001791.9A patent/CN116828262B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5983263A (en) * | 1998-01-02 | 1999-11-09 | Intel Corporation | Method and apparatus for transmitting images during a multimedia teleconference |
| WO2007109091A2 (en) * | 2006-03-16 | 2007-09-27 | Smartdrive Systems, Inc. | Vehicle event recorded systems and networks having parallel communication links |
| CN103580773A (en) * | 2012-07-18 | 2014-02-12 | 中兴通讯股份有限公司 | Method and device for transmitting data frame |
| CN104796731A (en) * | 2015-04-16 | 2015-07-22 | 无锡天脉聚源传媒科技有限公司 | Video processing method and device |
| CN105392020A (en) * | 2015-11-19 | 2016-03-09 | 广州华多网络科技有限公司 | Internet video direct broadcasting method and system |
| CN111935201A (en) * | 2019-05-13 | 2020-11-13 | 华为技术有限公司 | In-vehicle communication system, in-vehicle communication method and device |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118118709A (en) * | 2024-01-18 | 2024-05-31 | 常州恒安智能科技有限公司 | Digital video information transmission method based on IP network |
| CN117880605A (en) * | 2024-03-12 | 2024-04-12 | 深圳市诚立业科技发展有限公司 | Short message video quick playing method and system based on 5G technology |
| CN117880605B (en) * | 2024-03-12 | 2024-05-14 | 深圳市诚立业科技发展有限公司 | Short message video quick playing method and system based on 5G technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116828262B (en) | 2024-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116828262B (en) | Video signal splitting method and device | |
| CN110149555B (en) | Video processing method and video receiving device | |
| CN103618911B (en) | Video streaming providing method and device based on video attribute information | |
| CN102984495A (en) | Video image processing method and device | |
| CN110324721B (en) | Video data processing method and device and storage medium | |
| CN111083536B (en) | Method and device for adjusting video code rate | |
| CN111263243B (en) | Video coding method and device, computer readable medium and electronic equipment | |
| CN111970565A (en) | Video data processing method and device, electronic equipment and storage medium | |
| CN111031032A (en) | Cloud video transcoding method and device, decoding method and device, and electronic device | |
| CN106341698B (en) | Video live broadcast processing method and device, storage medium and terminal equipment | |
| CN112492324A (en) | Data processing method and system | |
| KR20230144620A (en) | Point cloud encoding and decoding method, point cloud encoding and decoding device, computer-readable medium, and electronic device | |
| CN113794903A (en) | Video image processing method and device and server | |
| CN103929682B (en) | Method and device for setting key frames in video live broadcast system | |
| CN103780325A (en) | Satellite transmission monitoring system | |
| CN111758262A (en) | Video transmission method and device, aircraft, playing equipment and storage medium | |
| CN114501039B (en) | Ultra-low-time-delay video data transmission method and device | |
| US20230058707A1 (en) | Methods, systems, and media for selecting video formats for adaptive video streaming | |
| CN110798688A (en) | High-definition video compression coding system based on real-time transmission | |
| CN102883213A (en) | Subtitle extraction method and device | |
| CN113996056A (en) | Data sending and receiving method of cloud game and related equipment | |
| CN110401835B (en) | Image processing method and device | |
| CN113014940A (en) | Live broadcast optimization method and device | |
| CN112104872A (en) | Image transmission method and device | |
| CN111432271A (en) | Multi-screen interaction method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |