CN115988258B - Video communication method, storage medium and system based on internet of things (IoT) device - Google Patents
Video communication method, storage medium and system based on internet of things (IoT) device Download PDFInfo
- Publication number
- CN115988258B CN115988258B CN202310256987.2A CN202310256987A CN115988258B CN 115988258 B CN115988258 B CN 115988258B CN 202310256987 A CN202310256987 A CN 202310256987A CN 115988258 B CN115988258 B CN 115988258B
- Authority
- CN
- China
- Prior art keywords
- video
- mixed
- stream
- iot device
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention provides a video communication method, a storage medium and a system based on an internet of things (IoT) device, which are used for enabling the IoT device to communicate with other devices, at least two paths of video streams to be mixed from the other devices are mixed to generate a mixed stream video of picture-in-picture or parallel pictures, one mixed stream video generated by the IoT device through the method comprises a plurality of video stream contents to be mixed, which is equivalent to one-to-many video communication in a one-to-one video communication mode, the video communication mode can be supported by the IoT device and can be used for scenes of audio and video communication of the IoT platform, the problem of insufficient functions of the audio and video communication of the current IoT platform is solved, the scenes of audio and video communication of the IoT platform are expanded, the experience and the practicability of audio and video communication on the IoT device are improved, and the audio and video communication of the IoT device is enabled.
Description
Technical Field
The present invention relates to the field of video communication technologies, and in particular, to a video communication method, a storage medium, and a system based on an IoT platform.
Background
The IoT (Internet of Things ) refers to a network that utilizes information carriers such as the internet and a traditional telecommunication network to enable all devices capable of performing independent functions to achieve interconnection and intercommunication, and the IoT device is combined with the internet according to a agreed protocol through information sensors such as radio frequency identification, infrared sensors, a global positioning system and a laser scanner to enable the IoT device to exchange and communicate information, so as to achieve intelligent identification, positioning, tracking, monitoring and management of the IoT device. Based on the information exchange and communication principles of IoT devices, multiple IoT devices that communicate with each other can be used to realize video, but current IoT devices generally only support one-to-one video communication, and it is difficult to support one-to-many video communication due to the problem that IoT devices generally have insufficient hardware performance such as CPU and memory.
Disclosure of Invention
The technical problem to be solved by the invention is how to realize one-to-many video by utilizing an IoT device.
In order to solve the above technical problems, the present invention provides a video communication method based on an IoT device, which is used for the IoT device to communicate with other devices, comprising the following steps:
A. acquiring video streams from various other devices;
B. acquiring at least two paths of video streams to be mixed selected from all video streams by a user;
C. decoding the at least two paths of video streams to be mixed to generate at least two items of YUV format data;
D. acquiring a mixed flow mode selected by a user;
E. analyzing according to YUV format data of each path of video streams to be mixed and a mixed stream mode selected by a user to obtain the size of a buffer space required for merging and storing video images of at least two paths of video streams to be mixed, and generating a buffer for storing the video images according to the size;
F. copying and storing the at least two video streams to be mixed into the buffer memory in sequence according to the memory distribution characteristics of YUV format data, and forming buffer memory data according to a mixed stream mode selected by a user;
G. and encoding the cached data to generate mixed stream video.
Preferably, in the step B, a selection order of the at least two video streams to be mixed selected by the user is obtained, and in the step F, the at least two video streams to be mixed are copied and stored in the buffer memory according to the selection order.
Preferably, in the step C, the mixed flow mode includes a picture-in-picture mode or a parallel picture mode.
Preferably, in the step E, if the mixed stream mode selected by the user is a parallel picture mode, the size of the buffer space required for merging and storing the video images of the at least two video streams to be mixed is the sum of the sizes of the video images of the at least two video streams to be mixed.
Preferably, in the step E, if the mixed stream mode selected by the user is a picture-in-picture mode, the size of the buffer space required for merging and storing the video images of the at least two paths of video streams to be mixed is the size of the buffer space required for the video images of the video streams to be mixed selected by the user as the main picture.
Preferably, the user selects the to-be-mixed stream video stream as the main picture, specifically, the to-be-mixed stream video stream selected by the user in the step B.
The present invention also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps in a video communication method as described above.
The present invention also provides an IoT device-based video communication system comprising an IoT device and at least two other devices, the IoT device comprising a computer-readable storage medium and a processor that are interconnected, the computer-readable storage medium being as described above.
The invention has the following beneficial effects: when an IoT device is used to perform one-to-many video to other devices, the size of a buffer space required for merging and storing at least two paths of video streams to be mixed is obtained by analyzing according to pixel data of the at least two paths of video streams to be mixed of the other devices and a mixed stream mode selected by a user, a buffer for storing video images is generated accordingly, then at least two items of YUV format data are copied and stored in the buffer in sequence, buffer data are formed according to a mixed stream mode selected by the user, and finally the buffer data are encoded to generate a mixed stream video, so that the mixed stream video contains a plurality of video stream contents to be mixed, which is equivalent to one-to-many video communication in a one-to-one video communication mode, and the video communication mode can be supported by the IoT device.
Drawings
Fig. 1 is a flow diagram of a video communication method based on IoT devices.
Detailed Description
The invention is further described in detail below in connection with the detailed description.
The present embodiment provides an IoT device-based video communication system, which includes an IoT device and at least two other devices, the IoT device including a computer-readable storage medium and a processor connected to each other, the computer-readable storage medium having a computer program stored thereon, the computer program when executed by the processor implementing an IoT device-based video communication method as shown in fig. 1 for one-to-many communication between the IoT device and the other devices (e.g., a mobile terminal with audio-video software, a web terminal with an audio-video system), comprising the following step A, B, C, D, E, F, G.
A. Video streams from various other devices are acquired.
Taking the example that the system comprises one IoT device and four other devices, the four other devices are a first device, a second device, a third device and a fourth device, the IoT device and the four other devices realize networking communication through a server. When an IoT device is in video communication with four other devices, the system obtains the video streams of the four other devices and uploads the video streams to the server after encoding.
B. And acquiring at least two paths of video streams to be mixed selected by a user from the video streams.
In this embodiment, a user performs one-to-two video communication with an IoT device to a second device and a third device, before the video communication, the IoT device obtains video streams of each other device from a server, and displays video stream options of the other devices on the IoT device, the user selects the video stream of the second device from the video streams, then selects the video stream of the third device, and then uses the two video streams as video streams of the to-be-mixed streams, and then the system obtains the two video streams of the to-be-mixed streams selected by the user, wherein pixel data of the video streams of the to-be-mixed streams of the second device and the third device are 640×480.
C. And respectively decoding at least two paths of video streams to be mixed to generate at least two items of YUV format data.
After the user selects the video stream of the second device and the video stream of the third device as the video streams to be mixed by using the IoT device, the system decodes the two paths of video streams to be mixed respectively to generate two items of YUV format data, specifically YUV420P data. It should be noted that the YUV format data is divided into three components Y, U, V, and a color space is defined according to one Luminance (Y component) and two chromaticities (UV component), where "Y" represents a brightness (luminence or Luma), that is, a gray value, and "U" and "V" represent chromaticities (chromance or Chroma), which serve to describe an image color and saturation for specifying a color of a pixel. In YUV420P data, YUV is based on standard 4:2:0 sampling, with every four Y components sharing a set of UV components. The pixel data of the video streams to be mixed of the second device and the third device are 640×480 based on the YUV format data.
D. And acquiring a mixed flow mode selected by a user.
In this embodiment, the system provides two mixed stream modes for the user to select, that is, a parallel picture mode and a picture-in-picture mode, and after the user selects the video stream of the second device and the video stream of the third device as the video stream to be mixed, the user can select one of the mixed stream modes to perform video mixed stream.
E. According to YUV format data of each path of video streams to be mixed and a mixed stream mode selected by a user, analyzing to obtain the size of a buffer space required for merging and storing video images of at least two paths of video streams to be mixed, and generating a buffer for storing the video images according to the size.
Taking a mixed stream mode selected by a user as an example, analyzing and obtaining the size of a buffer space required for merging video images of the video stream to be mixed of the second device and the video stream to be mixed of the third device according to YUV format data of the video stream to be mixed of the second device and YUV format data of the video stream to be mixed of the third device and the mixed stream mode of the parallel picture, and specifically the sum of the sizes of the video images of the two video streams to be mixed, because the pixel data of the video stream to be mixed of the second device and the video stream to be mixed of the third device are 640×480, the size of the buffer space required for merging and storing the video images of the two video streams to be mixed is 1280×480, thereby generating a buffer for storing the video images.
Taking a mixed stream mode selected by a user as an example, analyzing to obtain the size of a buffer space required for merging video images of the mixed stream video stream of the second device and the mixed stream video stream of the third device according to YUV format data of the mixed stream video stream of the second device and YUV format data of the mixed stream video stream of the third device and the mixed stream mode of the picture in picture, specifically the size of the buffer space required for the video images of the mixed stream video stream selected by the user as a main picture, wherein the user selects the mixed stream video stream as the main picture, specifically the mixed stream video stream selected by the user in the step B, namely the mixed stream video stream of the second device, because the pixel data of the mixed stream video stream of the second device is 640×480, the size of the buffer space required for merging the video images of the mixed stream video stream of the second device and the mixed stream video stream of the third device is 640×480, and generating a buffer for storing the video images according to the size.
F. At least two video streams to be mixed are copied and stored in a buffer memory in sequence according to the memory distribution characteristics of YUV format data, and buffer memory data are formed according to a mixed stream mode selected by a user.
Under the condition that a user selects a parallel picture mode, the system firstly selects the sequence of the video streams of the to-be-mixed stream of the second device in the step B, then selects the sequence of the video streams of the to-be-mixed stream of the third device, firstly copies the video streams of the to-be-mixed stream of the second device into a buffer memory according to the memory distribution characteristics of YUV format data, copies the video streams of the to-be-mixed stream of the third device into the buffer memory according to the memory distribution characteristics of the YUV format data, and then forms buffer data which correspond to the video images of the two paths of the video streams of the to-be-mixed stream and are distributed in parallel according to the parallel picture mode selected by the user. The video images of the to-be-mixed stream video streams of the second device and the third device are kept to be 640 multiplied by 480, so that 1280 multiplied by 480 video images of the to-be-mixed stream video streams of the second device and the third device can be generated according to the cache data.
Under the condition that a user selects a picture-in-picture mode, the system firstly selects the sequence of the video streams of the streams to be mixed of the second equipment in the step B, then selects the sequence of the video streams of the streams to be mixed of the third equipment, firstly copies and stores the video streams of the streams to be mixed of the second equipment into a buffer memory according to the memory distribution characteristics of YUV format data, copying and storing the video stream to be mixed of the third device into a buffer memory according to the memory distribution characteristics of YUV format data, and forming buffer data which is covered on the video image of the video stream to be mixed of the second device after the video image of the video stream to be mixed of the third device is reduced according to a picture-in-picture mode selected by a user. The video image of the to-be-mixed stream video stream of the second device is kept to be 640×480, and the video image of the to-be-mixed stream video stream of the third device is reduced to be 320×240, so that a 640×480 video image of which the main picture is the to-be-mixed stream video stream of the second device can be generated according to the cache data, and the 320×240 video image of the to-be-mixed stream video stream of the third device is covered on the main picture.
G. And encoding the cached data to generate the mixed stream video.
After the cached data is obtained, the cached data is encoded to generate a mixed stream video, and the mixed stream video comprises the to-be-mixed stream video stream content of the second device and the to-be-mixed stream video stream content of the third device, so that the IoT device performs one-to-two video communication in a one-to-one video communication mode, the video communication mode can be supported by the first IoT device and can be used for scenes of the audio and video communication of the IoT platform, the problem of insufficient functions of the audio and video communication of the current IoT platform is solved, the scenes of the audio and video communication of the IoT device platform are expanded, the experience and the practicability of the audio and video communication on the IoT device are improved, and the audio and video communication of the IoT device is enabled.
The above-described embodiments are provided for the present invention only and are not intended to limit the scope of patent protection. Insubstantial changes and substitutions can be made by one skilled in the art in light of the teachings of the invention, as yet fall within the scope of the claims.
Claims (8)
1. A video communication method based on an IoT device, for the IoT device to communicate with other devices, comprising the steps of:
A. acquiring video streams from various other devices;
B. acquiring at least two paths of video streams to be mixed selected from all video streams by a user;
C. decoding the at least two paths of video streams to be mixed to generate at least two items of YUV format data;
D. acquiring a mixed flow mode selected by a user;
E. analyzing according to YUV format data of the at least two paths of video streams to be mixed and a mixed stream mode selected by a user to obtain the size of a buffer space required for merging and storing video images of the at least two paths of video streams to be mixed, and generating a buffer for storing the video images according to the size;
F. copying and storing the at least two paths of video streams to be mixed into the cache according to the memory distribution characteristics of YUV format data in sequence, and forming cache data according to a mixed stream mode selected by a user;
G. and encoding the cached data to generate mixed stream video.
2. The IoT device-based video communication method according to claim 1, wherein in the step B, a selection order of the at least two streams of video to be mixed selected by a user is obtained, and in the step F, the at least two streams of video to be mixed are copied and stored in the buffer according to the selection order.
3. The IoT device-based video communication method of claim 1, wherein in step D, the mixed-flow manner comprises a picture-in-picture mode or a parallel picture mode.
4. The IoT device-based video communication method of claim 3, wherein in the step E, if the mixed stream mode selected by the user is a parallel picture mode, the size of the buffer space required for merging and storing the video images of the at least two video streams to be mixed is the sum of the sizes of the buffer spaces required for the video images of the at least two video streams to be mixed.
5. The IoT device-based video communication method according to claim 3 or 4, wherein in the step E, if the mixed stream mode selected by the user is a picture-in-picture mode, the size of the buffer space required for merging the video images storing the at least two video streams to be mixed is the size of the buffer space required for the video images of the video streams to be mixed selected by the user as the main picture.
6. The IoT device-based video communication method according to claim 5, wherein the user selects the to-be-mixed stream video stream as the main picture, in particular, the to-be-mixed stream video stream selected by the user in step B.
7. A computer readable storage medium having stored thereon a computer program, which when executed by a processor performs the steps in the video communication method according to any of claims 1 to 6.
8. An IoT device-based video communication system comprising an IoT device and at least two other devices, the IoT device comprising a computer-readable storage medium and a processor coupled to each other, wherein the computer-readable storage medium is recited in claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310256987.2A CN115988258B (en) | 2023-03-17 | 2023-03-17 | Video communication method, storage medium and system based on internet of things (IoT) device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310256987.2A CN115988258B (en) | 2023-03-17 | 2023-03-17 | Video communication method, storage medium and system based on internet of things (IoT) device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115988258A CN115988258A (en) | 2023-04-18 |
| CN115988258B true CN115988258B (en) | 2023-06-23 |
Family
ID=85968426
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310256987.2A Active CN115988258B (en) | 2023-03-17 | 2023-03-17 | Video communication method, storage medium and system based on internet of things (IoT) device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115988258B (en) |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101820524A (en) * | 2010-03-22 | 2010-09-01 | 中兴通讯股份有限公司 | Video display method for video conference |
| CN106341622B (en) * | 2015-07-06 | 2020-01-24 | 阿里巴巴集团控股有限公司 | Method and device for encoding multi-channel video stream |
| CN107707866A (en) * | 2017-10-25 | 2018-02-16 | 上海建为历保科技股份有限公司 | A kind of remote video communication method based on Internet of Things |
| CN108600682A (en) * | 2018-04-20 | 2018-09-28 | 湖南长宜物联科技有限公司 | Remote video communication method or apparatus based on Internet of Things |
| CN211352324U (en) * | 2018-12-17 | 2020-08-25 | 中音讯谷科技有限公司 | Multi-terminal video conference application system |
| CN112511858B (en) * | 2020-10-26 | 2023-04-07 | 深圳点猫科技有限公司 | Network course live broadcast mixed stream pushing method, system and storage medium |
| CN112532913A (en) * | 2020-11-30 | 2021-03-19 | 广州虎牙科技有限公司 | Video mixing method, video system and server |
| US20220286486A1 (en) * | 2021-03-02 | 2022-09-08 | Arris Enterprises Llc | Method and system for integrating internet of things (iot) devices and mobile client audiovisual into a video conferencing application |
| CN114500871B (en) * | 2021-12-15 | 2023-11-14 | 山东信通电子股份有限公司 | Multipath video analysis method, equipment and medium |
-
2023
- 2023-03-17 CN CN202310256987.2A patent/CN115988258B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115988258A (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8843559B2 (en) | Modifying payloads of digital streams for digital conferencing | |
| US9253340B2 (en) | Wireless camera with image sharing prioritization | |
| US20070024705A1 (en) | Systems and methods for video stream selection | |
| CN103259989B (en) | The display methods and device of screen content | |
| US20040086186A1 (en) | Information providing system and method, information supplying apparatus and method, recording medium, and program | |
| US20210377542A1 (en) | Video encoding and decoding method, device, and system, and storage medium | |
| US20130120592A1 (en) | Method for wireless sharing of images | |
| CN109151436B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN101640784A (en) | Device and method for controlling multi-image compounding in video conference system | |
| CN109257609B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN109274976B (en) | Data processing method and device, electronic equipment and storage medium | |
| US8917309B1 (en) | Key frame distribution in video conferencing | |
| CN111147801A (en) | Video data processing method and device for video networking terminal | |
| CN112637671A (en) | Screen sharing method and system | |
| CN113542660A (en) | Method, system and storage medium for realizing conference multi-picture high-definition display | |
| US8228980B2 (en) | Media gateway with overlay channels | |
| US20120308147A1 (en) | Image processing device, image processing method, and program | |
| CN113141352B (en) | Multimedia data transmission method and device, computer equipment and storage medium | |
| WO2021093882A1 (en) | Video meeting method, meeting terminal, server, and storage medium | |
| CN115988258B (en) | Video communication method, storage medium and system based on internet of things (IoT) device | |
| JP2005136566A (en) | Apparatus and method for converting moving picture, moving picture distribution apparatus, mail repeating device and program | |
| CN112752058A (en) | Method and device for adjusting attribute of video stream | |
| CN116546262A (en) | Data processing method, device, equipment and storage medium | |
| CN109309839B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN112738565B (en) | Interactive bandwidth optimization method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |