CN115209097A - Visual communication method, device and system - Google Patents

Visual communication method, device and system Download PDF

Info

Publication number
CN115209097A
CN115209097A CN202210659712.9A CN202210659712A CN115209097A CN 115209097 A CN115209097 A CN 115209097A CN 202210659712 A CN202210659712 A CN 202210659712A CN 115209097 A CN115209097 A CN 115209097A
Authority
CN
China
Prior art keywords
camera
video stream
network
network device
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210659712.9A
Other languages
Chinese (zh)
Inventor
陈建
刘勇军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN202210659712.9A priority Critical patent/CN115209097A/en
Publication of CN115209097A publication Critical patent/CN115209097A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network
    • H04N21/43632—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network involving a wired protocol, e.g. IEEE 1394
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network
    • H04N21/43637—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/268—Signal distribution or switching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

The application discloses a method, a device and a system for visual communication, and particularly, because first network equipment can convert received first video streams from data sent by a wireless network communication protocol into data sent by a Universal Serial Bus (USB) protocol, the first network equipment can be ensured to identify the first video streams sent by a first camera, therefore, the first network equipment can be connected with the wireless camera through a wireless network, the deployment position of the wireless camera can get rid of the limitation of a connecting line, and the flexibility of the deployment mode of the wireless camera is improved.

Description

Visual communication method, device and system
The present application is a divisional application, the original application having an application number of 201810482538.9 and a date of 2018, 5 and 18, the entire contents of which are incorporated herein by reference.
Technical Field
The present application relates to the field of communications technologies, and in particular, to a method, an apparatus, and a system for visual communication.
Background
At present, video services such as internet-based visual communication services are increasingly widely used. For example, an interface for connecting a USB camera is provided on a Set Top Box (STB). After the USB camera is connected to the set-top box through the interface, the set-top box can acquire videos shot by the USB camera. In one aspect, the set-top box may present the video to a local user via a display device. On the other hand, the set-top box may transmit the video to a remote set-top box over the internet, thereby causing the remote set-top box to present the video to a remote user via a display device. Thus, the visual communication service is realized.
Because the set-top box and the USB camera need to be connected by a USB cable, the deployment position of the camera is limited. Therefore, in the video service, the deployment mode of the USB camera is not flexible enough, and it is often difficult to meet the requirements, so that the user cannot use the video communication service conveniently.
Disclosure of Invention
The technical problem to be solved by the embodiments of the present application is to provide a method and an apparatus for visual communication, so that a set top box can acquire a shooting video through a wireless camera, and can avoid the limitation of a connecting line on the deployment position of the camera, so that the deployment mode of the camera is more flexible, and the requirements of video services in various scenes can be met, so that a user can use the visual communication services more conveniently.
In a first aspect, an embodiment of the present application provides a method for visual communication, where the method includes:
the method comprises the steps that first network equipment receives a first video stream sent by a first camera, wherein the first camera is a wireless camera connected to the first network equipment through a wireless network; the first network device converts the first video stream from data sent using a wireless network communication protocol to data sent using a Universal Serial Bus (USB) protocol; the first network device sends the converted first video stream to a first display device.
The first network equipment can convert the received video stream from the data sent by the wireless network communication protocol into the data sent by the Universal Serial Bus (USB) protocol, so that the visual communication application on the network equipment can identify the video stream data sent by the wireless camera. Therefore, the first network equipment can be connected with the USB camera based on the existing USB camera interface and can also be connected with the wireless camera through a wireless network, the deployment position of the wireless camera can not be limited by a connecting line, the flexibility of the deployment mode of the wireless camera is improved, and the number of the cameras which can be connected with the first network equipment is not limited by a hardware interface because the first network equipment can be connected with the wireless camera through the wireless network.
With reference to the first aspect, in a first possible implementation manner of the first aspect, the method further includes: the first network device identifies a plurality of cameras, including the first camera.
The interface to the USB camera provided on the set-top box is usually very limited due to cost considerations. Generally, only one USB camera interface is provided on one set-top box. Therefore, the number of cameras that can be connected simultaneously on the set-top box is limited. The first network equipment provided by the application can be connected with the first camera through a wireless network, and also can be connected with other wireless cameras or USB cameras and the like, so that the number of the cameras connected with the first network equipment is not limited by the number of hardware interfaces provided by the first network equipment.
With reference to the first aspect, in a second possible implementation manner of the first aspect, the first network device is a set top box.
With reference to the first aspect, in a third possible implementation manner of the first aspect, the method further includes: the first network equipment receives a second video stream sent by the first camera; the first network device converts the second video stream from data transmitted using a wireless network communication protocol to data transmitted using a USB protocol; the first network device sends the second video stream to the second network device. When the second network device also needs to obtain the video stream output by the first camera, the first camera may send the second video stream to the first network device, so that the first network device sends the second video stream to the second network device, and the second network device can obtain the second video stream obtained by shooting the video by the first camera.
With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, before the first network device receives the second video stream sent by the first camera, the method further includes: the first network device sends video coding parameters to the first camera, so that the first camera codes original video data shot by the first camera based on the video coding parameters to obtain the second video stream, and sends the second video stream to the first network device, wherein the video coding parameters include at least one of the following: a parameter indicating a resolution, a frame rate, a code rate, or an encoding format of the second video stream. In this way, the video coding parameters of the second video stream output by the first camera can be set.
With reference to the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, the method further includes: the first network device obtains network performance between the first network device and the second network device or network performance of the second network device, and determines the video coding parameter according to the network performance. The encoding of the wireless camera for the second video stream is dynamically adjusted according to the network performance between the first network device and the second network device or the network performance at the side of the second network device, so that when the network performance between the first network device and the second network device is poor or the network performance at the side of the second network device is poor, the data volume of the transmitted video stream is not too large, and the delay of the video playing at the second network device due to the fact that the data transmission time is too long can be avoided.
With reference to the first aspect, in a sixth possible implementation manner of the first aspect, the method further includes: the first network equipment receives a third video stream sent by a second camera; the first network device buffers the third video stream, and switches the video stream sent to the first display device from the first video stream to the third video stream after buffering the third video stream, so that the first display device switches from displaying the video corresponding to the first video stream to displaying the video corresponding to the third video stream, wherein the third video stream has the same video coding parameters as the first video stream. The third video stream output by the second camera is cached in advance, and then the video stream sent to the first display device is switched from the first video stream to the third video stream, so that smooth switching of a video picture displayed by the first display device can be realized.
With reference to the sixth possible implementation manner of the first aspect, in a seventh possible implementation manner of the first aspect, before the first network device receives the third video stream sent by the second camera, the method further includes: the first network equipment acquires a camera switching signal triggered by a holder of the first network equipment, disconnects with the first camera according to the camera switching signal and establishes connection with the second camera. The camera switching signal is issued to the first network equipment, so that the switching from the first camera to the second camera can be artificially triggered, and the use requirements of the holder of the first network equipment are met.
With reference to the sixth possible implementation manner of the first aspect, in an eighth possible implementation manner of the first aspect, the second camera is a wireless camera connected to the first network device through a wireless network; the first network device buffering the third video stream, including: the first network device converts the third video stream from data sent using a wireless network communication protocol to data sent using a USB protocol; the first network device buffers the converted third video stream. After the first network device converts the third video stream sent by the second camera from the data sent by the wireless network communication protocol into the data sent by the USB protocol, the visual communication application on the first network device can recognize the third video stream output by the second camera, and the first network device also caches the converted third video stream, so that the first display device realizes smooth switching when switching the shooting picture of the camera.
With reference to the sixth possible implementation manner of the first aspect, in a ninth possible implementation manner of the first aspect, the second camera is a USB camera connected to the first network device through a USB data line. In this embodiment, the first network device may be connected to not only the wireless camera but also the USB camera, and accordingly, when the first display device switches to display the picture taken by the wireless camera, the first display device may switch the video picture taken by the wireless camera to the video picture taken by the USB camera.
In a second aspect, an embodiment of the present application further provides an apparatus for visual communication, where the apparatus is a first network device, and includes: the first receiving unit is used for receiving a first video stream sent by a first camera, and the first camera is a wireless camera connected to the first network equipment through a wireless network; the first conversion unit is used for converting the first video stream from data sent by adopting a wireless network communication protocol into data sent by adopting a Universal Serial Bus (USB) protocol; and the first sending unit is used for sending the converted first video stream to the first display device.
With reference to the second aspect, in a first possible implementation manner of the second aspect, the method further includes: and the identification unit is used for identifying a plurality of cameras, wherein the plurality of cameras comprise the first camera.
With reference to the second aspect, in a second possible implementation manner of the second aspect, the first network device is a set top box.
With reference to the second aspect, in a third possible implementation manner of the second aspect, the method further includes: the second receiving unit is used for receiving a second video stream sent by the first camera; the second conversion unit is used for converting the second video stream from the data sent by adopting the wireless network communication protocol driver into the data sent by adopting the USB protocol driver; and the second sending unit is used for sending the second video stream to the second network equipment.
With reference to the third possible implementation manner of the second aspect, in a fourth possible implementation manner of the second aspect, the method further includes: a third sending unit, configured to send video coding parameters to the first camera before the second receiving unit receives a second video stream sent by the first camera, so that the first camera codes original video data captured by the first camera based on the video coding parameters to obtain the second video stream, and sends the second video stream to the first network device, where the video coding parameters include at least one of the following: a parameter indicating a resolution, a frame rate, a code rate, or an encoding format of the second video stream.
With reference to the fourth possible implementation manner of the second aspect, in a fifth possible implementation manner of the second aspect, the method further includes: an obtaining unit, configured to obtain network performance between the first network device and the second network device or network performance of the second network device; a determining unit, configured to determine the video coding parameter according to the network performance.
With reference to the second aspect, in a sixth possible implementation manner of the second aspect, the method further includes: the third receiving unit is used for receiving a third video sent by the second camera; a buffer unit, configured to buffer the third video stream; and the switching unit is used for switching the video stream sent to the first display device from the first video stream to the third video stream after the third video stream is cached, so that the first display device switches from displaying the video corresponding to the first video stream to displaying the video corresponding to the third video stream, and the third video stream and the first video stream have the same video coding parameters.
With reference to the sixth possible implementation manner of the second aspect, in a seventh possible implementation manner of the second aspect, the method further includes: a switching signal receiving unit, configured to acquire a camera switching signal triggered by a holder of the first network device; and the connection switching unit is used for disconnecting the first camera according to the camera switching signal and establishing connection with the second camera.
With reference to the sixth possible implementation manner of the second aspect, in an eighth possible implementation manner of the second aspect, the second camera is a wireless camera connected to the first network device through a wireless network; the cache unit is specifically configured to: and converting the third video stream from the data sent by adopting the wireless network communication protocol into the data sent by adopting the USB protocol, and caching the converted third video stream.
With reference to the sixth possible implementation manner of the second aspect, in a ninth possible implementation manner of the second aspect, the second camera is a USB camera connected to the first network device through a USB data line.
The apparatus for visual communication provided by the second aspect corresponds to the method for visual communication provided by the first aspect, so that various possible implementations of the apparatus for visual communication provided by the second aspect may refer to various possible implementations of the method for visual communication provided by the first aspect.
In a third aspect, an embodiment of the present application further provides a network device for visual communication, where the network device includes: a processor and a transceiver;
the processor is configured to read the software instructions in the memory, execute the software instructions to: driving the transceiver to receive a first video stream sent by a first camera, wherein the first camera is a wireless camera connected to the first network equipment through a wireless network; converting the first video stream from data sent using a wireless network communication protocol to data sent using a Universal Serial Bus (USB) protocol; the transceiver is driven to transmit the converted first video stream to a first display device.
With reference to the third aspect, in a first possible implementation manner of the third aspect, the processor further implements the following operations: a plurality of cameras is identified, the plurality of cameras including the first camera.
With reference to the third aspect, in a second possible implementation manner of the third aspect, the first network device is a set top box.
With reference to the third aspect, in a third possible implementation manner of the third aspect, the processor further implements the following operations: driving the transceiver to receive a second video stream transmitted by the first camera; the first network device converts the second video stream from data transmitted using a wireless network communication protocol to data transmitted using a USB protocol; the transceiver is driven to transmit the second video stream to the second network device.
With reference to the third possible implementation manner of the third aspect, in a fourth possible implementation manner of the third aspect, before the processor drives the transceiver to receive the second video stream transmitted by the first camera, the processor further implements the following operations: driving the transceiver to transmit to the first camera, so that the first camera encodes original video data captured by the first camera based on the video encoding parameters to obtain the second video stream, and transmitting the second video stream to the first network device, wherein the video encoding parameters include at least one of: a parameter indicating a resolution, a frame rate, a code rate, or an encoding format of the second video stream.
With reference to the fourth possible implementation manner of the third aspect, in a fifth possible implementation manner of the third aspect, the processor further implements the following operations: and acquiring the network performance between the first network equipment and the second network equipment or the network performance of the second network equipment, and determining the video coding parameters according to the network performance.
With reference to the third aspect, in a sixth possible implementation manner of the third aspect, the processor further implements the following operations: driving the transceiver to receive a third video stream transmitted by a second camera; the first network device buffers the third video stream, and switches the video stream sent to the first display device from the first video stream to the third video stream after buffering the third video stream, so that the first display device switches from displaying the video corresponding to the first video stream to displaying the video corresponding to the third video stream, where the third video stream has the same video coding parameters as the first video stream.
With reference to the sixth possible implementation manner of the third aspect, in a seventh possible implementation manner of the third aspect, before the processor drives the transceiver to receive the third video stream transmitted by the second camera, the processor further implements the following operations: and acquiring a camera switching signal triggered by the holder of the first network equipment, disconnecting the camera switching signal from the first camera, and establishing connection with the second camera.
With reference to the sixth possible implementation manner of the third aspect, in an eighth possible implementation manner of the third aspect, the second camera is a wireless camera connected to the first network device through a wireless network; the processor implements an operation of caching the third video stream, and specifically includes: converting the third video stream from data transmitted using a wireless network communication protocol to data transmitted using a USB protocol; and buffering the converted third video stream.
With reference to the sixth possible implementation manner of the third aspect, in a ninth possible implementation manner of the third aspect, the second camera is a USB camera connected to the first network device through a USB data line.
The network device for visual communication provided by the third aspect corresponds to the method for visual communication provided by the first aspect, so that various possible implementations of the network device for visual communication provided by the second aspect may refer to various possible implementations of the method for visual communication provided by the first aspect.
In a fourth aspect, an embodiment of the present application further provides a communication system, including a first network device, a first camera, and a first display device; the first camera is a wireless camera connected to the first network equipment through a wireless network; the first camera is used for shooting or receiving a first video stream and sending the first video stream to the first network equipment; the first network device is configured to receive the first video stream sent by the first camera, convert the first video stream from data sent by using a wireless network communication protocol to data sent by using a Universal Serial Bus (USB) protocol, and send the converted first video stream to the first display device; the first display device is used for receiving and playing the converted first video stream.
With reference to the fourth aspect, in a first possible implementation manner of the fourth aspect, the communication system includes a plurality of cameras, the plurality of cameras includes the first camera, and the first network device may identify the plurality of cameras.
With reference to the fourth aspect, in a second possible implementation manner of the fourth aspect, the first network device is a set top box.
With reference to the fourth aspect, in a third possible implementation manner of the fourth aspect, the display device further includes a second network device and a second display device; the first camera is used for shooting or receiving a second video stream and sending the second video stream to the first network equipment; the first network device is configured to receive the second video stream sent by the first camera, convert the second video stream from data sent by using a wireless network communication protocol to data sent by using a Universal Serial Bus (USB) protocol, and send the converted second video stream to the second network device; the second network device is configured to receive the converted second video stream sent by the first network device, and send the converted second video stream to a second display device; and the second display device is used for receiving and playing the converted second video stream.
With reference to the third possible implementation manner of the fourth aspect, in a fourth possible implementation manner of the fourth aspect, before the first network device receives the second video stream sent by the first camera, the first network device is further configured to send video coding parameters to the first camera, so that the first camera encodes original video data shot by the first camera based on the video coding parameters to obtain the second video stream, and send the second video stream to the first network device, where the video coding parameters include at least one of: a parameter indicating a resolution, a frame rate, a code rate, or an encoding format of the second video stream.
With reference to the fourth possible implementation manner of the fourth aspect, in a fifth possible implementation manner of the fourth aspect, the first network device is further configured to obtain a network performance between the first network device and the second network device or a network performance of the second network device, and determine the video encoding parameter according to the network performance.
With reference to the fourth aspect, in a sixth possible implementation manner of the fourth aspect, the mobile terminal further includes a second camera; the second camera is used for shooting or receiving a third video stream and sending the third video stream to the first network equipment; the first network device is configured to receive the third video stream sent by the second camera, cache the third video stream, and switch the video stream sent to the first display device from the first video stream to the third video stream after caching the third video stream; the first display device is used for receiving the third video stream and switching to play the video corresponding to the third video stream from playing the video corresponding to the first video stream; wherein the third video stream has the same video coding parameters as the first video stream.
With reference to the sixth possible implementation manner of the fourth aspect, in a seventh possible implementation manner of the fourth aspect, the first network device is further configured to, before receiving the third video stream sent by the second camera, acquire a camera switching signal triggered by a holder of the first network device, disconnect from the first camera according to the camera switching signal, and establish connection with the second camera.
With reference to the sixth possible implementation manner of the fourth aspect, in an eighth possible implementation manner of the fourth aspect, the second camera is a wireless camera connected to the first network device through a wireless network; the first network device is specifically configured to convert the third video stream from data sent using a wireless network communication protocol to data sent using a USB protocol; and caching the converted third video stream.
With reference to the sixth possible implementation manner of the fourth aspect, in a ninth possible implementation manner of the fourth aspect, the second camera is a USB camera connected to the first network device through a USB data line.
The communication system provided by the fourth aspect corresponds to the method for visual communication provided by the first aspect, so that various possible implementations of the communication system provided by the fourth aspect may refer to various possible implementations of the method for visual communication provided by the first aspect.
In a fifth aspect, embodiments of the present application provide a computer-readable storage medium having stored therein instructions, which, when run on a computer or processor, cause the computer or processor to perform a method as in the first aspect or any one of its possible designs.
In a sixth aspect, embodiments of the present application provide a computer program product comprising instructions which, when run on a computer or processor, cause the computer or processor to carry out the method as in the first aspect or any one of its possible designs.
In the implementation manner of the embodiment of the application, because the first network device can convert the received first video stream from the data sent by using the wireless network communication protocol to the data sent by using the USB protocol, it is ensured that the first network device can identify the first video stream sent by the first camera, and therefore, the first network device can be connected with the wireless camera through a wireless network without using a hardware interface on the first network device, so that the deployment position of the wireless camera can get rid of the limitation of a connecting line, and the flexibility of the deployment manner of the wireless camera is improved. Therefore, the method for visual communication provided by the embodiment of the application can meet the requirements of video services in various scenes, so that a user can use the video services more conveniently.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
Fig. 1 is a schematic diagram of a network system framework involved in an application scenario in an embodiment of the present application;
fig. 2 is a flowchart illustrating a method for visual communication according to an embodiment of the present application;
fig. 3 is a schematic flow chart of another method for visual communication according to an embodiment of the present application;
fig. 4 is a schematic flowchart of another method for visual communication according to an embodiment of the present application;
fig. 5 is a flowchart illustrating a scene embodiment of a method for visual communication according to an embodiment of the present application;
FIG. 6 is a schematic diagram of an exemplary application scenario in an embodiment of the present application;
fig. 7 is a schematic diagram of an internal driving of a set-top box according to an embodiment of the present application;
FIG. 8 is a schematic structural diagram of a visual communication apparatus according to an embodiment of the present application;
FIG. 9 is a schematic diagram of a network device for visual communication according to an embodiment of the present application;
fig. 10 is a block diagram of a communication system according to an embodiment of the present application.
Detailed Description
In general, based on the consideration of cost, the set-top box is only provided with one USB camera interface, and each interface can only provide data communication service with the set-top box for one USB camera, so the number of USB camera interfaces provided on the set-top box limits the number of USB cameras that can be connected to each set-top box, and the set-top box can only be connected to one USB camera at a time, but cannot be connected to multiple USB cameras at the same time. In addition, the USB camera and the set-top box are usually connected by a USB cable, so that the deployment position of the USB camera is limited by the USB cable. Therefore, in the existing video service, the deployment of the USB camera is limited by hardware conditions, the deployment mode is not flexible enough, and often cannot meet the requirements of the video service in various scenes, so that a user cannot use the video service conveniently.
In order to solve the above technical problem, an embodiment of the present application provides a visual communication method, which can be applied to a set top box and can avoid a limitation of a connection line of hardware on a deployment position of a wireless camera. Specifically, in the process of displaying a video picture shot by the wireless camera on the display device, most of the existing set-top boxes are connected with the USB camera, and the visual communication application in the set-top box is data supporting the USB protocol, so that the set-top box can virtualize the wireless camera into the USB camera, and specifically, convert the received video stream from data sent by the wireless network communication protocol into data sent by the universal serial bus USB protocol, so as to ensure that the visual communication application can identify the video stream data sent by the wireless camera. Therefore, the set top box can be connected with the wireless camera through the wireless network, so that the deployment position of the wireless camera can not be limited by a connecting line, and the flexibility of the deployment mode of the wireless camera is improved. Therefore, the method for visual communication provided by the embodiment of the application can meet the requirements of video services in various scenes, so that a user can use the video services more conveniently.
It should be noted that the method described above may be applied not only to a set-top box, but also to other network devices, such as a gateway device.
For example, one of the scenarios in the embodiment of the present application may be applied to the scenario shown in fig. 1. In this scenario, the network device may be connected to the USB camera 1021, or may be connected to multiple wireless cameras such as the wireless camera 1022 and the wireless camera 1023, and when the user 101 needs to view a video picture shot by the wireless camera 1022 on the display device 104, the user may send a shooting instruction by using a visual communication application on the network device 103 to trigger the wireless camera 1022 to start shooting a video, and then the wireless camera 1022 may send a video stream of the video to the network device 103, where the wireless camera 1022 and the network device 103 are connected through a wireless network; then, the network device 103 converts the received video stream from data transmitted using the wireless network communication protocol to data transmitted using the USB protocol, and transmits the converted video stream to the display device 104, so that the user 101 can view a video picture captured by the wireless camera 1022 on the display device 104.
It is to be understood that the above scenario is only one example of a scenario provided in the embodiment of the present application, and the embodiment of the present application is not limited to this scenario.
It is to be understood that, in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" for describing an association relationship of associated objects, indicating that there may be three relationships, e.g., "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, various non-limiting embodiments accompanying the present application examples are described below with reference to the accompanying drawings. It should be apparent that the embodiments described are some, but not all embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 2, fig. 2 shows a flowchart of a method for visual communication in an embodiment of the present application, where the method may specifically include:
s201: the first network equipment receives a first video stream sent by a first camera, wherein the first camera is a wireless camera connected to the first network equipment through a wireless network.
In an exemplary embodiment, a user may trigger to start the first camera to start shooting a video, for example, the user may utilize a visual communication application on the first network device to issue a shooting instruction to the first network device, so that the first camera starts shooting the video after receiving the shooting instruction, or the user manually starts the first camera, and then the first camera sends a first video stream obtained by shooting to the first network device through a wireless network, so that the first network device can receive the first video stream. The first camera may be a wireless camera connected to the first network device through a wireless network; in some scenarios, the first network device may be a set-top box providing visual communication.
In practical applications, the first camera may be registered with the first network device, so that the first network device can recognize the first camera. For example, in an example, a first camera may send a multicast message to a network device on a fixed multicast address and port, where the multicast message may include attribute information of the first camera, such as a device identifier of the first camera, a format of an output video stream, and a video encoding parameter that can be supported; the first network device may identify the first camera according to the received multicast message, and set a corresponding device description path for the first camera, for example, a device description path of "/dev/video1" may be automatically generated under a "/dev/" directory, where "video1" is a device identifier of the first camera.
Of course, the first network device may recognize not only the first camera but also other cameras, including other wireless cameras and USB cameras besides the first camera. As an example, when a first network device needs to identify multiple wireless cameras, each wireless camera may send a multicast message to a local set top box with a fixed multicast address and port, and the multicast message sent by each wireless camera may include an identifier of the wireless camera; in this way, the first network device may identify and obtain the plurality of wireless cameras according to the received multicast message, and generate a corresponding device description path for each wireless camera, so that the first network device may distinguish the plurality of wireless cameras according to the device description path.
For example, assume that a first wireless camera and a second wireless camera exist in a local area network where a first network device is located, and both the first wireless camera and the second wireless camera send a multicast message containing "video1" and "video2" identifications to the first network device with a same fixed multicast address 224.0.1.187 and port 5638, where "video1" is an identification of the first wireless camera and "video2" is an identification of the second wireless camera. Then, the Android operating system or the Linux operating system on the first network device automatically generates a device description path of the/dev/video 1 and a device description path of the/dev/video 2 under the "/dev/" directory according to the received multicast message. In this way, the first network device can recognize that two wireless cameras exist in the local area network where the first network device is located, and can distinguish each wireless camera by using the device description path.
In addition, in an example of identifying a USB camera, the USB camera is connected to a first network device by using a USB cable, the first network device identifies whether there is a USB camera connected to the first network device through a USB protocol in an operating system, and if the first network device detects that there is a USB camera through the USB protocol, the first network device generates a corresponding device description path for the USB camera, for example, a device description path of "/dev/video0" may be automatically generated under a "/dev/" directory, so that the first network device may distinguish the USB camera according to the device description path.
Note that, because the wireless cameras and the first network device are connected through a wireless network, the number of the wireless cameras that can be connected by the first network device is not limited by a hardware interface on the first network device, the first network device can be connected to and identified by the plurality of wireless cameras, and the first camera connected to the first network device can be any one of the plurality of wireless cameras.
S202: the first network device converts the first video stream from data sent using a wireless network communication protocol to data sent using a Universal Serial Bus (USB) protocol.
In this embodiment, the first network device and the first camera are connected by using a wireless network, so that the first video stream sent by the first camera to the first network device is data sent by using a wireless network communication protocol, and in consideration of the existing video service, most of the visual communication applications in the first network device (such as a gateway device like a set-top box) are video stream data supporting a USB protocol to implement the video service. Therefore, in order to reduce modifications to the existing first network device as much as possible, in an optional implementation, after receiving the first video stream, the first network device may convert the data sent by the first video stream from the wireless network communication protocol to the data sent by the USB protocol, so that only the received video stream data needs to be converted on the first network device side, and the existing first network device may identify and process the video data sent by the wireless camera without specially adding a visual communication application matched with the wireless camera.
As an example, the first video stream may be converted by virtualizing a first camera (wireless camera) into a USB camera. Specifically, a virtual USB camera driver may be built in the first network device, and after the first camera sends the first video stream to the first network device through the wireless network, the virtual USB camera driver in the first network device may convert the received first video stream into video stream data sent by using a USB protocol.
S203: and the first network equipment sends the converted first video stream to the first display equipment.
It is to be understood that after the first network device completes the conversion of the first video stream, the first video stream may be sent to the first display device, so that the display device can display a video picture captured by the first camera according to the received first video stream.
In this embodiment, because the first network device can convert the received first video stream from the data sent by using the wireless network communication protocol to the data sent by using the USB protocol, it is ensured that the first network device can identify the first video stream sent by the first camera, and therefore, when the first network device is connected to the first camera through the wireless network, the first network device does not need to use a hardware interface on the first network device, so that the deployment position of the wireless camera can be free from the limitation of a connecting line, and the flexibility of the deployment mode of the wireless camera is improved. Therefore, the method for visual communication provided by the embodiment of the application can meet the requirements of video services in various scenes, so that a user can use the video services more conveniently.
In an optional scenario, the video captured by the first camera may also be provided to a local user and a remote user in different local area networks, so that both the local user and the remote user can view the video image captured by the first camera. For this reason, the present application provides another embodiment of a method for visual communication, in which a video picture captured by a first camera may be displayed on two different display devices, so that visual communication may be performed between a local user and a remote user. Referring to fig. 3, fig. 3 is a schematic flowchart illustrating a method for visual communication according to an embodiment of the present application, where the method may specifically include:
s301: the first network equipment receives a first video stream and a second video stream which are sent by the wireless camera.
In an exemplary application scenario, when a local user needs to perform visual communication with a remote user, the local user may trigger to start a wireless camera to start shooting a video, for example, manually start the wireless camera, or issue a shooting start instruction to the wireless camera by using a first network device, and then the wireless camera outputs a first video stream and a second video stream of the shot video to the first network device. The wireless camera is connected with the first network equipment through a wireless network and transmits data; the first video stream and the second video stream are both data transmitted by adopting a wireless network communication protocol.
Further, the video encoding parameters of the second video stream may be preset. Specifically, before the first camera outputs the second video stream, the first network device may send the video coding parameter to the first camera, and then the first camera may code the original video data captured by the first camera based on the video coding parameter to obtain the second video stream, and then the first network device may instruct the first camera to output the second video stream. Note that the first video stream output by the first camera may have the same video encoding parameters as the second video stream, or may have different video encoding parameters from the second video stream. That is, the first video stream may be original video data obtained when the first camera takes a video, and in an alternative case, the first video stream may be a video in YUYV format; optionally, the video data may also be obtained by encoding the original video data according to the video encoding parameters.
Wherein the video coding parameters of the second video stream may comprise at least one of the following parameters: parameters indicating the resolution, frame rate, code rate and encoding format of the second video stream. The video coding parameter may be any one of the parameters, or may be a combination of any several parameters, which is not limited herein.
In practical applications, an encoder for encoding a video stream is usually configured inside the wireless camera, and before the first network device transmits the video stream to another network device, the transmitted video stream needs to be encoded. Therefore, on one hand, the encoder in the wireless camera can be fully utilized; on the other hand, because the encoding is completed at the wireless camera side, the first network device does not need to encode the video stream to be transmitted any more, so that the first network device does not need to be configured with an encoder, and the cost and the resource use of the first network device are reduced.
In practical applications, when the first network device performs data communication with different second network devices, network performances of different second network sides are not the same. For example, when a first network device performs data communication with a network device a (a second network device), the network performance of the network device a side may be better, and the data transmission speed is relatively faster, whereas when the first network device performs data communication with a network device B (a second network device), the network performance of the network device B side may be worse, the data transmission speed is relatively slower, and the network performance of the network device a and the network device B side have a larger difference. Moreover, the network performance of the communication network is not stable and constant during the data communication between the first network device and the second network device. For example, during the peak period of network usage, there are more users using the network, which results in poor network performance between the first network device and the second network device and relatively slow network transmission rate; during the network utilization peak period, the number of users using the network is small, the network performance between the first network device and the second network device is high, and the network transmission rate is relatively high.
Therefore, in an alternative embodiment, the first network device may count the network performance between the first network device and the second network device and/or the network performance of the second network device, and determine the video coding parameters according to the network performance. For example, if the network performance between the first network device and the second network device is poor, or the network performance on the second network device side is poor, indicating that the data transmission speed is slow, the resolution of the video may be set to 320 × 240 and/or the frame rate may be set to 5fps, so as to reduce the data amount of the video, thereby reducing the data transmission time and delay; if the network performance between the first network device and the second network device is good, or the network performance at the second network device side is good, indicating that the data transmission speed is fast, the resolution of the video may be set to 1920 × 1080 and/or 30fps, so as to improve the definition of the video picture.
There are various embodiments for testing the network performance between the first network device and the second network device, and for testing the network performance at the second network device side. In an example of testing network performance between two network devices, a first network device may send a test signal based on an RTP (Real-time Transport Protocol)/RTCP (RTP Control Protocol) Protocol to a second network device multiple times, where the test signal may include a data packet with a certain data amount, and then the second network device returns the test signal to the first network device after receiving the test signal, and the first network device may count the data amount of the data packet in the test signal returned by the second network device each time, and the time interval between sending and receiving the test signal by the first network device.
In an embodiment of testing the network performance of the second network device, the second network device may measure the data transmission traffic of the second network device within a period of time, determine, based on a preset measurement criterion, whether the network performance of the second network device is good or bad, and send a determination result of the network performance to the first network device, so that the first network device can determine the network performance of the second network device.
In this embodiment, because the second video stream received by the first network device may be obtained by encoding by the wireless camera, after determining the video encoding parameter according to the network performance, the first network device may send the video encoding parameter using the wireless network communication protocol to the wireless camera through the wireless network, so that the wireless camera performs video encoding on the captured video according to the received video encoding parameter to generate the second video stream. It can be understood that, according to the network performance between the first network device and the second network device or the network performance at the second network device side, the encoding of the second video stream by the wireless camera is dynamically adjusted, so that when the network performance between the first network device and the second network device is poor or the network performance at the second network device side is poor, the data volume of the transmitted video stream is not too large, and thus, the delay in playing the video at the second network device due to too long data transmission time can be avoided.
S302: the first network equipment converts the data sent by the first video stream and the second video stream by adopting a wireless network communication protocol into the data sent by adopting a USB protocol.
In this embodiment, the first network device is connected to the wireless camera through a wireless network communication protocol, and in order to reduce modifications to the existing first network device as much as possible, in an optional implementation, the first video stream and the second video stream may be converted into data sent by using a USB protocol, so that the existing first network device may identify and process the video data sent by the wireless camera only by converting the received video stream data at the first network device side, without specially adding a visual communication application matched with the wireless camera.
It should be noted that, because the first network device is wirelessly connected to the wireless camera, if the first network device needs to issue a control command to the wireless camera, for example, an instruction to issue configuration parameters of the second video stream, an instruction to start video shooting, and the like, the control command sent by using the USB protocol needs to be converted into a control command sent by using a wireless network communication protocol, so that the control command can be smoothly transmitted from the first network device to the wireless camera. Specifically, when a control instruction needs to be issued to the wireless camera, the first network device may obtain a first control instruction sent by using a USB protocol, where the first control instruction may be generated by a visual communication application on the first network device or generated by another device connected to the first network device; then, the first network device may convert the first control instruction into a second control instruction sent by using a wireless network communication protocol, and send the second control instruction to the wireless camera through the wireless network.
S303: and the first network equipment sends the converted first video stream to the first display equipment and sends the converted second video stream to the second network equipment.
In some exemplary scenarios, the first network device, the first display device, and the wireless camera may be deployed in the same local area network, while the second network device, which is a remote network device with respect to the local area network, may be deployed in another local area network. Therefore, when the local user and the remote user in different local area networks carry out visual communication, the local user can display the video picture shot by the wireless camera by using the first display equipment in the local area network where the local user is located, and transmit the video to the second network equipment in the local area network where the remote user is located by using the first network equipment, so that the remote user can display the video picture by using the second display equipment connected with the second network equipment, and the visual communication between the local user and the remote user is realized.
In an optional scenario, the first network device and the second network device in this embodiment may be network devices having a wireless connection function, such as a set top box, and the first network device and the second network device may access a plurality of wireless cameras through a WIFI (wireless network communication) protocol, where the wireless cameras may be WIFI cameras and the like. Of course, the first network device may access the USB camera through the USB cable while accessing the wireless camera.
In this embodiment, in the process of performing visual communication between the local user and the remote user, because the first network device can convert the received video stream from the data sent by using the wireless network communication protocol to the data sent by using the USB protocol, it is ensured that the first network device can identify the video stream sent by the first camera, and therefore, the first network device can be wirelessly connected with the wireless camera without using a hardware interface on the first network device, so that the deployment position of the wireless camera can get rid of the limitation of a connection line, and the flexibility of the deployment mode of the wireless camera is improved. Therefore, the visual communication method provided by the embodiment of the application can meet the visual communication requirements in various scenes, so that a user can use the visual communication service more conveniently.
In addition, if there is a need to switch cameras during the process of performing visual communication between the local user and the remote user, for example, the local user wants to view video pictures shot by different cameras, and the like, at this time, the local user may also complete the switching of the cameras by using the first network device. Based on this, the embodiment of the present application further provides another embodiment of a method for visual communication, which is different from the previous embodiment, and this embodiment focuses on describing in detail a process in which the first network device completes the switching of the camera.
Referring to fig. 4, fig. 4 is a flowchart illustrating a method for visual communication in an embodiment of the present application, where the method may specifically include:
s401: the method comprises the steps that first network equipment receives a first video stream and a second video stream sent by a first camera, wherein the first camera is a wireless camera.
S402: the first network equipment converts the data sent by the first video stream and the second video stream by adopting a wireless network communication protocol into the data sent by adopting a USB protocol.
S403: and the first network equipment sends the converted first video stream to the first display equipment and sends the converted second video stream to the second network equipment.
It should be noted that, similar to S301 to S303 in the previous embodiment, S401 to S403 in the present embodiment may refer to the description of the relevant parts in the previous embodiment, which is not described herein again.
S404: and the second network equipment receives the converted second video stream sent by the first network equipment and performs video decoding on the converted second video stream.
In practical applications, an encoder may be configured in the first camera, so that the second video stream output by the first camera has been previously video-encoded by the encoder in the first camera, and therefore, after receiving the converted second video stream, the second network device may perform video decoding on the converted second video stream by using a decoder configured in the second network device.
S405: and the second network equipment transmits the video stream generated after the video decoding to the second display equipment.
Wherein the second network device may be deployed in the same local area network as the second display device.
After the second network device performs video decoding on the converted second video stream, the second network device may send the video stream generated after the video decoding to the second display device, so that the second display device can display the video picture captured by the first camera based on the received video stream.
S406: when the camera switching signal is acquired, the first network device sends a starting instruction to a second camera indicated by the camera switching signal, wherein the second camera is connected with the first network device through a wireless network.
In the process of visual communication between a local user and a far-end user, if the local user or the far-end user has a requirement for switching the cameras, a camera switching signal can be sent to the first network device for triggering the switching from the first camera to the second camera, so that the first network device can acquire the camera switching signal. The camera switching signal may be triggered by a holder of the first network device, and the second camera is also connected to the first network device through a wireless network.
Note that there are various embodiments in which the first network device acquires the camera switching signal. In one example, a local user may press a switch button on a first network device to indicate that the first network device needs to switch cameras, thereby causing the first network device to generate a camera switching signal to switch to a second camera based on operation of the local user for the switch button; in another example, a local user may switch cameras by using a visual communication application on a first network device, specifically, the user may select a "switch" option for switching to a second camera in the visual communication application, and a camera switching signal for switching to the second camera is generated by the visual communication application, so that the first network device acquires the switching signal. In another example, if there is a need to switch a camera for the remote user, the remote user may send a camera switching signal for switching the camera by using the second network device, and then the second network device sends the camera switching signal to the first network device, so that the first network device acquires the camera switching signal
After acquiring the camera switching signal, the first network device may establish connection with the second camera according to the switching signal. Specifically, a turn-on instruction of the second camera may be generated based on the camera switching signal and sent to the second camera. And then, the first network equipment sends the opening instruction to the second camera so that the second camera starts to output the video stream according to the opening instruction.
In addition, in some embodiments, to further enable smooth switching of video pictures on the first display device, before the first camera outputs the third video stream, the first network device may issue video coding parameters to the second camera, the video coding parameters being the same as those of the first video stream output by the first camera. In this way, the second camera can output the third video stream having the same video coding parameters as the first video stream based on the received video coding parameters, so that the video pictures shot by the second camera and the video pictures shot by the first camera and displayed by the first display device have the same video coding parameters such as resolution, frame rate and the like, and further smooth switching of the video pictures is realized.
It should be noted that the second camera may be a USB camera or a wireless camera. If the second camera is a USB camera, the first network equipment and the second camera can be in wired connection by adopting a USB line, and the first network equipment can send the generated starting instruction of the USB protocol to the second camera through the USB line; if the second camera is a wireless camera, the first network device and the second camera can be in wireless connection based on a wireless network communication protocol, and after the first network device generates a starting instruction of a USB protocol, the starting instruction can be converted into the starting instruction of the wireless network communication protocol, and the starting instruction is sent to the second camera through a wireless network.
S407: and the first network equipment receives the third video stream and the fourth video stream sent by the second camera.
After the second camera acquires the opening instruction, the second camera can respond to the opening instruction and send the video stream of the shot video to the first network device. It should be noted that, if the second camera is a USB camera, the second camera may directly send the third video stream and the fourth video stream sent by using the USB protocol to the first network device through the USB line; and if the second camera is a wireless camera, the second camera can send a third video stream and a fourth video stream which are sent by adopting a wireless network communication protocol to the first network equipment through a wireless network.
S408: the first network device buffers the third video stream and switches the video stream transmitted to the first display device from the first video stream to the buffered third video stream after buffering the third video stream.
It is to be understood that, before buffering the received third video stream, the first network device may convert the data of the third video stream sent by using the wireless network communication protocol into the data sent by using the USB protocol, and then the first network device buffers the converted third video stream.
It should be noted that, in this embodiment, when the camera connected to the first network device is switched, the video stream sent by the first network device to the first display device is not switched, but the video stream sent to the first display device is switched from the first video stream to the buffered third video stream after the first network device has buffered the third video stream for a period of time. By buffering the third video stream and then switching the video streams, the video pictures displayed by the first display device can be smoothly switched.
In an alternative embodiment, at least two buffer queues, assumed to be buffer queue Q, may be set in the first network device 1 And a buffer queue Q 2 And the buffer is used for buffering the video stream received by the first network equipment. The first network device may queue the buffer Q before the local user signals the handover 1 The first video stream cached in the first cache is sent to first display equipment, so that the first display equipment displays the picture shot by the first camera according to the first video stream; when the first network device receives the third video stream, the third video stream may be buffered in the buffer queue Q 2 In the buffer queue Q 2 The first display device may stop receiving the buffering queue Q when the buffered third video stream reaches a certain data amount, or after the buffering time of the third video stream reaches a preset time 1 Receives the first video stream of (1), and receives the buffer queue Q 2 So that the first display device can display the video pictures shot by the second camera based on the received third video stream. Therefore, after a local user or a remote user sends a camera switching signal, a video picture displayed by the first display device can be switched from a video picture shot by the first camera to a video picture shot by the second camera, and because the third video stream is cached in the first network device in advance, when the first display device switches between the video pictures shot by the two cameras, the switching time interval can be shortened as much as possible, so that the smooth switching of the video pictures can be realized.
In some embodiments, to further enable smooth switching of video pictures on the first display device, the first network device may issue video coding parameters to the second camera before the second camera outputs the third video stream, where the video coding parameters are the same as the video coding parameters of the first video stream output by the first camera. In this way, the second camera can output the third video stream having the same video coding parameters as the first video stream based on the received video coding parameters, so that the video pictures shot by the second camera and the video pictures shot by the first camera and displayed by the first display device have the same video coding parameters such as resolution, frame rate and the like, and further smooth switching of the video pictures is realized.
S409: the first network device sends the fourth video stream to the second network device.
In this embodiment, there is no limitation on the execution sequence between step S408 and step S409, that is, step S408 may be executed before step S409, step S409 may be executed before step S408, or step S408 and step S409 may be executed simultaneously.
S410: the second network device buffers the decoded fourth video stream.
In this embodiment, the original video data may be encoded in the second camera in advance, so as to obtain the fourth video stream, where the original video data is the video data obtained by shooting with the second camera. In this way, since the encoding is completed on the wireless camera side, the first network device may not be used to encode the fourth video stream output by the second camera, but may directly transmit to the second network device. Therefore, after receiving the fourth video stream, the second network device may import the received fourth video stream into a decoder for decoding, and then buffer the decoded fourth video stream.
S411: the second network device switches the video stream sent to the second display device from the second video stream to the buffered fourth video stream.
In an alternative embodiment, at least two buffer queues, assumed to be buffer queues R · Q, may also be provided in the second network device 1 And buffer queue R.Q 2 And the buffer is used for buffering a video stream which is received by the second network equipment and generated by video decoding. Before the second network equipment switches pictures shot by the camera, the video stream generated by the second video stream through video decoding can be buffered in a buffer queue R.Q 1 In the middle, the firstThe two display devices can be based on a buffer queue R.Q 1 The video stream buffered in the buffer queue displays the video picture shot by the first camera, and after the fourth video stream is received by the second network device, the video stream generated by video decoding the fourth video stream can be buffered in the buffer queue R.Q 2 In the buffer queue R.Q 2 The second display device may stop receiving the buffering queue R · Q after the buffered video stream reaches a certain data amount or the buffering time of the fourth video stream reaches a preset time 1 Of the video stream, and a receiving buffer queue R.Q 2 To make the first display device based on the buffer queue r.q 2 The video stream cached in the buffer memory displays the video pictures shot by the second camera. Therefore, after a local user or a remote user sends a camera switching signal, a video picture displayed by the second display device can be switched from a video picture shot by the first camera to a video picture shot by the second camera, and because a video stream generated after a sixth video stream is subjected to video decoding is cached in the second network device in advance, when the second display device switches between the video pictures shot by the two cameras, the switching time interval can be shortened as much as possible, so that smooth switching of the video pictures displayed by the second display device can be realized.
In some embodiments, to further achieve smooth switching of video pictures on the second display device, the first network device may issue video coding parameters to the second camera before the second camera outputs the fourth video stream, where the video coding parameters are the same as those of the second video stream output by the first camera. In this way, the second camera may output, based on the received video coding parameter, a fourth video stream having the same video coding parameter as the second video stream, so that the video coding parameters, such as the resolution, the frame rate, and the like, of the video picture shot by the second camera and the video picture shot by the first camera, which are displayed by the second display device, are the same, thereby further achieving smooth switching of the video pictures.
S412: the first network device disconnects from the first camera.
In this embodiment, the first network device may disconnect the connection with the first camera according to the switching signal after the video pictures played by the first display device and the second display device are both switched to the video pictures shot by the second camera. In this way, the first network device may stop receiving the video stream output by the first camera, so that the network bandwidth of the first network device side may be reduced.
In this embodiment, when the first display device and the second display device display the video picture captured by the first camera, if a local user has a need to switch from the first camera to the second camera, the first network device and the second network device may respectively cache a video stream output by the second camera, so that the video pictures displayed by the first display device and the second display device are switched from the video picture captured by the first camera to the video picture captured by the second camera, thereby realizing switching display of the video pictures captured by different cameras on the display device in a process of performing visual communication between the local user and a remote user.
In order to facilitate understanding of the technical solutions of the embodiments of the present application, specific exemplary application scenarios are described below. Referring to fig. 5 and fig. 6 together, fig. 5 is a flowchart illustrating a scene embodiment of a method for visual communication in the embodiment of the present application, and fig. 6 is an application scene diagram illustrating the scene embodiment. In the scene, a local user and a remote user respectively utilize a local set top box and a remote set top box to carry out visual communication, the local set top box and the remote set top box can be connected with a plurality of cameras, and video pictures shot by the cameras can be displayed on a digital television connected with the local set top box and a digital television connected with the remote set top box. The method specifically comprises the following steps:
s501: camera in local set top box identification local area network
It can be appreciated that prior to visual communication between the local user and the remote user, the local set-top box can identify which cameras are present in the local area network in which the local user is located. The camera in the local area network where the local set-top box is located at least comprises one wireless camera and one USB camera, or at least comprises two wireless cameras.
As an example of identifying the wireless camera, the wireless camera may be connected to the set top box through a wireless network communication protocol, and may send a multicast message to the local set top box by using a fixed multicast address and port, where the multicast message may include an identifier of the wireless camera; then, the local set-top box can identify the wireless camera according to the received multicast message, and generate a corresponding device description path for the wireless camera, so that the local set-top box can distinguish a plurality of cameras according to the device description path.
For example, a first wireless camera and a second wireless camera exist in a local area network where the local set top box is located, and the first wireless camera and the second wireless camera send a multicast message containing "video1" and "video2" identifications to the local set top box with the same fixed multicast address 224.0.1.187 and port 5638, where "video1" is the identification of the first wireless camera, and "video2" is the identification of the second wireless camera. And then, the Android operating system or the Linux operating system on the local set top box automatically generates a device description path of the/dev/video 1 and a device description path of the/dev/video 2 under the "/dev/" directory according to the received multicast message. Thus, the local set-top box can recognize that two wireless cameras exist in the local area network where the local set-top box is located, and can distinguish each wireless camera by using the device description path.
As an example of identifying a USB camera, the USB camera is connected to a local set top box by a USB cable, the local set top box identifies whether the USB camera is connected to the local set top box by using a USB protocol in an operating system, and if the local set top box detects that the USB camera is connected to the local set top box by using the USB protocol, the local set top box generates a corresponding device description path for the USB camera, for example, the device description path may be automatically generated under a "/dev/" directory, so that the local set top box can distinguish the USB camera according to the device description path.
It should be noted that, considering that a local set-top box is usually connected to only one USB camera in practical applications, if the camera connected to the local set-top box includes both a USB camera and a wireless camera, an identification rule for the USB camera may be preset, for example, the identifier of the USB camera may be set to "video0", and the identifiers of other cameras in addition to the USB camera represent the wireless cameras, etc. Therefore, the local set top box can distinguish the USB camera from the wireless camera according to the identification rule.
S502: and the local set top box virtualizes the identified wireless camera into a USB camera.
It can be understood that, in view of the existing visual communication service, most set top boxes acquire videos by using USB cameras, and implement the visual communication service based on video streams under a USB protocol, therefore, in order to reduce modifications to the existing set top boxes as much as possible, a local set top box may virtualize identified wireless cameras into USB cameras.
As an exemplary embodiment of virtualizing a USB camera, as shown in fig. 7, a "virtual USB camera driver" may be added to an operating system on a set-top box, so as to convert data transmitted by a WIFI protocol driver into data recognizable by the USB protocol driver, and convert data transmitted by the USB protocol driver into data recognizable by the WIFI protocol driver, and each wireless camera corresponds to one "virtual USB camera driver". The USB protocol driver is used for driving the local set top box to process data of the USB protocol, and the WIFI protocol driver is used for driving the wireless camera to process the data of the WIFI protocol. In an alternative embodiment, the USB protocol driver may specifically be a UVC (USB Video Class) protocol driver.
In this way, data (such as video stream) of the WIFI protocol sent by the wireless camera to the local set top box can be converted into data of the USB protocol by using the "virtual USB camera driver", so that for the local set top box, the wireless camera can be regarded as a virtual USB camera, the virtual USB camera is still operated in a manner of operating the USB camera, and only in the process of data transmission between the wireless camera and the wireless camera, protocol conversion needs to be performed on the transmitted data. For example, when the visual communication application on the local set top box issues a video shooting start instruction to the wireless camera, the local set top box converts the start instruction adopting the USB protocol into the start instruction adopting the WIFI protocol, and sends the converted start instruction to the corresponding wireless camera, so that the wireless camera starts video shooting.
S503: and the local set top box sends the obtained first video stream sent by adopting the USB protocol to the digital television for displaying, and sends the second video stream sent by adopting the USB protocol to the remote set top box.
As an example, if the local user wants to see a video picture taken by the first camera on a digital television, the local user may select the first camera using a visual communication application on the local set-top box. In this way, the visual communication application can generate a starting instruction for the first camera based on the selection of the user on the first camera, and transmit the instruction to the first camera through the local set-top box, so that the first camera starts the shooting of the video and outputs two paths of video streams, namely the first video stream and the second video stream, according to the starting instruction. In an alternative example, the video format of the original video data generated by the video captured by the camera may be YUYV format, and in a general case, the video format of the first video stream may be YUYV format, and the video format of the second video stream may be a video format obtained by encoding the first camera according to the video encoding parameters acquired in advance, and of course, if the original video data needs to be encoded to obtain the first video stream, the video format of the first video stream may also be a video format obtained by encoding according to the video encoding parameters acquired in advance.
It should be noted that, if the first camera is a USB camera, the local set-top box may send a start instruction for the first camera to the first camera through a USB cable, and then the first camera sends a first video stream and a second video stream that use a USB protocol to the local set-top box through the USB cable; if the first camera is a wireless camera, after the local set top box acquires the opening instruction, the virtual USB camera drive is utilized to convert the opening instruction into a WIFI protocol from the USB protocol, and the opening instruction of the WIFI protocol is sent to the wireless camera, then the wireless camera sends a first video stream and a second video stream of the WIFI protocol to the local set top box through a wireless network, and the virtual USB camera drive in the local set top box converts the first video stream and the second video stream of the WIFI protocol into a first video stream and a second video stream which adopt the USB protocol.
Note that the local set-top box may also test the network bandwidth with the remote set-top box. In one example, an RTP (real time Transport Protocol)/RTCP (real time Transport Control Protocol) module configured in the local set-top box for transmitting the video stream may perform a statistical test on a network transmission rate between the local set-top box and the remote set-top box to determine whether a network bandwidth is sufficient. Then, the local set-top box can dynamically adjust the video format and the frame rate of the video stream output by the first camera according to the network bandwidth obtained by the test, and send the video format and the frame rate to the first camera.
Thus, when the network bandwidth between the local set-top box and the remote set-top box is relatively abundant and the network transmission rate is relatively high, the video format and the frame rate of the video stream output by the first camera may be increased, for example, the video format of the video stream may be adjusted to 1080P, the resolution of the video stream may be 1920 × 1080, and the frame rate may be 30fps, so that the definition of a video picture viewed by a remote user on the digital television is higher; when the network bandwidth between the local set-top box and the remote set-top box is insufficient and the network transmission rate is slow, the resolution and the frame rate of the video stream output by the first camera may be reduced, for example, the video format of the video stream may be adjusted to "standard definition", the resolution is 320 × 240, and the frame rate is 5fps, so as to reduce the data amount of the video stream transmitted from the local set-top box to the remote set-top box, thereby reducing the play delay of the video stream, and ensuring that a remote user can smoothly view a video picture shot by the first camera.
The local set-top box can adjust the resolution and the frame rate of the video stream into a plurality of levels. For example, the resolutions may be divided into 1920 × 1080, 1280 × 720, 640 × 480, 320 × 240, etc. in order of high to low, and the frame rates may be divided into 30fps, 20fps, 15fps, 5fps, etc. in order of high to low.
After obtaining a first video stream and a second video stream adopting a USB protocol, a local set-top box may send the first video stream to a digital television connected to the local set-top box, so that a local user can view a video picture captured by a first camera on the digital television; and the local set-top box sends the second video stream to the far-end set-top box through a data channel pre-established with the far-end set-top box, so that the far-end set-top box decodes the second video stream and sends the decoded second video stream to a digital television connected with the far-end set-top box, and a far-end user can watch a video picture shot by the first camera on the digital television.
S504: and the local set top box sends the acquired camera switching signal to the second camera and obtains a third video stream and a fourth video stream which are obtained based on the video stream output by the second camera.
In this embodiment, when a local user or a remote user needs to switch a video screen shot by a camera, a trigger operation for switching the camera may be executed, so that the local set-top box obtains a switching signal generated based on the trigger operation; then, the local set top box determines a second camera appointed by the switching signal based on the switching signal and the equipment description path, generates a starting instruction aiming at the second camera, and sends the starting instruction to the second camera, so that the second camera outputs two paths of video streams based on the starting instruction, and the local set top box can obtain a third video stream and a fourth video stream based on the video streams output by the second camera. The process of obtaining the third video stream and the fourth video stream by the local set-top box may refer to the description of relevant parts in the foregoing embodiments, which is not described herein again.
In practical application, if the network bandwidth between the local set-top box and the remote set-top box is relatively stable, the video formats and frame rates of the video streams output by the first camera and the second camera may be kept unchanged. Therefore, in this embodiment, when the first camera needs to be switched to the second camera, the local set top box may further send the video parameters, such as the video format and the frame rate, of the first camera to the second camera, so that the second camera outputs the corresponding video stream based on the video parameters.
S505: and the local set top box caches the obtained third video stream, and switches the video stream sent to the digital television connected with the local set top box from the first video stream to the third video stream after caching the third video stream.
A buffer queue Q can be arranged in a local set-top box 1 And a buffer queue Q 2 The digital television connected with the local set-top box can buffer the queue Q from the buffer queue 1 Or buffer queue Q 2 And acquiring the video stream. Before the digital display switches the pictures shot by the camera, the local set-top box may buffer the first video stream in the buffer queue Q 1 The digital television can be based on the buffer queue Q 1 The first video stream cached in the middle displays the video picture shot by the first camera, and after the local set top box receives the third video stream, the third video stream can be cached in the buffer queue Q 2 After the buffering of the third video stream is started, the digital television can stop receiving the buffering queue Q 1 While receiving the buffer queue Q 2 So that the digital television is based on the buffer queue Q 2 The cached third video stream displays the video pictures shot by the second camera. Therefore, after a local user sends a camera switching signal, the video pictures displayed by the digital television can be switched from the video pictures shot by the first camera to the video pictures shot by the second camera, and the digital television has two video streams since the third video stream is cached in the local set top box in advanceWhen the video pictures shot by the camera are switched, the time interval of the switching can be shortened as much as possible, so that the smooth switching of the video pictures displayed by the digital television can be realized.
S506: and the local set top box transmits the fourth video stream to the remote set top box.
S507: and the far-end set top box carries out buffering after decoding the received fourth video stream video, and switches the video stream sent to the digital television connected with the far-end set top box from the second video stream to the fourth video stream after starting buffering the decoded fourth video stream.
In this embodiment, a buffer queue R · Q may be set in the remote set-top box 1 And buffer queue R.Q 2 . Before the digital television connected with the far-end set-top box switches the video pictures shot by the camera, the video stream generated by the second video stream after video decoding can be buffered in a buffer queue R.Q 1 The digital television can be based on the buffer queue R.Q 1 The video stream cached in the middle buffer memory displays the video picture shot by the first camera, when the far-end set-top box receives the fourth video stream, the video decoding can be firstly carried out on the fourth video stream, and the video stream generated by carrying out the video decoding on the sixth video stream is cached in the buffer queue R.Q 2 In the buffer queue R.Q 2 After starting buffering video stream, the digital television can stop receiving buffer queue R.Q 1 In the video stream, and receive the buffer queue R.Q 2 So that the digital television is based on the buffer queue R.Q 2 The video stream cached in the buffer memory displays the video pictures shot by the second camera. Therefore, the digital television can switch and display the video pictures shot by different cameras, and the switching time interval can be shortened as much as possible, so that the smooth switching of the video pictures displayed by the digital television can be realized.
In this embodiment, the local set-top box can be connected to a plurality of cameras at the same time; moreover, the camera and the local set top box can not be connected through a connecting line, so that the deployment position of the camera is not limited by the connecting line when the camera is deployed, the flexibility of the deployment mode of the camera is improved, the requirement of visual communication under various scenes can be met, and the visual communication service can be conveniently and rapidly used by a user.
In addition, the embodiment of the application also provides a visual communication device. Referring to fig. 8, fig. 8 is a schematic structural diagram of a visual communication apparatus in an embodiment of the present application, where the apparatus 800 may be a first network device, and the apparatus 800 may specifically include:
a first receiving unit 801, configured to receive a first video stream sent by a first camera, where the first camera is a wireless camera connected to the first network device through a wireless network, and for example, the first receiving unit 801 may be specifically configured to execute the method in step 201, specifically please refer to the description of step 201 in the method embodiment, and details are not repeated here;
a first conversion unit 802, configured to convert the first video stream from data sent by using a wireless network communication protocol to data sent by using a universal serial bus USB protocol, where for example, the first conversion unit 802 may be specifically configured to execute the method in step 202, and please refer to the description of step 202 in the method embodiment, which is not described herein again;
a first sending unit 803, configured to send the converted first video stream to a first display device, where the first sending unit 803 may be specifically configured to execute the method in step 203, and please refer to the description of the step 203 part in the method embodiment, which is not described herein again.
Because the first conversion unit 802 can convert the received first video stream from the data sent by the wireless network communication protocol to the data sent by the USB protocol, it is ensured that the visual communication application on the network device can identify the video stream data sent by the wireless camera, and thus, the first network device can be connected to the wireless camera through the wireless network, which not only makes the number of the wireless cameras connected to the first network device not limited by the number of hardware interfaces provided by the first network device, but also makes the first network device simultaneously connected to multiple wireless cameras, and the deployment position of the wireless camera can not be limited by a connecting wire, thereby improving the flexibility of the deployment mode of the wireless camera.
In an alternative implementation, the apparatus 800 further includes:
the identification unit is used for identifying a plurality of cameras, and the plurality of cameras comprise the first camera.
In an alternative implementation, the first network device may be a set top box.
In an optional implementation manner, the apparatus 800 may further include:
a second receiving unit, configured to receive a second video stream sent by the first camera, where the second receiving unit may be specifically configured to execute the method in step 301, and please refer to the description of step 301 in the method embodiment, which is not described herein again;
a second conversion unit, configured to convert the second video stream from data sent by using a wireless network communication protocol driver to data sent by using a USB protocol driver, and the function of the second conversion unit is similar to that of the first conversion unit 802;
a second sending unit, configured to send the second video stream to the second network device.
In an optional implementation, the apparatus 800 further includes:
a third sending unit, configured to send, before the second receiving unit receives the second video stream sent by the first camera, video coding parameters to the first camera, so that the first camera codes, based on the video coding parameters, original video data captured by the first camera to obtain the second video stream, and sends the second video stream to the first network device, where the video coding parameters include at least one of the following: a parameter indicating a resolution, a frame rate, a code rate, or an encoding format of the second video stream. In this way, the first network device can implement setting of the video coding parameters of the second video stream output by the first camera.
In an alternative embodiment, the apparatus 800 further comprises:
an obtaining unit, configured to obtain network performance between the first network device and the second network device or network performance of the second network device;
a determining unit, configured to determine the video coding parameter according to the network performance.
The first network device can dynamically adjust the encoding of the wireless camera on the second video stream according to the network performance between the first network device and the second network device or the network performance at the side of the second network device, so that when the network performance between the first network device and the second network device is poor or the network performance at the side of the second network device is poor, the data volume of the transmitted video stream is not too large, and the delay of playing the video at the second network device caused by the overlong data transmission time can be avoided.
In an alternative implementation, the apparatus 800 further includes:
the third receiving unit is used for receiving a third video sent by the second camera;
a buffer unit, configured to buffer the third video stream;
a switching unit, configured to switch a video stream sent to the first display device from the first video stream to the third video stream after the third video stream is buffered, so that the first display device switches from displaying a video corresponding to the first video stream to displaying a video corresponding to the third video stream, where the third video stream has the same video coding parameters as the first video stream.
It can be understood that the smooth switching of the video picture displayed by the first display device can be achieved by caching the third video stream output by the second camera in advance and then switching the video stream sent to the first display device from the first video stream to the third video stream, and the second camera can output the third video stream having the same video coding parameters as the first video stream based on the received video coding parameters, so that the video picture shot by the second camera and the video picture shot by the first camera and having the same video coding parameters such as resolution, frame rate, code rate, video format and the like are the same, and the smoothness of the video picture switched by the first display device is further increased.
In an optional implementation, the apparatus 800 further includes:
a switching signal receiving unit, configured to acquire a camera switching signal triggered by a holder of the first network device;
and the connection switching unit is used for disconnecting the first camera according to the camera switching signal and establishing connection with the second camera. The camera switching signal is sent to the first network equipment, so that the switching from the first camera to the second camera can be triggered manually, and the use requirements of a holder of the first network equipment are met.
In an optional implementation manner, the second camera is a wireless camera connected to the first network device through a wireless network; the cache unit is specifically configured to:
and converting the third video stream from data sent by adopting a wireless network communication protocol into data sent by adopting a USB protocol, and caching the converted third video stream.
After the first network device converts the third video stream sent by the second camera from the data sent by the wireless network communication protocol into the data sent by the USB protocol, the visual communication application on the first network device can recognize the third video stream output by the second camera, and the first network device also caches the converted third video stream, so that the first display device realizes smooth switching when switching the shooting picture of the camera.
In an optional implementation manner, the second camera is a USB camera connected to the first network device through a USB data line.
It should be noted that the first network device may be connected to both the wireless camera and the USB camera, and accordingly, when the first display device switches to display the picture taken by the wireless camera, the first display device may switch the video picture taken by the wireless camera to the video picture taken by the USB camera.
In this embodiment, because the first network device can convert the received first video stream from the data sent by using the wireless network communication protocol to the data sent by using the USB protocol, it is ensured that the first network device can identify the first video stream sent by the first camera, and therefore, the first network device can be connected with the wireless camera through the wireless network, so that the deployment position of the wireless camera is not limited by the connecting line, and the flexibility of the deployment mode of the wireless camera is improved. Therefore, the device for visual communication provided by the embodiment of the application can meet the requirements of video services in various scenes, so that a user can use the video services more conveniently and rapidly.
In addition, the embodiment of the application also provides a network device for visual communication. Referring to fig. 9, fig. 9 is a schematic diagram of a network device provided in an embodiment of the present application, where the network device 900 includes: a processor 901 and a transceiver 902;
the processor 901 is configured to read software instructions in the memory, execute the software instructions to:
driving the transceiver 902 to receive a first video stream sent by a first camera, where the first camera is a wireless camera connected to the first network device through a wireless network, please refer to the description of step 201 in the method embodiment, which is not described herein again;
converting the first video stream from data sent by using a wireless network communication protocol to data sent by using a Universal Serial Bus (USB) protocol, please refer to the description of step 202 in the method embodiment, which is not described herein again;
for please refer to the description of the step 203 in the method embodiment for driving the transceiver 902 to transmit the converted first video stream to the first display device, which is not described herein again.
In an alternative implementation, the processor 901 further implements the following operations:
a plurality of cameras is identified, the plurality of cameras including the first camera.
In an optional implementation manner, the first network device is a set top box.
In an alternative implementation, the processor 901 further implements the following operations:
driving the transceiver 902 to receive a second video stream transmitted by the first camera;
the first network equipment converts the second video stream from data transmitted by adopting a wireless network communication protocol to data transmitted by adopting a USB protocol;
the transceiver 902 is driven to transmit the second video stream to the second network device.
In an optional implementation manner, before the processor 901 drives the transceiver 902 to receive the second video stream transmitted by the first camera, the processor 901 further implements the following operations:
driving the transceiver 902 to send to the first camera, so that the first camera encodes original video data captured by the first camera based on the video encoding parameters to obtain the second video stream, and sending the second video stream to the first network device, where the video encoding parameters include at least one of: a parameter indicating a resolution, a frame rate, a code rate, or an encoding format of the second video stream.
In an alternative implementation, the processor 901 further implements the following operations:
and acquiring the network performance between the first network equipment and the second network equipment or the network performance of the second network equipment, and determining the video coding parameters according to the network performance.
In an alternative implementation, the processor 901 further implements the following operations:
driving the transceiver 902 to receive a third video stream transmitted by a second camera;
the first network device buffers the third video stream, and switches the video stream sent to the first display device from the first video stream to the third video stream after buffering the third video stream, so that the first display device switches from displaying the video corresponding to the first video stream to displaying the video corresponding to the third video stream, where the third video stream has the same video coding parameters as the first video stream.
In an optional implementation manner, before the processor 901 drives the transceiver 902 to receive the third video stream transmitted by the second camera, the processor 901 further implements the following operations:
and acquiring a camera switching signal triggered by the holder of the first network equipment, disconnecting the camera switching signal from the first camera, and establishing connection with the second camera.
In an optional implementation manner, the second camera is a wireless camera connected to the first network device through a wireless network; the processor 901 implements an operation of caching the third video stream, which specifically includes:
converting the third video stream from data transmitted using a wireless network communication protocol to data transmitted using a USB protocol; and caching the converted third video stream.
In an optional implementation manner, the second camera is a USB camera connected to the first network device through a USB data line.
In this embodiment, because the first network device can convert the received first video stream from the data sent by using the wireless network communication protocol to the data sent by using the USB protocol, it is ensured that the first network device can identify the first video stream sent by the first camera, and therefore, the first network device can be connected with the wireless camera through the wireless network, so that the deployment position of the wireless camera is not limited by the connecting line, and the flexibility of the deployment mode of the wireless camera is improved.
In addition, the embodiment of the application also provides a communication system. Referring to fig. 10, fig. 10 is a schematic diagram illustrating an architecture of a communication system according to an embodiment of the present application, where the communication system includes: a first network device 1001, a first camera 1002, and a first display device 1003;
the first camera 1002 is a wireless camera connected to the first network device 1001 through a wireless network;
the first camera 1002 is configured to capture or receive a first video stream, and send the first video stream to the first network device 1001;
the first network device 1001 is configured to receive the first video stream sent by the first camera 1002, convert the first video stream from data sent by using a wireless network communication protocol to data sent by using a universal serial bus USB protocol, and send the converted first video stream to the first display device 1003;
the first display device 1003 is configured to receive and play the converted first video stream.
In an alternative implementation, the communication system includes a plurality of cameras, the plurality of cameras includes the first camera 1002, and the first network device 1001 may identify the plurality of cameras.
In an alternative implementation, the first network device 1001 is a set-top box.
In an alternative implementation, the communication system further comprises a second network device 1004 and a second display device 1006;
the first camera 1002 is configured to capture or receive a second video stream, and send the second video stream to the first network device 1001;
the first network device 1001 is configured to receive the second video stream sent by the first camera 1002, convert the second video stream from data sent by using a wireless network communication protocol to data sent by using a Universal Serial Bus (USB) protocol, and send the converted second video stream to the second network device 1004; the second network device 1004 is configured to receive the converted second video stream sent by the first network device 1001, and send the converted second video stream to a second display device 1006;
the second display device 1006 is configured to receive and play the converted second video stream.
In an optional implementation manner, before the first network device 1001 receives the second video stream sent by the first camera 1002, the first network device 1001 is further configured to send video coding parameters to the first camera 1002, so that the first camera 1002 encodes original video data captured by the first camera 1002 based on the video coding parameters to obtain the second video stream, and send the second video stream to the first network device 1001, where the video coding parameters include at least one of: a parameter indicating a resolution, a frame rate, a code rate, or an encoding format of the second video stream.
In an optional implementation manner, the first network device 1001 is further configured to obtain a network performance between the first network device 1001 and the second network device 1004 or a network performance of the second network device 1004, and determine the video encoding parameter according to the network performance.
In an optional implementation, the communication system further includes a second camera 1005;
the second camera 1005 is configured to capture or receive a third video stream, and send the third video stream to the first network device 1001;
the first network device 1001 is configured to receive the third video stream sent by the second camera 1005, buffer the third video stream, and switch the video stream sent to the first display device 1003 from the first video stream to the third video stream after the third video stream is buffered;
the first display device 1003 is configured to receive the third video stream and switch from playing a video corresponding to the first video stream to playing a video corresponding to the third video stream;
wherein the third video stream has the same video coding parameters as the first video stream.
In an optional implementation manner, the first network device 1001 is further configured to acquire a camera switching signal triggered by a holder of the first network device 1001 before receiving a third video stream sent by a second camera 1005, disconnect from the first camera 1002 according to the camera switching signal, and establish connection with the second camera 1005.
In an alternative implementation manner, the second camera 1005 is a wireless camera connected to the first network device 1001 through a wireless network; the first network device 1001 is specifically configured to convert the third video stream from data sent by using a wireless network communication protocol to data sent by using a USB protocol, and cache the converted third video stream.
In an alternative implementation manner, the second camera 1005 is a USB camera connected to the first network device 1001 through a USB data line.
In this embodiment, the first network device in the communication system may convert the received data sent by the video stream according to the wireless network communication protocol into data sent by the USB protocol, so that the first network device may be connected to the wireless cameras through the wireless network, which not only makes the number of the wireless cameras in the communication system not limited by the number of hardware interfaces provided by the first network device, but also makes the first network device simultaneously connected to a plurality of wireless cameras, and the deployment position of the wireless cameras may not be limited by a connection line, thereby improving the flexibility of the deployment mode of the wireless cameras in the communication system.
Embodiments of the present application also provide a computer-readable storage medium having stored therein instructions, which when executed on a computer, cause the computer to perform one or more steps of any one of the methods described above. The respective constituent modules of the above-described network device for visual communication may be stored in the computer-readable storage medium if they are implemented in the form of software functional units and sold or used as independent products.
Based on such understanding, the embodiments of the present application further provide a computer program product, which substantially or partially contributes to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, and include several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor therein to execute all or part of the steps of the method according to each embodiment of the present application.
In the names of "… …", "… …", etc., the "first" mentioned in the embodiments of the present application is used only for name identification, and does not represent the first in sequence. The same applies to "second" etc.
As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the above embodiment methods can be implemented by software plus a general hardware platform. Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, which may be stored in a storage medium, such as a read-only memory (ROM)/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network communication device such as a router, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.
All the embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, the method embodiments and apparatus embodiments are substantially similar to the system embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the system embodiments for related points. The above-described embodiments of the apparatus and system are merely illustrative, wherein modules described as separate parts may or may not be physically separate, and parts shown as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The above description is only an exemplary embodiment of the present application, and is not intended to limit the scope of the present application.

Claims (22)

1. A method of visual communication, the method comprising:
a first network device receives a first video stream sent by a first camera, wherein the first camera is a wireless camera connected to the first network device through a wireless network;
the first network equipment converts the first video stream from data sent by adopting a wireless network communication protocol into data sent by adopting a Universal Serial Bus (USB) protocol;
and the first network equipment sends the converted first video stream to first display equipment.
2. The method of visual communication of claim 1, further comprising:
the first network device identifies a plurality of cameras, including the first camera.
3. The method of visual communication of claim 1, wherein the first network device is a set-top box.
4. A method of visual communication according to any one of claims 1-3, wherein the method further comprises:
the first network equipment receives a second video stream sent by the first camera;
the first network equipment converts the second video stream from data transmitted by adopting a wireless network communication protocol to data transmitted by adopting a USB protocol;
and the first network equipment sends the second video stream to the second network equipment.
5. The method of visual communication according to claim 4, wherein before the first network device receives the second video stream transmitted by the first camera, the method further comprises:
the first network device sends video coding parameters to the first camera, so that the first camera codes original video data shot by the first camera based on the video coding parameters to obtain a second video stream, and sends the second video stream to the first network device, wherein the video coding parameters include at least one of the following: a parameter indicating a resolution, a frame rate, a code rate, or an encoding format of the second video stream.
6. The method of visual communication of claim 5, further comprising:
the first network device obtains network performance between the first network device and the second network device or network performance of the second network device, and determines the video coding parameters according to the network performance.
7. The method of visual communication of claim 1, wherein the method further comprises:
the first network equipment receives a third video stream sent by a second camera;
the first network device buffers the third video stream, and switches the video stream sent to the first display device from the first video stream to the third video stream after buffering the third video stream, so that the first display device switches from displaying the video corresponding to the first video stream to displaying the video corresponding to the third video stream, where the third video stream has the same video coding parameters as the first video stream.
8. The method of visual communication according to claim 7, wherein before the first network device receives the third video stream transmitted by the second camera, the method further comprises:
the first network equipment acquires a camera switching signal triggered by a holder of the first network equipment, disconnects with the first camera according to the camera switching signal and establishes connection with the second camera.
9. The method of visual communication of claim 7, wherein the second camera is a wireless camera connected to the first network device through a wireless network;
the first network device caching the third video stream, including:
the first network equipment converts the third video stream from data transmitted by adopting a wireless network communication protocol to data transmitted by adopting a USB protocol;
the first network device caches the converted third video stream.
10. The method of visual communication of claim 7, wherein the second camera is a USB camera connected to the first network device through a USB data line.
11. An apparatus for visual communication, the apparatus comprising:
the first receiving unit is used for receiving a first video stream sent by a first camera, and the first camera is a wireless camera connected to the first network equipment through a wireless network;
the first conversion unit is used for converting the first video stream from data sent by adopting a wireless network communication protocol into data sent by adopting a Universal Serial Bus (USB) protocol;
and the first sending unit is used for sending the converted first video stream to the first display device.
12. The apparatus for visual communication according to claim 11, further comprising:
the identification unit is used for identifying a plurality of cameras, and the plurality of cameras comprise the first camera.
13. The apparatus for visual communication of claim 11, wherein the first network device is a set-top box.
14. An apparatus for visual communication according to any one of claims 11-13, wherein the apparatus further comprises:
the second receiving unit is used for receiving a second video stream sent by the first camera;
the second conversion unit is used for converting the second video stream from the data sent by adopting the wireless network communication protocol driver into the data sent by adopting the USB protocol driver;
a second sending unit, configured to send the second video stream to the second network device.
15. The apparatus for visual communication of claim 14, further comprising:
a third sending unit, configured to send, before the second receiving unit receives the second video stream sent by the first camera, video coding parameters to the first camera, so that the first camera codes, based on the video coding parameters, original video data captured by the first camera to obtain the second video stream, and sends the second video stream to the first network device, where the video coding parameters include at least one of the following: a parameter indicating a resolution, a frame rate, a code rate, or an encoding format of the second video stream.
16. The apparatus for visual communication of claim 15, further comprising:
an obtaining unit, configured to obtain network performance between the first network device and the second network device or network performance of the second network device;
a determining unit, configured to determine the video coding parameter according to the network performance.
17. The apparatus for visual communication according to claim 11, further comprising:
the third receiving unit is used for receiving a third video sent by the second camera;
a buffer unit, configured to buffer the third video stream;
a switching unit, configured to switch a video stream sent to the first display device from the first video stream to the third video stream after the third video stream is buffered, so that the first display device switches from displaying a video corresponding to the first video stream to displaying a video corresponding to the third video stream, where the third video stream has the same video coding parameters as the first video stream.
18. The apparatus for visual communication of claim 17, further comprising:
a switching signal receiving unit, configured to acquire a camera switching signal triggered by a holder of the first network device;
and the connection switching unit is used for disconnecting the first camera and establishing connection with the second camera according to the camera switching signal.
19. The apparatus for visual communication according to claim 17, wherein the second camera is a wireless camera connected to the first network device through a wireless network;
the cache unit is specifically configured to: and converting the third video stream from data sent by adopting a wireless network communication protocol into data sent by adopting a USB protocol, and caching the converted third video stream.
20. The apparatus of visual communication according to claim 17, wherein the second camera is a USB camera connected to the first network device through a USB data line.
21. A network device for visual communication, the network device comprising: a processor and a transceiver;
the processor is configured to read software instructions in the memory, execute the software instructions to:
driving the transceiver to receive a first video stream sent by a first camera, wherein the first camera is a wireless camera connected to the first network equipment through a wireless network;
converting the first video stream from data sent using a wireless network communication protocol to data sent using a Universal Serial Bus (USB) protocol;
and driving the transceiver to transmit the converted first video stream to a first display device.
22. A method of adjusting resolution and frame rate of a video stream, wherein the resolution of the video comprises 1920 × 1080, 1280 × 720, 640 × 480, 320 × 240, in high-to-low order, and the frame rate comprises 30fps, 20fps, 15fps, 5fps, etc., in high-to-low order;
when the first network bandwidth and the first network transmission rate are transmitted, adjusting the resolution of the video to 1920 x 1080 and the frame rate to 30fps;
adjusting the resolution of the video to 320 × 240 and the frame rate to 5fps during transmission at a second network bandwidth and a second network transmission rate, so as to reduce the data volume of the video stream transmitted from the local set top box to the remote set top box,
the first network bandwidth is greater than the second network bandwidth, and the first network transmission rate is greater than the second network transmission rate.
CN202210659712.9A 2018-05-18 2018-05-18 Visual communication method, device and system Pending CN115209097A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210659712.9A CN115209097A (en) 2018-05-18 2018-05-18 Visual communication method, device and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810482538.9A CN110505441B (en) 2018-05-18 2018-05-18 Visual communication method, device and system
CN202210659712.9A CN115209097A (en) 2018-05-18 2018-05-18 Visual communication method, device and system

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN201810482538.9A Division CN110505441B (en) 2018-05-18 2018-05-18 Visual communication method, device and system

Publications (1)

Publication Number Publication Date
CN115209097A true CN115209097A (en) 2022-10-18

Family

ID=68584046

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202210659712.9A Pending CN115209097A (en) 2018-05-18 2018-05-18 Visual communication method, device and system
CN201810482538.9A Active CN110505441B (en) 2018-05-18 2018-05-18 Visual communication method, device and system

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN201810482538.9A Active CN110505441B (en) 2018-05-18 2018-05-18 Visual communication method, device and system

Country Status (1)

Country Link
CN (2) CN115209097A (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110868572A (en) * 2019-11-28 2020-03-06 重庆紫光华山智安科技有限公司 Image acquisition processing device, image acquisition method and system
WO2022104638A1 (en) * 2020-11-19 2022-05-27 深圳市大疆创新科技有限公司 Image transmission data processing method, image transmission device, terminal device, and storage medium
CN115118601A (en) * 2022-05-26 2022-09-27 广东跃昉科技有限公司 Transmission bandwidth allocation method and device for multi-path USB camera and computer equipment

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7949117B2 (en) * 2005-04-28 2011-05-24 Apple Inc. Heterogeneous video conferencing
CN202135246U (en) * 2011-07-20 2012-02-01 中国华录集团有限公司 High definition video communication system based on high definition set top box
CN103686328A (en) * 2013-12-23 2014-03-26 乐视致新电子科技（天津）有限公司 Method and device for adding camera to smart television
CN105915828A (en) * 2015-12-11 2016-08-31 乐视致新电子科技（天津）有限公司 Split television realization method and split television
CN105357582A (en) * 2015-12-11 2016-02-24 四川长虹网络科技有限责任公司 Method for intelligent set top box to control television set to display monitoring video
CN106941597A (en) * 2016-01-04 2017-07-11 中国移动通信集团公司 A kind of video call method and device
CN106294240A (en) * 2016-07-29 2017-01-04 中国电子科技集团公司第四十研究所 A kind of radio universal serial bus interface system based on Wireless Fidelity

Also Published As

Publication number Publication date
CN110505441B (en) 2022-06-14
CN110505441A (en) 2019-11-26

Similar Documents

Publication Publication Date Title
CN108810636B (en) Video playing method, virtual reality equipment, server, system and storage medium
US8253775B2 (en) Multipoint conference system, multipoint conference method, and program
US20140334381A1 (en) Video streaming in a wireless communication system
WO2011142569A2 (en) Method and apparatus for transmitting and receiving layered coded video
CN110505441B (en) Visual communication method, device and system
JP2011259365A (en) Camera system, video selection apparatus, and video selection method
CN101568027A (en) Method, device and system for forwarding video data
KR101281415B1 (en) System and method for improving home network gui response time and presentation
CN111866526A (en) Live broadcast service processing method and device
CN112203126B (en) Screen projection method, screen projection device and storage medium
CN113132686A (en) Local area network video monitoring implementation method based on domestic linux system
US11871079B2 (en) Client and a method for managing, at the client, a streaming session of a multimedia content
JP2022171421A (en) Video creation device, program production system, and program
CN112135163A (en) Video playing starting method and device
CN114513658B (en) Video loading method, device, equipment and medium
CN110012307A (en) Video transmission method, device and main terminal equipment
WO2022222533A1 (en) Video playing method, apparatus and system, and computer-readable storage medium
CN116360724A (en) Processing method and electronic equipment
JP2015062311A (en) Camera system, image selection apparatus, and image selection method
CN116708885A (en) Screen projection method, screen projection device, connection device, screen projection device and storage medium
CN117440174A (en) Video processing method and device, electronic equipment and storage medium
CN117440175A (en) Method, apparatus, system, device and medium for video transmission
KR20100023490A (en) Apparatus for transmitting-receiving of moving picture
CN117440176A (en) Method, apparatus, device and medium for video transmission
CN113473059A (en) Data processing method, intelligent device and intelligent processing system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination