TW202147833A - Remote-end instant image supporting system and method - Google Patents

Abstract

The present invention relates to a remote-end instant image supporting system, comprising: a local-end image capturing device having a lens to instantly acquire a video signal and applying a streaming media technology to encode and transmit the video signal; a local-end recasting device establishing a wireless communication link with the local-end image capturing device by a wireless communication technology to receive the video signal to decode, applying a web real-time communication technology to re-encode the decoded video signal as a video, and recasting the video; and a remote-end playing device executing an internet browser and establishing a communication link with the local-end recasting device by a communication technology to receive the video, and playing the video in the internet browser.

Description

Remote real-time image support system and method

The present invention relates to a remote real-time image support system, in particular to a remote real-time video stream that receives and decodes video streams through a set of re-casting devices at the local end, encodes them through different real-time communication protocols and then re-sends them to the remote devices. Video support system.

With the popularization of mobile devices, the improvement of hardware performance and the completion of infrastructure, modern people have become accustomed to sharing and interacting with audio and video through mobile devices or OTT (over-the-top) set-top boxes. Information is not traditional pre-recorded or pre-made content, but emphasizes real-time audio-visual interaction. For example, the currently very popular webcast is one of them.

In addition, the Internet instant messaging (IM) software that almost everyone uses, such as Line, Skype, Messenger, Telegram or WeChat, etc., these IMs basically have the function of audio-visual dialogue and interaction, which can be said to be the life of modern people. , has been inseparable from this real-time audio-visual interaction.

At present, most of the main usage modes are that one camera lens is used for a specific application (App), and the real-time image can be displayed and stored on the App, but most of them do not carry out subsequent sharing operations, such as sharing to another through the App. However, the above-mentioned real-time audio-visual interaction method obviously has the following disadvantages:

(1) If you want to communicate, most of them are through the built-in camera of your mobile device or laptop. Or a desktop computer is connected to an external USB camera or IP camera. This mode of use is not suitable for many special application scenarios. There are many difficulties; if you want to use the coil lens to reach into a small space, or to perform detection or maintenance in a location where the mobile phone lens cannot enter, the current audio-visual interaction method cannot be satisfied.

(2) At present, most of the usage methods are mainly for a camera to correspond to an App. The App can display real-time images and store them, but most of them do not carry out subsequent sharing operations, such as sharing to another mobile device or device through the App. superior.

(3) At present, most of the audio-visual interaction is carried out through the audio-visual server. The industry needs to spend more resources for construction and maintenance operations, and the corresponding users need to pay more expensive use costs. In addition, if the audio-visual When it comes to sensitive or private content, users are often discouraged from using it.

For this reason, the inventor, after careful trial and research, and a spirit of perseverance, finally came up with the present "remote real-time image support system and method", which can overcome the above shortcomings. The following is a brief description of the present invention.

In view of the shortcomings of the conventional technology, the present invention proposes a remote real-time image support system, in addition to being provided with a dedicated small hypertext markup language (HTML) platform, which can provide point-to-point video, and can also realize access to the Internet on the terminal equipment. After the image captured by the camera equipment is converted into a WebRTC-P2P video stream, the remote end can view and manage the on-site status and identify the on-site personnel.

The system structure of the present invention is very simple, and the overall system is constructed on the On a device that is available at hand, the implementation of the system of the present invention does not require the cooperation of additional special equipment. The system of the present invention uses an external Wi-Fi camera or a USB camera as the near-end camera device, and uses a smart phone as the near-end recast device with WebRTC And P2P technology, it can implement low-cost and high-efficiency real-time video streaming, and directly share the video and audio from the camera lens to another remote playback device, usually a mobile device or device, such as another smartphone or OTT network set-top box.

Accordingly, the present invention provides a remote real-time image support system, which includes: a near-end camera device, which has a camera to capture video signals in real time, and uses streaming media technology to encode and transmit the video signals; near-end re-casting equipment, It establishes a wireless communication link with the near-end camera device through wireless communication technology, so as to receive and decode the video signal, and apply the web instant communication technology to re-encode the decoded video signal into video and replay the video; and remote playback The apparatus executes a web browser and establishes a communication link with the near-end recasting device via communication technology to receive the video, and play the video in the web browser.

Preferably, the remote real-time image support system further includes: a web page real-time communication server, which acts as a bridge device between the near-end replay equipment and the far-end playback device, so as to provide the near-end real-time image support system. A point-to-point communication link conforming to the web page instant communication technology is established between the replay equipment and the remote playback device.

Preferably, the web page instant messaging server is based on an interactive connection establishment framework and cooperates with the NAT conversation to traverse the application server and the NAT relay traversal server to complete the NAT cross.

Preferably, the near-end re-casting device also includes a video stream conversion frame stream function library element, a special function library element for webpage instant messaging, a device storage function library element, a small hypertext transfer protocol service function library, Video capture program module, video storage program module, video access Program module, image writing file program and image file reading program.

Preferably, the near-end re-projection device converts the video signal into a video with multiple frames through the video stream conversion frame stream library element, and stores the converted video through the device storage function library element for video files.

Preferably, the small-scale hypertext transfer protocol service library is configured to build a small-scale hypertext transfer protocol server on the local area network interface to which the near-end replay device belongs for the remote playback device to access.

The present invention further provides a remote real-time image support method, which includes: capturing a video signal in real time through a near-end camera device including a camera, and applying streaming media technology to encode and transmit the video signal; A wireless communication technology establishes a wireless communication connection with the near-end camera device, so as to receive and decode the video signal, and apply the web instant communication technology to re-encode the decoded video signal into video and replay the video; and use remote playback The device executes a web browser and establishes a communication link with the near-end recasting device via the second communication technology, and plays the video in the web browser.

NE: Proximal

RE: remote

100: Remote real-time image support system of the present invention

110: Proximal camera device

111: Routing device

115: Webcam

116: Camera

120: Proximal Recast Equipment

121: TV

125: STB

126: Mobile Devices

130: Remote playback device

131: Set-top box device

132: Tablet device

133: Smartphone

134: Notebook

135: Desktop Computer

200: Server

301: The first picture

302: Second screen

401: The first picture

402: Second screen

403: Third screen

1300: Remote real-time image support method of the present invention

1301-1303: Implementation steps

11-15: Implementation steps

21-25: Implementation steps

31-32: Implementation steps

41-42: Implementation steps

51-52: Implementation steps

FIG. 1 shows a schematic diagram of the overall system structure of the remote real-time image support system of the present invention.

FIG. 2 and FIG. 3 show schematic diagrams of establishing a point-to-point communication link between the near-end replay device and the far-end playback device according to the present invention.

FIG. 4 shows a time sequence diagram of establishing a P2P communication link through STUN and TURN under the ICE framework of the present invention.

FIG. 5 is a schematic diagram showing the system architecture among various software core components of the present invention.

FIG. 6 is a schematic diagram showing the system structure of the remote playback device of the present invention accessing the login page of the web platform through a browser.

FIG. 7 is a schematic diagram showing the remote playback device of the present invention viewing the image frame captured by the near-end camera device through the browser to view the near-end replay device.

FIG. 8 is a schematic diagram illustrating the video and audio capture program module executed in the near-end re-casting device of the present invention.

FIG. 9 is a schematic diagram showing the video and audio storage program module executed in the near-end replay device of the present invention.

FIG. 10 is a schematic diagram showing the video and audio access program module executed in the near-end replay device of the present invention.

FIG. 11 and FIG. 12 are schematic diagrams showing the split screen of the HTML webpage according to the present invention.

FIG. 13 shows a flowchart of the implementation steps of the remote real-time image support method of the present invention.

The present invention will be fully understood by the following examples, so that those skilled in the art can complete it accordingly. However, the implementation of the present invention is not limited by the following examples; The limitation on size, dimension and scale is not included, and the size, dimension and scale of the present invention are not limited by the drawings of the present invention during the actual implementation of the present invention.

The term "preferably" used herein is non-exclusive and should be understood as "preferably, but not limited to," and any steps described or recited in any specification or claim may be performed in any order, and are not limited to those described in the claim order, the scope of the present invention should be determined only by the appended claims and their equivalents, not by the examples exemplified by the embodiments; the term "comprising" and its Variation, as it appears in the description and claims, is an open-ended term that does not have a restrictive meaning and does not exclude other features or steps.

FIG. 1 shows a schematic diagram of the overall system structure of the remote real-time image support system of the present invention; the remote real-time image support system 100 of the present invention includes a near-end camera device 110 located at the near-end NE or local end and a near-end heavy-duty camera. The projection equipment 120, and the far-end playback device 130 located at the far-end RE, etc., between the near-end camera device 110 and the near-end re-casting device 120, preferably through wireless local area communication for establishing a wireless local area network (WLAN) A communication link is established using technologies, such as Wi-Fi communication technology, Bluetooth (BT) communication technology or Bluetooth Low Energy (BLE) communication technology.

The communication link between the near-end replay device 120 and the far-end playback device 130 is preferably established through a communication technology for establishing a wide area network (WAN), such as Wi-Fi communication technology, Bluetooth communication technology, Bluetooth Bud low-power communication technology, Sub-1G communication technology, 868MHz communication technology, 916MHz communication technology, 926MHz communication technology, NB-IOT communication technology, Zigbee communication technology, Xbee communication technology, Z-Wave communication technology, LoRa communication technology, B Ethernet communication technology, fixed network communication technology, peer-to-peer (P2P) communication technology, telecommunication communication technology, 4G communication technology, 5G communication technology, machine-to-machine (M2M) communication technology, Internet of Things (IoT) communication technology and combinations thereof one.

The near-end camera device 110 is preferably a video streaming broadcasting device, after being activated, a series of captured video signals can be used to form an audio and video stream using a streaming media technology, such as but not limited to Real-Time Transport Protocol (RTP), Protocols such as Real-Time Transmission Control Protocol (RTCP) or Real-Time Streaming Protocol (RTSP), using H.264 encoding format for encoding and compression, and then segmented transmission through the Internet to cast (casting) audio and video streams to Designated device, as long as the user can connect to the Internet, after decoding the audio and video stream using the same protocol, without downloading the media file of the entire video signal, the user can directly watch the video content on the Internet, via Streaming media technology enables data packets of video signals to be transmitted like water.

The near-end camera device 110 covers a camera with a camera on the body without a processing unit such as a CPU, MCU or ECU, etc. Such cameras need to be connected to other host devices, such as but not limited to laptops, smart phones or tablet devices, etc. Play the camera function, such cameras are, for example, but not limited to, a USB camera (USB cam) or a web camera (webcam), etc. The near-end camera device 110 also covers a camera with a camera and a processing unit on the body, and this type of camera can function independently. Camera function, such cameras are, for example, but not limited to, Wi-Fi cameras or IP cameras; the near-end camera device 110 can be deployed in any position, and is basically a non-fixed device.

The near-end re-casting device 120 is preferably a network video stream receiving device, and has the computing capability to support decoding and re-encoding processing between communication protocols. After the audio and video streams are received by the channel, they can be played directly after decoding without downloading any files. Then, the near-end re-casting device 120 decodes the audio and video streams and stores them as video media files, and applies the WebRTC technology to recreate the audio and video streams. The video is encoded and recasted to the remote playback device 130; the near-end recast device 120 covers various media devices with processing units such as CPU, MCU or ECU and network modules, such as but not limited to OTT ( over-the-top) set-top box devices, mobile devices, smartphones, tablet devices, laptops, desktops, etc.; the near-end recast device 120 can be deployed anywhere and is basically a non-stationary device .

The remote playback device 130 is a networked media device, and provides an Internet browser (browser), and when the browser is used to play the video signal stream sent by the near-end replay device 120, the remote playback device 130 Covers a variety of processing units such as CPU, MCU or ECU Devices such as network modules, such as but not limited to set-top box devices, mobile devices, smart phones, tablet devices, notebook computers, desktop computers, etc.; the remote playback device 130 can be deployed in any location, basically is a non-fixed device.

In this embodiment, the near-end camera device 110 , the near-end replay device 120 and the far-end playback device 130 are preferably a Wi-Fi camera, a set-top box and a smart phone, respectively, and the near-end camera device 110 uses Wi-Fi The Fi communication technology establishes a connection with a nearby Wi-Fi router 111, and then connects to the near-end recast device 120. The near-end re-cast device 120 uses the TV 121 as a display, and the near-end re-cast device 120 uses the application The combination of the Ethernet communication technology and the 5G communication technology establishes a communication link with the remote playback device 130 .

Figures 2 and 3 show schematic diagrams of establishing a point-to-point communication link between the near-end replay device and the far-end playback device of the present invention; as shown in Figure 2, the near-end camera device 110 is, for example, but not limited to: Wi-Fi -Fi network camera 115 or USB camera 116, the near-end replay device 120 is, for example, but not limited to, a set-top box 125 or a mobile device 126, and the far-end playback device 130 is, for example, but not limited to, a set-top box device 131, a tablet Device 132 , smartphone 133 , laptop 134 or desktop 135 .

Between the near-end replay device 120 and the far-end playback device 130, preferably a peer-to-peer communication link (P2P) can be used to reduce network resource consumption, but as shown in FIG. 3 It is shown that before the near-end recasting device 120 and the far-end playback device 130 establish a communication link, no one knows the other party, which means that both parties do not know where the other party is. Therefore, the establishment of the P2P communication link initially requires a WebRTC server (WebRTC signaling server) 200 is involved in bridging.

However, once the P2P communication link is established, unless the communication link is interrupted for some reason, the server 200 does not need to intervene in the communication process, but if the communication link is interrupted for some reason, the server 200 does not need to intervene in the communication process. It is necessary to intervene again and exchange the connection information again to reconstruct the P2P communication link. Assuming that the near-end replay device 120 is the set-top box 125, and the far-end playback device 130 is the set-top box 131, the P2P communication link establishment process is as follows stated:

The STB 125 sends a request for connecting with the STB 131 to the server 200 (as in step 11); the server 200 sends the request from the STB 125 to the STB 131 (as in step 12); the STB 131 responds to the server 200, and answers that communication is possible (as in step 13); the server 200 sends the STB 125 the relevant link information of the STB 131, including information such as IP and PORT number (as in step 13). 14); and the set-top box 125 and the set-top box 131 start communication after establishing a connection (as in step 15).

Figure 4 shows the time sequence diagram of the present invention to establish a P2P communication link through STUN and TURN under the ICE framework; in theory, as long as the device can connect to the Internet, a direct connection can be established through the above steps to communicate directly, but In many cases, the above-mentioned P2P communication link cannot be established directly due to problems such as IP masking technology based on Network Address Translation (NAT) or firewalls. Therefore, the Interactive Connection Establishment (ICE) framework of WebRTC must be imported to assist in the ICE framework. Only then can the P2P communication link be successfully established, and the video signal stream of the near-end recast device 120 is streamed to the remote playback device 130 for real-time playback and display.

Through the ICE framework, NAT traversal and other thorny issues are handled through protocols such as NAT Talk Traversal Application (STUN) and NAT Relay Traversal (TURN), when both devices are on their own private networks , connected to the public network through NAT, the private IP of the remote computer cannot be known at this time, so through the STUN server, when the devices at both ends are connected to the STUN server, they can know their respective network IPs, and then through the NAT The server's internal routing table comparison will correspond to the internal private IP, and a P2P communication link can be established at this time.

As shown in Figure 4, when two different near-end recast devices (Peer A) and far-end playback devices (Peer B) want to establish P2P communication, they must know each other's IP and PORT, but if When the two devices, Peer A and Peer B, have no external IP, they need to perform NAT traversal through the STUN server and the TURN server. The signal channel (Signal Channel) transmits the invitation message of device Peer A to device Peer B. When device Peer B receives the invitation SDP from device Peer A from the signal channel, it will immediately answer (Answer) device A, if device Peer B is the answer Agree, at this time, the two computers will become candidates for connection under the ICE framework and start to send messages to each other.

Fig. 5 is a schematic diagram showing the system architecture among various software core components of the present invention; Fig. 6 is a schematic diagram showing the system architecture of the remote playback device of the present invention accessing the login page of the web platform through an executing browser; Fig. 7 is a schematic diagram showing the present invention A schematic diagram of the remote playback device viewing the image captured by the near-end camera device through the browser to view the near-end replay device; this embodiment is described by taking the Android operating system as an example. The near-end camera device 110, Both the near-end replay device 120 and the far-end playback device 130 are compatible devices that use or support the Android operating system. The operating procedures between the devices and the operating system are as follows:

FFmpeg is pre-compiled into a dynamic link library (DLL) component form dedicated to Android App to decode the video stream sent by the near-end camera device 110 as a program library for receiving and converting the internal stream of the near-end re-cast device 120 , which is used for subsequent transmission identification and can be called by the Android App at any time.

Enable the application (App) installed in the near-end re-casting device 120 and executed under the Android operating system, and call the function internally referencing FFmpeg through the native development kit (NDK). The library component then reads in and decodes the audio and video streams sent by the near-end camera device 110, and converts the audio and video streams into realistic frame streams (as in step 21).

Record the frame stream as a video multimedia file through the Android App, and provide it as input for other applications, such as but not limited to: storing on the local recast device 120, and performing images on the local recast device 120 Recognition or face recognition (as in step 22).

The Android App calls the built-in WebRTC dedicated function library component of the Android operating system, inputs the decoded frame stream into the WebRTC dedicated function library, and reconverts the frame stream into an audio and video stream conforming to the WebRTC streaming specification (as in step 23).

The near-end replay device 120 establishes a connection with the WebRTC server 200 through the WebRTC dedicated function library, and waits for a request from the remote playback device 130 to establish a P2P communication link (as in step 24); and the near-end replay device 120 agrees to the request , and from the near-end recasting device 120 via the P2P communication link, the video stream from the near-end camera device 110 is decoded and re-encoded, and then re-casts the video and audio stream from the near-end camera device 110 to the far-end playback device 130 flow (as in step 25).

The user of the remote playback device 130 does not need to install a special application program, but only needs to operate a browser to log in to a specific website platform, as shown in FIG. As shown in FIG. 7 , the video and video contents played on the near-end re-casting device 120 and the far-end playback device 130 are highly consistent.

The functions called by the RLR device 120 through the WebRTC dedicated function library are listed as follows:

MediaStream (ie getUserMedia): Get the stream of video and sound from the camera and microphone.

RTCPeerConnection: Responsible for connection establishment, data encryption and bandwidth management.

RTCDataChannel: responsible for transmitting general data.

FIG. 8 is a schematic diagram showing the video and audio capture program module executed in the near-end re-casting device of the present invention; the video and audio capture program module executed in the near-end re-projection device 120 , and its operating procedure is as follows:

The Android App concatenates the function library components internally referencing FFmpeg through the NDK, and then reads and decodes the video and audio streams sent by the near-end camera device 110 (as in step 31); and the FFmpeg function library components decode the video and audio streams into Realistic frame (frame) stream, and converted to MP4 format (as in step 32).

The video capture program module is responsible for capturing video or audio from MP4 video stream, including obtaining the parameter configuration (Parameter Configuration) of video and audio, as well as video and audio encoding/decoding (Encoding/Decoding) and other functions, and the call is as follows function:

class: public class EM_PlaybackObject

function:

(1) Constructor function EM_PlaybackObject is used to create a PlaybackObject.

(2) void encode()--encode the audio and video of the PlaybackObject.

(3) void decode()--decode the audio and video of PlaybackObject.

FIG. 9 is a schematic diagram showing the video and audio storage program module executed in the near-end re-casting device of the present invention; the video and audio storage program module executed in the near-end re-casting device 120, its operating procedure is as follows:

Android App calls the built-in Device Storage library component of the Android operating system, which is used for internal or external file access and management to obtain files Store space information, coordinate necessary storage locations, and access MP4 format data to prepare for storage (as in step 41); and store the library component through the device, and the near-end replaying device 120 selects the MP4 format data to be stored locally. The internal storage is an MP4 file, or the external storage is performed on an external expansion device as an MP4 file (as in step 42).

The video storage program module includes the video writing file program, which is responsible for writing the captured MP4 video streaming data to the local file, so that it can be read, replayed or transferred to other networked TVs, set-top boxes or other mobile devices, and call the following functions:

class: public class EM_PlaybackObject

function:

(1) void createEMPO( )--called by the Constructor to create an object.

(2) String[ ]getFileInfo( )--Get file information.

(3) int saveRecord( )--Write the current video and audio data into the file.

(4)int closeEMPO( )--Close the file audio and video file reading module.

The video storage program module includes the video file reading program, which is responsible for reading the previously archived MP4 video streaming data from the file, replaying or transferring it to other networked TVs, set-top boxes or other mobile devices according to user needs, and Call the following function:

class: public class EM_PlaybackObject

function:

(1)void openEMPO( )--Open an existing file as an object

(2) String[ ]getFileInfo( )--get file information

(3)int readRecord( )--read the next audio and video data

(4)int seek(int position)--jump to the file position

(5)int closeEMPO( )--close the file

FIG. 10 is a schematic diagram showing the video and audio access program module executed in the near-end re-casting device of the present invention; when the remote playback device 130 requests the near-end re-casting device 120 to access and view the MP4 file through the P2P communication link, Its operating procedure is as follows:

The Android App calls a small Hypertext Transfer Protocol (HTTP) service library component, which calls a series of HTTP Server and Client programming interface frameworks through AndroidAsync, for building a small HTTP on the local area network interface of the near-end replay device 120 The server is accessed by the remote playback device 130 (as in step 51); the Android App calls and requests the device storage library element to access MP4 files in internal storage or external storage through a small HTTP service library element (eg Step 52); and the user of the remote playback device 130 can view the video content contained in the MP4 file stored in the near-end replay device 120 by browsing the HTTP service through the browser tool software supporting the HTML5 standard.

FIG. 11 and FIG. 12 are schematic diagrams showing the split screen of the HTML webpage according to the present invention; when the user views the MP4 file stored in the near-end replay device 120 or the real-time image from the near-end camera device 110 through the browser, the browser Access and open the small HTTP server built on the local area network interface of the near-end recasting device 120, and access the specified HTML page. This HTML page is designed to be divided into multiple screens. As shown in Figure 11, the HTML The webpage is divided into two screens, the first screen 301 at the top is a screen for displaying MP4 files, and the second screen 302 at the bottom provides an instant messaging program interface, which allows the near-end replaying device 120 and the near-end camera device 110 to communicate with each other. The operator and the user of the remote playback device 13 conduct a real-time dialogue.

The HTML webpage shown in FIG. 12 is divided into three screens, the first screen 401 at the top is a screen for displaying MP4 files, and the second screen 402 in the middle is provided as an electronic advertisement (EDM) display space, the lower third screen 403 provides a remote control interface, allowing the user of the remote playback device 13 to control the equipment located at the near end.

In the remote real-time image support system 100 proposed by the present invention, the near-end camera device 110 adopts an external Wi-Fi camera or a USB camera, and is equipped with WebRTC and P2P communication links, which can implement low-cost and high-efficiency real-time video streaming replay Technologies, relevant contextual applications are as follows but not limited to the following.

Face recognition: Using an OTT set-top box device and a USB camera, and running the face recognition program of TensorFlow Lite on the remote playback device 130, and the OTT set-top box is further connected with a set of electronic locks, it can be used for low-cost and high-efficiency processing. Face recognition, the system automatically unlocks the electronic lock directly from the remote end.

Remote equipment maintenance: Use any smart phone with a micro-USB camera. The micro-USB camera can be fixed or extended with a coiled tube. The length of the connection is nearly one meter. It is the details. The real-time image is sent to the senior technicians at the remote end, so that the senior technicians can view the equipment status in real time and guide the on-site personnel to carry out troubleshooting or related maintenance matters. The HTML page is shown in Figure 11.

Remote video surveillance: Wi-Fi camera is used as the near-end camera device, the user's smartphone is used as the far-end playback device, and through the gateway (gateway) with other sensors, regional monitoring and control can be performed, such as monitoring potted plants and gardens , sprinkle water if necessary to achieve the purpose of remote monitoring, and its HTML page is shown in Figure 12.

Smart home security home security: use the Wi-Fi camera as the door surveillance camera, and through the gateway (gateway) with other sensors, the image obtained by the Wi-Fi camera is used as the home security portal image source, and other controls can be performed later, such as But not limited to:

Gas safety: When you go out with a temperature sensor, you can know whether the gas is turned off through the image, and you can also transfer the image to the security personnel. Fire alarm: With the alarm, if an alarm occurs, you can understand the current situation through artificial intelligence sound and image recognition. Safe security: with PIR sensor, if someone approaches, you can send a notification and understand the current situation through the image.

FIG. 13 shows a flowchart of the implementation steps of the remote real-time image support method of the present invention; in summary, the present invention also includes a remote real-time image support method 1300, preferably including the following steps: real-time capture through a near-end camera device including a camera Obtain the video signal, encode and transmit the video signal using streaming media technology (step 1301); use the near-end re-casting device to establish a wireless communication link with the near-end camera device via the first wireless communication technology to receive and decode the video signal, and apply the web instant messaging technology to re-encode the decoded video signal into video and replay the video (step 1302); use the remote playback device to execute a web browser and replay the near-end via the second communication technology The device establishes a communication link and plays the video in the web browser (step 1303).

To sum up, the present invention proposes to establish a P2P real-time video sharing and communication platform, which is based on the cloud service platform (PaaS) technology, and then extends to establish a video sharing module through the WebRTC P2P streaming technology, thereby expanding the existing PaaS platform. and the remote real-time image support system proposed by the present invention supports web browsers for real-time voice or video, and supports various current mainstream mobile device platforms (Android/iOS/Windows), with cross-platform integration effect.

The Wi-Fi camera and USB camera deployed at the near end in the system transmit the video and audio signals to the near-end recast device through a wireless or USB cable. The location of the playback device, after which WebRTC is no longer required The server directly transmits and shares the near-end video and audio to the remote playback device through P2P mode, especially the near-end replay device can be an OTT set-top box device, which can penetrate into the family living room or public area, while the mobile device can enter field for personal use.

The above embodiments of the present invention can be arbitrarily combined or replaced with each other, so as to derive more implementation aspects, but none of them deviate from the scope of protection of the present invention. More embodiments of the present invention are further provided as follows:

Embodiment 1: A remote real-time image support system, which includes: a near-end camera device, which has a camera to capture video signals in real time, and uses streaming media technology to encode and transmit the video signals; Establishing a wireless communication link with the near-end camera device through wireless communication technology to receive and decode the video signal, and apply the web instant communication technology to re-encode the decoded video signal into video and replay the video; and a remote playback device , which executes a web browser and establishes a communication link with the near-end recasting device via communication technology to receive the video and play the video in the web browser.

Embodiment 2: The remote real-time image support system as described in Embodiment 1, further comprising: a web page real-time communication server, which acts as a bridge device between the near-end replay device and the remote playback device to provide A point-to-point communication link conforming to the web page instant communication technology is established between the near-end replay device and the far-end playback device.

Embodiment 3: The remote real-time image support system as described in Embodiment 2, wherein the web instant messaging server is based on the interactive connection establishment framework and cooperates with the network address translation dialogue to traverse the application server and the network location. The address translation relay traverses the program server to complete the network address translation traversal.

Embodiment 4: The remote real-time image support system as described in Embodiment 1, wherein the The near-end replay equipment also includes video and audio stream conversion frame stream function library components, special function library components for webpage instant messaging, device storage function library components, small hypertext transfer protocol service function library, and video and audio capture program module. group, video storage program module, video access program module, video write file program, and video file read program.

Embodiment 5: The remote real-time image support system as described in Embodiment 4, wherein the near-end reprojection device converts the video signal into a video with a plurality of frames through the video stream conversion frame stream library component, And the converted video is stored as a video file through the device storage library element.

Embodiment 6: The remote real-time image support system as described in Embodiment 4, wherein the small hypertext transfer protocol service function library is configured to build a small hypertext transfer protocol on the local area network interface to which the near-end replay device belongs. TTP server for remote playback device to access.

Embodiment 7: The remote real-time image support system according to Embodiment 1, wherein the near-end camera device is a Wi-Fi network camera, an IP camera, a network camera or a USB camera, and the near-end re-casting device is a camera. Top box device, mobile device, smart phone, tablet device, notebook computer, desktop computer or processing storage unit device, the remote playback device is a set-top box device, mobile device, smart phone, tablet device, notebook computer or desktop computer.

Embodiment 8: The remote real-time image support system as described in Embodiment 1, wherein the streaming media technology is RTSP, RTCP or RTSP.

Embodiment 9: The remote real-time image support system according to Embodiment 1, wherein the first wireless communication technology is Wi-Fi communication technology, Bluetooth communication technology or Bluetooth low power communication technology, and the second communication technology is The technology department is Wi-Fi communication technology, Bluetooth communication technology, Bluetooth low power communication technology, Sub-1G communication technology, 868MHz communication technology, 916MHz communication technology, 926MHz Communication technology, NB-IOT communication technology, Zigbee communication technology, Xbee communication technology, Z-Wave communication technology, LoRa communication technology, Ethernet network communication technology, fixed network communication technology, point-to-point communication technology, telecommunication communication technology, 4G communication technology, 5G communication technology, machine-to-machine communication technology, Internet of Things communication technology and one of their combinations.

Embodiment 10: A remote real-time image support method, comprising: capturing video signals in real time through a near-end camera device including a camera, and applying streaming media technology to encode and transmit the video signals; A wireless communication technology establishes a wireless communication connection with the near-end camera device to receive and decode the video signal, and apply the web instant communication technology to re-encode the decoded video signal into video and replay the video; and use remote playback The device executes a web browser and establishes a communication link with the near-end recasting device via the second communication technology, and plays the video in the web browser.

The various embodiments of the present invention can be arbitrarily combined or replaced with each other, so as to derive more implementation modes, but none of them deviate from the intended protection scope of the present invention. The recorder shall prevail.

NE: Proximal

RE: remote

100: Remote real-time image support system of the present invention

110: Proximal camera device

111: Routing device

120: Proximal Recast Equipment

121: TV

130: Remote playback device

Claims (10)

A remote real-time image support system, comprising: a near-end camera device, which has a camera to capture a video signal in real time, and encodes and transmits the video signal using a stream of streaming media technology; A near-end re-casting device, which establishes a wireless communication connection with the near-end camera device through a wireless communication technology to receive and decode the video signal, and applies a web instant communication technology to re-encode the decoded video signal into a video and replay the video; and A remote playback device executes a web browser and establishes a communication link with the near-end recasting device via a communication technology to receive the video and play the video in the web browser. The remote real-time image support system as described in claim 1, further comprising: A web-based instant messaging server, which acts as a bridge device between the near-end replay device and the far-end playback device, to provide the establishment of conformance to the web page between the near-end re-cast device and the far-end playback device One of the instant messaging technologies is a point-to-point communication link. The remote real-time image support system as claimed in claim 2, wherein the web instant messaging server is based on an interactive connection establishment framework and cooperates with a network address translation dialogue through the application server and a network address The network address translation traversal is completed by switching the relay traversal process server. The remote real-time image support system according to claim 1, wherein the near-end re-casting device further comprises an audio-video stream conversion frame stream function library element, a webpage instant messaging dedicated function library element, and a device storage function library components, a small hypertext transfer protocol service library, a video and audio capture A program module, an audio-visual storage program module, an audio-visual access program module, an image writing file program, and an image file reading program. The remote real-time video support system of claim 4, wherein the near-end reprojection device converts the video signal into a video with a plurality of frames through the video stream conversion frame stream library component, and converts the video signal into a video with a plurality of frames through the video stream conversion frame stream library component. The device storage library component stores the converted video as a video file. The remote real-time video support system of claim 4, wherein the small hypertext transfer protocol service library is configured to build a small hypertext transfer protocol server on the local area network interface to which the near-end replay device belongs device for remote playback device access. The remote real-time image support system as claimed in claim 1, wherein the near-end camera device is a Wi-Fi network camera, an IP camera, a network camera or a USB camera, and the near-end recast device is a A set-top box device, a mobile device, a smart phone, a tablet device, a notebook computer, a desktop computer or a processing storage unit device, the remote playback device is a set-top box device, a mobile device, A smartphone, a tablet device, a laptop or a desktop computer. The remote real-time image support system of claim 1, wherein the streaming media technology is a real-time transmission protocol, a real-time transmission control protocol or a real-time streaming protocol. The remote real-time image support system according to claim 1, wherein the first wireless communication technology is a Wi-Fi communication technology, a Bluetooth communication technology or a Bluetooth low power communication technology, and the second communication technology It is a Wi-Fi communication technology, a Bluetooth communication technology, a Bluetooth low power communication technology technology, a Sub-1G communication technology, a 868MHz communication technology, a 916MHz communication technology, a 926MHz communication technology, a NB-IOT communication technology, a Zigbee communication technology, an Xbee communication technology, a Z-Wave communication technology, a LoRa Communication technology, Ethernet network communication technology, fixed network communication technology, point-to-point communication technology, telecommunication communication technology, 4G communication technology, 5G communication technology, machine-to-machine communication technology, Internet of Things communication technology and one of its combinations. A remote real-time image support method, comprising: Capture a video signal in real time through a near-end camera device including a camera, and apply a stream of streaming media technology to encode and transmit the video signal; Use a near-end re-casting device to establish a wireless communication connection with the near-end camera device through a first wireless communication technology to receive and decode the video signal, and apply a web instant communication technology to re-encode the decoded video signal into a video and recast the video; and A remote playback device is used to execute a web browser, and a communication link is established with the near-end recasting device through a second communication technology, and the video is played in the web browser.

Images