WO2017111183A1

WO2017111183A1 - Portable radio device and method for providing real-time mirroring using same

Info

Publication number: WO2017111183A1
Application number: PCT/KR2015/014040
Authority: WO
Inventors: 김용진; 임희섭
Original assignee: 모다정보통신 주식회사
Priority date: 2015-12-21
Filing date: 2015-12-21
Publication date: 2017-06-29

Abstract

Disclosed are a portable radio device and a method for providing mirroring using the same. The portable radio device according to one embodiment of the present invention provides a mirroring service regarding a source device, which provides a networking service to a plurality of sync devices by using a network resource of the source device and stream in real-time screen data and audio data received from the source device to the plurality of sync devices in a multicasting manner.

Description

Portable wireless device and method for providing real time mirroring using same

The present embodiment relates to a portable wireless device and a method for providing real-time mirroring using the same.

The contents described below merely provide background information related to this embodiment and do not constitute a prior art.

While the performance of mobile devices is evolving dramatically, the small screen is still a significant drawback. Accordingly, a technology that enables users to naturally view images viewed on a mobile device on a large screen like a TV is attracting attention. In general, a device that provides an image is called a source device or a master device, and a device that receives an image is called a sink device or a client device.

Typical technologies include Daas (Display as a Service), Miracast, Airplay, and Chromecast. These technologies support screen casting or mirroring, which displays the floating screen on the source device as it is on the sink device. In addition, unlike the method of transmitting the entire screen information, these technologies support a method of transmitting the video signal of the content or sending the content file to the remote device by streaming to play it on the remote device. It also supports casting technology that sends Internet address and playback information of content stored online and in the cloud to an HDMI dongle connected to a sink device.

However, these technologies are based on the premise that the source device has a wireless communication (eg, Wi-fi) function so that the source device can connect with the sink device, and mainly deals with a form in which one source device is connected to one sink device.

The present embodiment provides a portable wireless device mounted on a source device to provide a wireless networking service using a network resource of the source device and providing a mirroring service for the source device to a plurality of sink devices, and a method of providing the same. The purpose is to.

According to an aspect of the present invention, a wired communication interface capable of wired communication with a source device; A wireless communication interface capable of wirelessly communicating with sink devices located nearby; A data transmission / reception module configured to receive screen data and sound data of the source device through the wired communication interface; A device management module that controls connection with sink devices through the wireless communication interface and manages connection information of connected sink devices; A signal processing module for generating streaming data by encoding the screen data and the sound data received from the source device into a preset format; A streaming transmission module for managing a streaming service provided to each sink device; And providing a wireless networking service to sink devices connected through the wireless communication interface using network resources of the source device, and multicasting the streaming data to sink devices based on a control command of the streaming transmission module. And a proxy module for streaming in a fashion.

Embodiments of the portable wireless device may further include one or more of the following features.

In some embodiments, the portable wireless device further comprises a mode management module for managing an operating mode in which the portable wireless device operates, wherein the operating mode is a first that simultaneously provides a wireless networking service and the streaming service. Mode, and a second mode providing only the streaming service.

In some embodiments, the mode management module determines the operation mode based on at least one of a user's setting, a source device's property, or a sink device's property.

In some embodiments, the streaming transmission module, in the first mode, performs an interactive procedure for the real-time streaming service with each sink device connected to the wireless communication interface, and the interactive procedure is successfully performed. Control the proxy module to provide the streaming data only to sink devices.

In some embodiments, the streaming transmission module, in the second mode, all sink devices connected to the wireless communication interface without an interactive procedure for a real time streaming service with each sink device connected to the wireless communication interface. Control the proxy module to send the streaming data to the client.

In some embodiments, the mode management module receives a control command for the operation mode from either the source device or the sink devices, and determines the operation mode according to the received control command.

In some embodiments, the mode management module determines whether the source device is communicatively connected to a network, and determines the operation mode according to the determination result.

In some embodiments, the portable wireless device is powered using power delivered from the source device via the wired interface.

According to another aspect of the present invention, in the method of operating a portable wireless device having a wired communication interface capable of wired communication with a source device and a wireless communication interface capable of wirelessly communicating with sink devices located in the vicinity, the wireless communication Obtaining information about sink devices connected through an interface; Receiving screen data and sound data from the source device; Generating streaming data by encoding screen data and sound data received from the source device in a preset format; Providing a wireless networking service to sink devices connected through the wireless communication interface using network resources of the source device; Receiving screen data and sound data from the source device, encoding the received screen data and sound data into a preset format, and streaming the streaming data to the sink devices in a multicast manner. A method of operating a wireless device is provided.

According to another aspect of the invention, the wired communication interface capable of wired communication with the source device; A wireless communication interface capable of wirelessly communicating with sink devices located nearby; And providing a networking service to a plurality of sink devices connected through the wireless communication interface using network resources of the source device, and encoding screen data and voice data received from the source device in a preset format and streaming. Provided is a portable wireless device including a controller for generating data and real-time streaming the generated streaming data in a multicasting manner.

The portable wireless device according to an exemplary embodiment of the present invention may provide a networking service to a plurality of sink devices by using network resources of a source device, and may display screen data and voice data received from the source device. It is possible to provide a mirroring service for the source device to the real-time streaming in a multicasting manner.

1 is a diagram illustrating a mirroring system according to an embodiment of the present invention.

2 is a block diagram schematically illustrating a configuration of a portable wireless device according to an embodiment of the present invention.

3 is a flowchart illustrating a method of operating a portable wireless device according to an embodiment of the present invention.

4 is a flowchart illustrating a method of operating a portable wireless device in AP / streaming mixed mode according to one embodiment of the invention.

5 is a flowchart illustrating a method of operating a portable wireless device in a multicast-only mode according to an embodiment of the present invention.

This study was conducted as part of the R & D project of the Ministry of Science, ICT and Future Planning and the Information and Communication Technology Promotion Center. [13-911-04-006, Development of Social M2M Service]

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In addition, in describing the present invention, when it is determined that the detailed description of the related well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

In addition, in describing the component of this invention, terms, such as 1st, 2nd, A, B, (a), (b), can be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. Throughout the specification, when a part is said to include, 'include' a certain component, which means that it may further include other components, except to exclude other components unless otherwise stated. . In addition, as described in the specification. The terms 'unit' and 'module' refer to a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software.

As shown in FIG. 1, the mirroring system may allow a source device 100, a portable access point 200, and a plurality of

sink devices

300a and 300b to communicate with each other. Are connected and built.

The source device 100 is connected to communicate with the network. In addition, the source device 100 is also connected to communicate with the portable wireless device 200. The network may be formed according to various wired and wireless communication network standards such as LAN, Wireless LAN, LTE, and WiBro.

The portable wireless device 200 is a portable device such as a USB dongle, and is connected to the source device 100 through, for example, a USB interface, and indirectly connected to a network through the source device 100. The portable wireless device 200 is connected to communicate with the

sink devices

300a and 300b wirelessly.

The portable wireless device 200 provides an open Wi-Fi AP function by utilizing the network resources of the source device 100. Accordingly, the portable wireless device 200 uses a network to which the source device 100 is connected as a backbone network of the

sink devices

300a and 300b. In addition, the portable wireless device 200 receives screen data and sound data being displayed on the source device 100 from the source device 100 and transmits the screen data and sound data to the plurality of

sink devices

300a and 300b. Streams screen data and sound data in real time. That is, the portable wireless device 200 provides a real time streaming service for the source device 100 to the plurality of

sink devices

300a and 300b.

The portable wireless device 200 may operate in an AP / streaming mixed mode or a streaming only mode based on a user's setting or attributes of the source device 100 or the

sink devices

300a and 300b. In the streaming only mode, the portable wireless device 200 may stream audio data or video data received from the source device 100 to the

sink devices

300a and 300b connected in a wireless communication in real time. Provide mirroring service. In the AP / streaming mixed mode, portable wireless device 200 provides wireless networking services (ie, AP functionality) to sink

devices

300a and 300b using a wireless communication interface. At the same time, the portable wireless device 200 provides the

sink devices

300a and 300b with a mirroring service for the source device 100 that performs real-time streaming of screen data and sound data of the source device 100.

The mirroring system according to the present invention does not transmit or stream the content itself owned or reproduced by the source device 100 to the

sink devices

300a and 300b, but the displayed screen data and the reproduced sound data of the source device 100. Note that it transmits. In addition, the portable wireless device 200 according to the present invention is connected to the source device 100 instead of the

sink devices

300a and 300b, and connects a multimedia device such as a TV not equipped with a Wifi function to the source device 100. It is not used to do that.

Such a mirroring system is particularly useful in a situation in which it is necessary to share a screen and sound played in one multimedia device with a plurality of multimedia devices. For example, when a portable wireless device is mounted on a TV, the TV serves as a source device. While watching a TV located in the living room, the user may continue to watch the TV screen by using another multimedia device that may serve as a sink device located in a study or a bathroom. As another example, a presenter's laptop may be equipped with a portable wireless device, even if the meeting venue does not have facilities to display meeting materials (eg documents, video, etc.) such as large monitors or projectors. The notebook acts as a source device. In this case, other attendees can play the same data on their laptop or mobile phone as the meeting material played on the presenter's laptop.

As shown in FIG. 2, the portable wireless device 200 according to the present embodiment includes a USB interface 210, a wireless communication interface 220, and a controller 230.

The USB interface 210 is connected to a source device in a USB manner to provide a communication interface between the portable wireless device 200 and the source device. In some implementations, the USB interface 210 is connected to the source device via a cable. In some other implementations, the portable wireless device 200 is implemented to insert the USB terminal of the USB interface 210 directly into the USB socket of the source device. The portable wireless device 200 is driven using power delivered from the source device via the USB interface 210. In some embodiments, the portable wireless device 200 may be configured to have a charging circuit (not shown) and to charge a rechargeable battery (not shown) using an external power source. In another embodiment, the USB interface 210 may be implemented with an HDMI interface.

The wireless communication interface 220 is connected to communicate with the sink devices wirelessly. The wireless communication interface 220 may be implemented according to various wireless communication standards such as WLAN (WiFi), WPAN, Bluetooth, HSDPA, Wibro, and the like.

The controller 230 utilizes network resources of the source device connected through the USB interface 210 to open the Wi-Fi AP function (ie, wireless networking service) to sink devices connected through the wireless communication interface 220. To provide. That is, the controller 230 uses the network to which the source device is connected as the backbone network of the sink devices. In addition, the controller 230 receives the screen data being displayed on the source device and the sound data being played back from the source device through the USB interface 210, and transmits the screen data and sound data of the source device to the plurality of sink devices. Stream live.

As shown in FIG. 2, in some embodiments, the controller 230 may include a data transmission / reception module 231, a device management module 232, a signal processing module 233, a streaming transmission module 234, and a proxy module. module 235, and a mode management module 236.

The data transmission / reception module 231 requests screen data and sound data from a source device connected through the USB interface 210 and receives screen data and sound data from the source device.

The device management module 232 controls connection of sink devices recognized through the wireless communication interface 220 and manages connection information such as MAC address and IP address of the connected sink devices.

The signal processing module 233 encodes the screen data and the sound data received from the source device corresponding to the sink device requesting the real-time streaming service. In some embodiments, the source device may compress or code the screen displayed on the monitor and the sound output from the speaker to be transmitted to the portable wireless device 200, in which case the signal processing module 233 may decode the received data. After that, it may be encoded in a specific format corresponding to the sink device. In some embodiments, the signal processing module 233 may include a voice signal processing module (not shown) and an image signal processing module (not shown) for processing screen data and sound data, respectively.

The streaming transmission module 234 manages a streaming service provided to each sink device. For example, control is performed on starting, stopping, and ending a streaming service, and determining a target device to provide a service.

In the AP / streaming mixed mode, the streaming transmission module 234 performs an interactive procedure for the real time streaming service with each sink device connected to the wireless communication interface 220, and the sink device for which the interactive procedure has been successfully performed. The proxy module 235 to provide streaming data only to them. The streaming transmission module 234 may determine whether to provide a real time streaming service in response to each streaming service request received from the sink devices. For example, the streaming transmission module 234 may determine whether to approve each streaming service request in consideration of the number of sink devices using a streaming service, a data transmission rate, and the like. In some embodiments, the request message for requesting the streaming service may include data about a port not being used by the sink device or a port designated by the user of the sink device. In this case, the streaming transmission module 234 opens a User Datagram Protocol (UDP) socket to a port included in the request message. Messages (eg, streaming stop request, termination request, etc.) generated in the process of providing a streaming service are transmitted through a socket.

In the streaming only mode, the streaming transmission module 234 transmits the streaming data to all the sink devices connected to the wireless communication interface 220 without an interactive procedure for the real time streaming service with each sink device. Control module 235. However, in some other embodiments, the streaming transmission module 234 may perform an interactive procedure for the real-time streaming service with each sink device even in the streaming only mode.

The proxy module 235 performs a relay function (ie, a wireless networking service) that allows sink devices connecting through the wireless communication interface 220 to indirectly access the network of the source device through the portable wireless device 200. do. In addition, the proxy module 235 performs a streaming service provided to the plurality of sink devices according to a control command transmitted from the streaming transmission module 234. For example, the proxy module 235 stores the streaming data delivered from the streaming transmission module 234 in a cache and distributes the multicast method to a plurality of sink devices that are the targets of the streaming service. Further, the proxy module 235 provides only a streaming service or a wireless networking service according to a streaming only mode or an AP / streaming mixed mode, and simultaneously provides a streaming service.

The mode management module 236 manages a mode in which the portable wireless device 200 operates (ie, an AP / streaming mixed mode or a streaming only mode) based on a user's setting or a property of a source device or a sink device.

In some embodiments, the user may set the mode of the portable wireless device 200 through an application installed on the source device or the sink device, and the mode management module 236 may adjust the mode setting according to the user input from the source device or the sink device. According to the instructions, the operating mode of the portable wireless device 200 can be determined.

In some other embodiments, the mode management module 236 determines whether the source device is communicatively connected to the network, and operates in the AP / streaming mixed mode or the streaming only mode according to the determination result. That is, if it is determined that the source device is not communicatively connected to the network, the mode management module 236 sets the streaming only mode, while if it is determined that the source device is communicatively connected to the network, the mode management module ( 236 sets to the AP / streaming mixed mode.

In some other embodiments, the mode management module 236 operates according to the nature of the sink device (eg, whether it has other communication capabilities, such as LTE, LAN, or whether other networking services are required while using the streaming service). Determine the mode to do.

When the portable wireless device is connected to the source device through the USB interface, it is powered on and activates the wireless communication interface (S310). The portable wireless device is connected to one or more sink devices via an activated wireless communication interface. The portable wireless device manages connection information such as MAC addresses and IP addresses of recognized sink devices.

As described above, the portable wireless device may operate in an AP / streaming mixed mode and a streaming service only mode. The portable wireless device determines whether the current operation mode is a streaming service-only mode (S320), and provides only a real-time streaming service or a real-time streaming service simultaneously with a wireless networking service according to the determination result. The portable wireless device may determine a mode to operate based on the user's setting or the attributes of the source device or the sink device.

In the AP / streaming mixed mode (No in S320), the portable wireless device provides a wireless networking service (ie, AP function) to the sink device using a wireless communication interface (S330). At the same time, the portable wireless device provides the sink device with a mirroring service for the source device, which streams screen data and sound data of the source device in real time. For example, when a request message for requesting a live streaming service is received from one or more sink devices while providing a wireless networking service, the portable wireless device multicasts the received data while receiving screen data and sound data from the source device. Live streaming in a (Muiticast) method (S340 ~ S350).

In the streaming service only mode (YES in S320), the portable wireless device does not provide a wireless networking service (ie, an AP function), but provides only a real time streaming service (S350).

In the AP / streaming mixed mode, the portable wireless device provides a wireless networking service (ie AP function) to one or more sink devices using a wireless communication interface. At the same time, the portable wireless device provides a mirroring service for the source device to the sink device requesting the streaming service in real time streaming the screen data and the sound data of the source device.

When the portable wireless device requests a network IP address from the sink device, the portable wireless device configures a wireless network such as assigning a network IP address to the sink device and provides wireless networking service to the sink device. That is, the portable wireless device performs an open AP function by using the network resource of the source device.

When receiving a request message for requesting a streaming service from the sink device while providing the wireless networking service, the sink device responds with a success or failure to the multicast request.

The portable wireless device requests real time screen data and sound data from the source device, encodes data received from the source device with a predefined codec, and transmits the data to the sink device in the form of a streaming media packet. The sink device receiving the streaming plays back the received packets using a media codec through a predetermined application.

The streaming transmission to the sink device stops streaming transmission to the terminal when there is a request to stop transmission from the sink device (or when the application of the sink device is terminated).

In the streaming only mode, the streaming service is immediately started between the portable wireless device and the sink device without an interactive procedure such as a streaming service request and a response thereto.

When the portable wireless device requests a network IP address from the sink device, the portable wireless device establishes a wireless network connection by assigning a network IP address to the sink device. Once the wireless network connection is established, the portable wireless device transmits real-time picture data and sound data received from the source device to the sink device in the form of streaming media packets.

Such a transmission scheme is useful when the sink device cannot install a specific application for using the mirroring service, such as a TV, and thus cannot perform an interactive procedure with the portable wireless device. In addition, since the mobile device does not transmit or receive data for a wireless networking service (i.e., AP function), the sink device does not need to filter (or an application for) streaming media packets from data received from the portable wireless device. Can be used.

The above description is merely illustrative of the technical idea of the present embodiment, and those skilled in the art to which the present embodiment belongs may make various modifications and changes without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment but to describe the present invention, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

100: source device 200: portable wireless device

300a, 300b: Sink device 210: USB interface

220: wireless communication interface 230: control unit

231: data transmission / reception module 232: device management module

233: signal processing module 234: streaming transmission module

235: proxy module 236: mode management module

Claims

A wired communication interface capable of wired communication with a source device;

A wireless communication interface capable of wirelessly communicating with sink devices located nearby;

A data transmission / reception module configured to receive screen data and sound data of the source device through the wired communication interface;

A device management module that controls connection with sink devices through the wireless communication interface and manages connection information of connected sink devices;

A signal processing module for generating streaming data by encoding the screen data and the sound data received from the source device into a preset format;

A streaming transmission module for managing a streaming service provided to each sink device; And

Provides a wireless networking service to the sink devices connected through the wireless communication interface using the network resources of the source device, and multicasts the streaming data to the sink devices based on a control command of the streaming transmission module. Proxy module for streaming to

Including, portable wireless device.
The method of claim 1,

And a mode management module for managing an operation mode in which the portable wireless device operates.

Wherein the operation mode includes a first mode for simultaneously providing the wireless networking service and the streaming service, and a second mode for providing only the streaming service.
The method of claim 2,

The mode management module,

And determine the operation mode based on at least one of a user's setting, a source device's properties, or a sink device's properties.
The method of claim 2,

In the first mode,

The streaming transmission module,

Performing an interactive procedure for the real-time streaming service with each sink device connected to the wireless communication interface, and controlling the proxy module to provide the streaming data only to the sink devices for which the interactive procedure has been successfully performed. A portable radio device.
The method of claim 2,

In the second mode,

The streaming transmission module,

And the proxy module is configured to transmit the streaming data to all sink devices connected to the wireless communication interface without an interactive procedure for a live streaming service with each sink device.
The method of claim 2,

The mode management module,

And a control command for the operation mode from one of the source device or the sink devices, and determining the operation mode according to the received control command.
The method of claim 2,

The mode management module,

And determining whether the source device is communicatively connected to a network, and determining the operation mode according to the determination result.
The method of claim 1,

And is driven using power delivered from the source device via the wired interface.
A method of operating a portable wireless device having a wired communication interface capable of wired communication with a source device and a wireless communication interface capable of wirelessly communicating with sink devices located nearby,

Acquiring information about sink devices connected through the wireless communication interface;

Receiving screen data and sound data from the source device;

Generating streaming data by encoding screen data and sound data received from the source device in a preset format;

Providing a wireless networking service to sink devices connected through the wireless communication interface using network resources of the source device;

Receiving screen data and sound data from the source device, encoding the received screen data and sound data into a preset format, and streaming the streaming data to the sink devices in a multicast manner;

A method of operating a portable wireless device comprising a.
The method of claim 9,

Determining an operation mode in which the portable wireless device operates based on at least one of a user's setting, a source device's property, or a sink device's property,

Here, the operation mode includes a first mode for simultaneously providing the wireless networking service and the streaming service, and a second mode for providing only the streaming service, operating method of a portable wireless device.
The method of claim 10,

In the first mode,

The portable wireless device performs an interactive procedure for the real time streaming service with each sink device connected to the wireless communication interface, and provides the streaming data only to the sink devices for which the interactive procedure has been successfully performed. Characterized in that the method of operation of a portable radio device.
The method of claim 10,

In the second mode,

The portable wireless device transmits the streaming data to all sink devices connected to the wireless communication interface without an interactive procedure for a real-time streaming service with each sink device. Way.
The method of claim 10,

The process of determining the operation mode,

Receiving a control command for the operation mode from any one of the source device or the sink device, and determining the operation mode according to the received control command. Way.
The method of claim 10,

The process of determining the operation mode,

Determining whether the source device is communicatively connected to a network, and determining the operation mode according to a determination result.
A wired communication interface capable of wired communication with a source device;

A wireless communication interface capable of wirelessly communicating with sink devices located nearby; And

Provides a networking service to a plurality of sink devices connected through the wireless communication interface using network resources of the source device, encodes screen data and voice data received from the source device into a preset format, and provides streaming data. A control unit for generating a real-time streaming of the generated streaming data in a multicasting manner;

Portable wireless device comprising a.