JP2016143954A - Radio communication device and radio communication method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a radio communication device that can be operated at an accessibility mode which guarantees browsing corresponding to a bodily trouble or deterioration of a function of an operator.SOLUTION: A source device confirms an accessibility mode or not when respective sink units are connected. Then, the source device determines whether meta data (sound, video, text vibration information and the like, for example) for supporting a trouble is to be added or not when original image/sound data is transmitted to the sink unit at the accessibility mode. The source device changes a transmitting method of the meta data according to whether the accessibility mode of the respective sink units is on or off.SELECTED DRAWING: Figure 5A

Description

  The technology disclosed in the present specification relates to a wireless communication apparatus and a wireless communication method that can operate in an accessibility mode that guarantees browsing corresponding to an operator's physical disability or reduced function.

  There is a model that can select a user interface for a healthy person and a person with a disability in an information device such as a smartphone or a tablet terminal (for example, see Patent Document 1). In addition, various communication methods for transmitting video from information devices such as smartphones, tablet terminals, and notebook PCs specify transmission methods for only display and operation for healthy people, and do not consider accessibility for people with disabilities. is the current situation. Recently, in wireless network communication and P2P (Peer to Peer) direct communication, there are increasing use cases in which a plurality of terminals are grouped to perform wireless video transmission (see, for example, Patent Document 2). Currently, there is no protocol corresponding to the operation for users.

  On the other hand, even if the communication method that can be transmitted is determined for the exchange between the transmitting device and the receiving device, when multiple terminals are grouped to realize a common service, in a multi-sync topology or multi-source topology environment, New challenges arise.

  For example, when considering a connection form in a multi-sync topology environment, it is desirable that a transmission apparatus having content data connects to all sink devices and transmits optimum data to the reception apparatus. However, in an environment in which healthy persons and persons with disabilities are mixed, there is a new problem of how to control connection between the source device and the sink device and ensure consistency in the mixed environment.

  Furthermore, when considering the connection form in a multi-source topology environment, it is desirable that the received data from each source device be in the same format because the sink device is a single platform, but check the setting values before transmission. There is a need to. There is a need to provide an effective management method for sink devices.

JP 2012-226754 A U.S. Patent Application Publication No. 20131139210

  An object of the technology disclosed in the present specification is to provide an excellent wireless communication apparatus and wireless communication method that can suitably operate in the accessibility mode in a topology environment in which terminals in the normal mode and the accessibility mode are mixed. It is to provide.

The present application has been made in consideration of the above problems, and the technology according to claim 1
A wireless communication unit that wirelessly communicates as a source device with a plurality of sink devices;
A control unit for controlling connection with the sink device and transmission of content data to the sink device;
Comprising
Each of the plurality of sink devices is operable in one of a normal mode and one or more types of accessibility modes that guarantee browsing corresponding to an operator's physical disability or reduced functionality,
The control unit manages types of accessibility modes of the plurality of sink devices;
A wireless communication device.

  According to the technology described in claim 2 of the present application, the control unit of the wireless communication device according to claim 1 transmits the content data after arbitrating the accessibility mode of the plurality of sink devices. Is configured to start.

  According to the technology described in claim 3 of the present application, the control unit of the wireless communication device according to claim 2 confirms whether there is no problem for the operator to change the sink device to change to the accessibility mode. It is configured to display a popup.

  According to the technology described in claim 4 of the present application, the control unit of the wireless communication device according to claim 1 grasps that it is connected to the sink device in the accessibility mode, and the accessibility mode to be connected from now on. It is configured to transmit necessary accessibility mode metadata based on the result of ORing the types.

  According to the technology described in claim 5 of the present application, the control unit of the wireless communication device according to claim 4 transmits accessibility mode metadata in either a multicast packet or a unicast packet. It is configured to let you.

  According to the technology described in claim 6 of the present application, the control unit of the wireless communication device according to claim 1 groups the plurality of sink devices according to the same accessibility mode type and supports each group. Configured to send metadata for accessibility mode.

  According to the technique described in claim 7 of the present application, the control unit of the wireless communication apparatus according to claim 6 transmits accessibility mode metadata in either a multicast packet or a unicast packet. It is configured to let you.

Further, the technique described in claim 8 of the present application is:
A wireless communication unit that wirelessly communicates with the source device as a sink device;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
The control unit extracts and reproduces only reproducible metadata among a plurality of accessibility mode metadata received from the source device,
A wireless communication device.

Further, the technique according to claim 9 of the present application is
A wireless communication unit that performs wireless communication as a source device to the same sink device together with other source devices;
A control unit for controlling connection with the sink device and transmission of content data to the sink device;
Comprising
The sink device can operate in either a normal mode and one or more types of accessibility modes that ensure browsing in response to a physical disability or reduced functionality of the operator,
The control unit causes the sink device to be notified of connection in accessibility mode.
A wireless communication device.

Further, the technique according to claim 10 of the present application is
A wireless communication unit that wirelessly communicates as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
A user interface that selects between accessibility and normal operations,
Is a wireless communication device.

Moreover, the technique according to claim 11 of the present application is
A wireless communication unit that wirelessly communicates as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
The control unit causes UIBC data to be transmitted to the plurality of source devices.
A wireless communication device.

  According to the technology of claim 12 of the present application, the control unit of the wireless communication device of claim 11 is configured to transmit the UIBC data after converting the UIBC data so as to match the mode of each source device. Yes.

  According to the technique described in claim 13 of the present application, the control unit of the wireless communication apparatus according to claim 11 is configured to transmit UIBC data regardless of the mode of the source device.

  According to the technique of claim 14 of the present application, the control unit of the wireless communication apparatus of claim 11 is configured to transmit UIBC data after switching to a common mode in which all devices can be switched. Has been.

In addition, the technique described in claim 15 of the present application is:
A wireless communication unit that performs wireless communication as a source device to the same sink device together with other source devices;
A control unit for controlling connection with the sink device and transmission of content data to the sink device;
Comprising
The sink device can operate in either a normal mode and one or more types of accessibility modes that ensure browsing in response to a physical disability or reduced functionality of the operator,
When the operator changes the mode of the sink device during connection, the control unit causes a mode exchange message signal to be changed in the middle of transmission between the normal mode and the accessibility mode.
A wireless communication device.

Further, the technique described in claim 16 of the present application is:
A wireless communication unit that wirelessly communicates as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
The control unit causes each source device to request that the accessibility modes of the plurality of source devices match.
A wireless communication device.

Further, the technology described in claim 17 of the present application is:
A wireless communication unit that wirelessly communicates as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
The control unit grasps that it is connected to the source device in the accessibility mode, and causes the source device to transmit based on the result of taking the logical sum of the accessibility mode type to be connected.
A wireless communication device.

Further, the technique described in claim 18 of the present application is:
A wireless communication step of wirelessly communicating as a source device with a plurality of sink devices;
A control step for controlling connection to the sink device and transmission of content data to the sink device;
Have
Each of the plurality of sink devices is operable in one of a normal mode and one or more types of accessibility modes that guarantee browsing corresponding to an operator's physical disability or reduced functionality,
In the control step, the type of accessibility mode of the plurality of sink devices is managed.
A wireless communication method.

Moreover, the technology described in claim 19 of the present application is:
A wireless communication step of performing wireless communication as a sink device with a plurality of source devices;
A control step for controlling connection with the source device and transmission of content data from the source device;
Have
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
In the control step, UIBC data is transmitted to the plurality of source devices.
A wireless communication method.

In addition, the technique described in claim 20 of the present application is:
A wireless communication step of performing wireless communication as a source device to the same sink device together with other source devices;
A control step for controlling connection to the sink device and transmission of content data to the sink device;
Have
The sink device can operate in either a normal mode and one or more types of accessibility modes that ensure browsing in response to a physical disability or reduced functionality of the operator,
In the control step, when the mode of the sink device is changed while the operator is connected, a message message for mode exchange that changes the normal mode and the accessibility mode in the middle is transmitted in the middle of transmission.
A wireless communication method.

  According to the technology disclosed in this specification, an excellent wireless communication apparatus and wireless communication method that can operate favorably in the accessibility mode in a topology environment in which terminals in the normal mode and the accessibility mode are mixed. Can be provided.

  In addition, the effect described in this specification is an illustration to the last, and the effect of this invention is not limited to this. In addition to the above effects, the present invention may have additional effects.

  Other objects, features, and advantages of the technology disclosed in the present specification will become apparent from a more detailed description based on the embodiments to be described later and the accompanying drawings.

FIG. 1 is a diagram schematically illustrating a configuration example of a communication system 100 according to the first embodiment of the technology disclosed in this specification. FIG. 2 is a diagram schematically illustrating a functional configuration example of the information processing apparatus 200 that operates as a source device. FIG. 3 is a diagram schematically illustrating a functional configuration example of the information processing apparatus 300 that operates as a sink device. FIG. 4 is a diagram illustrating a connection sequence example between the source device 410 and the sink device 420 according to the first embodiment. FIG. 5A is a diagram illustrating a connection form in which both the source device and the sink device exchange in the accessibility mode. FIG. 5B is a diagram illustrating a connection form in which the source device in the normal mode and the sink device in the accessibility mode exchange with each other. FIG. 6 is a diagram showing an example of a connection sequence between the source device and each sink device in the topology environment shown in FIG. 5A. FIG. 7 is a diagram illustrating an example of a connection sequence between the source device 410 and the sink device 420 (an example using Miracast processing) in the first embodiment. FIG. 8 is a diagram showing types of obstacles, correspondence requests, and correspondence examples of each. FIG. 9 is a diagram showing bit assignments for accessibility support requests. FIG. 10A is a diagram illustrating a sequence example in which Capability Negotiation communication processing is performed between the source device and the sink device (communication example in which an accessibility function is requested from the source device). FIG. 10B is a diagram illustrating a sequence example in which Capability Negotiation communication processing is performed between the source device and the sink device (communication example in which a connection enable is requested from the source device). FIG. 11A is a diagram illustrating a sequence example in which Capability Negotiation communication processing is performed between the source device and the sink device (communication example in which an accessibility function is requested from the sink device). FIG. 11B is a diagram illustrating a sequence example in which Capability Negotiation communication processing is performed between the source device and the sink device (communication example in which a connection enable is requested from the sink device). FIG. 12 is a diagram illustrating an operation example of the sink device in each accessibility mode type. FIG. 13A is a flowchart showing a processing procedure for a source device to select a common accessibility mode from among a plurality of accessibility mode types for sink devices. FIG. 13B is a flowchart illustrating a processing procedure for the source device to select a common accessibility mode from among a plurality of accessibility mode types for sink devices. FIG. 14A is a diagram illustrating a connection form in which both the accessibility mode source device and the normal mode source device communicate with one accessibility mode sink device. FIG. 14B is a diagram illustrating a connection form in which both the normal mode and accessibility mode source devices communicate with one normal mode sink device. FIG. 15A is a diagram illustrating a communication processing example of image data and audio data using WebSocket between the source device 410 and the sink device 420. FIG. 15B is a diagram illustrating an example of a frame format of WebSocket exchanged between the source device 410 and the sink device 420. FIG. 15C is a diagram illustrating an example of a frame format of a WebSocket exchanged between the source device 410 and the sink device 420. FIG. 16 is a diagram schematically illustrating a configuration example of a communication system 1100 according to the second embodiment of the technology disclosed in this specification. FIG. 17 is a diagram illustrating a large screen display including buttons for designating accessibility operations and default operations. FIG. 18 is a diagram illustrating an example of a basic sequence from P2P direct connection to construction of a multisync topology environment.

  Hereinafter, embodiments of the technology disclosed in this specification will be described in detail with reference to the drawings.

[First Embodiment]
[Communication system configuration]
FIG. 1 schematically illustrates a configuration example of a communication system 100 according to the first embodiment of the technology disclosed in this specification. The illustrated communication system 100 is assumed to perform wireless connection in a multi-sync topology environment using P2P direct communication. Specifically, the communication system 100 includes three information processing devices 200, 300, and 400, and transmits data (for example, image data and audio data) transmitted from one information processing device 200 as a source device. The other two information processing apparatuses 300 and 400 are configured to receive as sink devices. Since there are a plurality of sink devices for one source device, the communication system 100 shown in FIG. 1 can be said to be a multi-sync topology environment.

  Each of the information processing apparatuses 200, 300, and 400 constituting the communication system 100 is a communication device having a wireless communication function. For example, the information processing devices 200, 300, and 400 are display devices (for example, personal computers) or portable information processing devices (such as smartphones and tablet terminals) that have a wireless communication function. In addition, the information processing apparatuses 200, 300, and 400 are configured by IEEE (Institut of Electrical and Electronics Engineers) 802.11, IEEE 802.15, IEEE 802.16, 3GPP (3rd Generation Partnership Decode C, W, B, and W). Division Multiple Access), GSM (registered trademark) (Global System for Mobile Communications), WiMAX (Worldwide Interoperability), TEMAX2L, TMAX It is a wireless communication device that conforms to -A (Advanced)). The information processing apparatuses 200, 300, and 400 can exchange various information by using any one of these wireless communication functions.

  Here, the information processing device 300 or 400 that is a sink device is a device having the same configuration in that it receives data transmitted from the information processing device 200 that is a source device. Further, the exchange between the information processing apparatus 200 and the information processing apparatus 400 is basically equivalent to the exchange between the information processing apparatus 200 and the information processing apparatus 300. Therefore, hereinafter, only the connection method between the information processing apparatus 200 and the information processing apparatus 300 will be described as the first embodiment, and the description of the connection method between the information processing apparatus 200 and the information processing apparatus 400 will be omitted. Note that a case where wireless communication using a wireless LAN (Local Area Network) is performed between the information processing apparatus 200 and the information processing apparatus 300 will be described as an example.

  As this wireless LAN, for example, Wi-Fi (Wireless Fidelity) Direct, TDLS (Tunneled Direct Link Setup), an ad hoc network, and a mesh network can be used. Moreover, as near field wireless AV (Audio Visual) transmission communication used for the communication system 100, for example, Wi-Fi CERTIFIED Miracast (technical specification name: Wi-Fi Display) can be used. Wi-Fi CERTIFIED Miracast uses Wi-Fi Direct and TDLS technologies to transmit audio and display images that are played back on one terminal to another terminal, and the other terminals can also transmit the audio, This is a mirroring technology that outputs image data.

  In addition, Wi-Fi CERTIFIED Miracast implements UIBC (User Input Back Channel) on TCP / IP (Transmission Control Protocol / Internet Protocol). UIBC is a technique for transmitting operation information of an input device such as a mouse or a keyboard from one terminal to the other terminal. Instead of Wi-Fi CERTIFIED Miracast, other remote desktop software (for example, VNC (Virtual Network Computing)) may be applied.

  Here, in Wi-Fi CERTIFIED Miracast, an image (video) is, for example, H.264. H.264 is used for compression / decompression. Also, for example, in Wi-Fi CERTIFIED Miracast, H. H.264 can be adjusted on the transmission side. H. For example, H.264 is not limited thereto. 265 (e.g. HEVC (high efficiency video coding), SHVC (scalable video coding extension of high-equality coding), MPEG (Moving Picture coding), MPEG (Moving Picture coding) . Also, for example, a line-based codec (for example, Wavelet, DCT (Discrete Cosine Transform) that compresses by compressing by bundling 1 line or more, or dividing 2 lines or more into 2 × 2 or more macroblocks. )). Also, for example, a codec that reduces a transmission rate without performing compression such as DCT or Wavelet by obtaining a difference from a previous code amount region of a specific code amount region (such as a picture or a bundle of multiple lines or a macro block). Can respond. In addition, an image (video) may be transmitted / received uncompressed between the information processing apparatus 200 and the information processing apparatus 300.

  In the communication system 100 according to the first embodiment, as data to be transmitted from the information processing apparatus 200 to the information processing apparatus 300, for example, image data and audio data generated by an imaging operation, storage of a hard disk, or the like It is assumed that content is stored in the apparatus (for example, content including image data and audio data). As the information processing apparatus 200 that is a data transmission source, an electronic device (for example, a personal computer, a game machine, a smartphone, or a tablet terminal) equipped with a camera may be used. In addition, as the information processing apparatus 300 that is a data transmission destination, another electronic device including a display unit (for example, an imaging device, a game machine, a smartphone, or a tablet terminal) may be used. When the information processing apparatus 200 has a tethering function, content stored in an ISP (Internet Services Provider) via a wireless or wired network may be a transmission target to the information processing apparatus 300.

  For example, image data generated by the imaging operation of the information processing device 200 is transmitted to the information processing device 300, and an image based on the received image data is displayed on the display unit on the information processing device 300 side.

  In FIG. 1, a range in which the information processing apparatus 200 can directly communicate by P2P direct connection using wireless communication, that is, an information transmission range is indicated by reference numeral 101. The information transmission range 101 is an information transmission range (service range) when the information processing apparatus 300 is used as a reference.

  Before describing the operation between the information processing apparatus 200 and the information processing apparatus 300 in detail, a method for constructing a multisync topology will be described.

  FIG. 18 shows an example of a basic sequence from P2P direct connection to construction of a multi-sync topology environment. The figure shows an operation example in which the information processing apparatus 400 joins a group in an environment in which the information processing apparatus 200 that is the source device is connected to the information processing apparatus 300 that is the sink device by P2P direct communication.

  The information processing apparatus 200 is already connected to the P2P client of the information processing apparatus 300 as a GO (Group Owner). For this reason, an invitation request process is performed from the information processing apparatus 200 to the information processing apparatus 400. Alternatively, provision discovery processing may be performed from the information processing apparatus 400 to the information processing apparatus 200. The operation sequence will be described by taking the invitation request process as an example. The information processing apparatus 400 returns the invitation response process to the information processing apparatus 200.

  Through the above processing, the information processing apparatus 200 connects the information processing apparatus 400 as a new P2P client. Then, by performing service discovery with each other, a multi-sync topology is constructed with the information processing device 200 as a source device and the information processing device 300 and the information processing device 400 as sink devices.

  After the multi-sync topology is constructed, the content transmitted from the information processing device 200 to the information processing device 300 or 400 and displayed may be the same content (image) or different content (image). For example, the image displayed on the information processing apparatus 200 can be transmitted as it is to the information processing apparatus 300, and the content stored in the storage unit of the information processing apparatus 200 can be transmitted to the information processing apparatus 400.

[Source device configuration]
FIG. 2 schematically illustrates a functional configuration example of the information processing apparatus 200 that operates as a source device in the communication system 100 according to the first embodiment.

  The information processing apparatus 200 includes an antenna 210, a wireless communication unit 220, a control signal reception unit 230, a control unit 240, an image / audio signal generation unit 250, an image / audio compression unit 260, and a stream transmission unit 270. I have. Hereinafter, each part will be described.

  Based on the control of the control unit 240, the wireless communication unit 220 uses wireless communication to communicate each piece of information (for example, image data) with another information processing device (for example, a sink device such as the information processing device 300). , Audio data, and control data) are transmitted / received via the antenna 210. For example, when transmission processing of image data to another information processing apparatus is performed, the image data generated by the image / sound signal generation unit 250 is compressed by the image / sound compression unit 260, and the compressed image data ( Image stream) is transmitted from the antenna 210 via the wireless communication unit 220.

  In addition, it is assumed that wireless communication unit 220 can transmit / receive each information to / from another information processing apparatus (for example, information processing apparatus 300) using a plurality of frequency channels. In the first embodiment, an example will be described in which the wireless communication unit 220 has a function capable of transmitting and receiving only frequencies at the same time or selected from three types of frequency channels of 2.4 GHz, 5 GHz, and 60 GHz. Thus, when the source device has a function capable of transmitting and receiving a plurality of frequency channels, the sink device (for example, the information processing apparatus 300) can control which frequency channel is used by each source device. it can.

  The control signal receiving unit 230 includes a control signal (for example, exchange with the information processing device 300) transmitted from another information processing device (for example, the information processing device 300) among the pieces of information received by the wireless communication unit 220. Information). Then, the control signal receiving unit 230 outputs the acquired control signal to the control unit 240.

  The control unit 240 performs control regarding each piece of information transmitted from the information processing apparatus 200. For example, the control unit 240 controls the image / sound signal generation unit 250 and the image / sound compression unit 260 based on the control signal received by the control signal receiving unit 230. Specifically, the control unit 240 performs control for changing the resolution of the image data to be transmitted and the number of audio channels, and control for changing the image area of the image data to be transmitted. That is, the control unit 240 controls transmission of a stream to be transmitted based on the control signal received by the control signal receiving unit 230. The transmission control of this stream is, for example, data transmission rate control and scalability transmission rate control.

  The control unit 240 also has a function of measuring a radio wave propagation state (link radio wave propagation state) when data is transmitted / received to / from a sink device (for example, the information processing apparatus 300) using wireless communication. And the measurement result (radio wave propagation measurement information) may be transmitted from the wireless communication unit 220 to the sink device.

  The radio wave propagation measurement information mentioned here is information used when determining whether or not the channel quality with the sink device is a quality that enables transmission and reception of image data and audio data. The radio wave propagation measurement information is used, for example, when stream transmission control is performed.

  The data transmission rate here means mainly the rate of occupying the communication path, and includes the meanings of communication rate and communication capacity. Further, the resolution is defined as an image quality index composed of elements such as an image data image frame (vertical and horizontal pixel numbers) and image data bit rate (compression rate), for example. Further, the stream throughput can be used as an image quality index. The number of audio channels is monaural (1.0ch), stereo (2.0ch), 5.1ch, 6.1ch, 9.1ch, high-resolution audio (High-Resolution Audio), etc. The meaning of the reproduction method shall be included. The number of audio channels is defined as an index of sound quality composed of elements such as the bit rate (compression rate) of audio data and the number of channels. Further, the stream throughput can be used as an index of sound quality.

  Further, the control unit 240 performs control for improving a state that cannot be stabilized by data rate control. For example, the control unit 240 grasps the system performance information of the sink device by exchanging information with the sink device (for example, the information processing apparatus 300). The system performance information referred to here is, for example, performance information related to the system of the sink device. For example, the system performance information includes usable frequency channels, resolutions, and transmission control protocols (TCP (Transmission Control Protocol), UDP (User Datagram Protocol)). Further, the system performance information is information indicating, for example, encryption method support, SD (Standard Definition) / HD (High Definition) / 4K support, and low power consumption mode support. For example, the control unit 240 can select a stream transmission control method that further improves the stability of the entire system of the communication system 100 according to whether or not the sink device supports the low power consumption mode.

  Further, the control unit 240 puts information on whether or not the information processing apparatus 200 is a mobile device into the exchange of information with the sink device. For example, information on whether or not the information processing apparatus 200 is a mobile device can be included in the capability information regarding the information processing apparatus 200. When the sink device such as the information processing device 300 recognizes that the information processing device 200 is a mobile device, there is no need to operate the information processing device 200 based on the relationship with the other information processing device connected thereto. Judgment can be made. Thus, when the sink device determines that there is no need to operate the information processing device 200, the information processing device 200 receives a transmission stop command from the sink device. When the control unit 240 grasps the transmission stop command, the control unit 240 can power down the functions of the image / audio signal generation unit 250, the image / audio compression unit 260, and the stream transmission unit 270 for a certain period of time. it can. In addition, the control unit 240 can also shift the wireless communication unit 220 to intermittent reception (a mode in which the wireless communication unit 220 is periodically woken up to the extent that a command can be received from the sink device, and the others are powered down).

  The image / audio signal generation unit 250 generates data (image data, audio data) to be output based on the control of the control unit 240, and outputs the generated data to the image / audio compression unit 260. For example, the image / audio signal generation unit 250 includes an imaging unit and an audio acquisition unit (both not shown). The imaging unit (for example, a lens, an imaging device, and a signal processing circuit) captures an object and generates an image (image data). In addition, a sound acquisition unit (for example, a microphone) acquires ambient sound when the image data is generated by the imaging unit. The data generated by the image / sound signal generation unit 250 is a transmission target to another information processing apparatus (for example, a sink device such as the information processing apparatus 300).

  The image / audio compression unit 260 compresses (encodes) the data (image data and audio data) generated by the image / audio signal generation unit 250 based on the control of the control unit 240. Then, the image / audio compression unit 260 outputs the compressed data (image data and audio data) to the stream transmission unit 270. Note that the image / sound compression unit 260 may be realized by execution of encoding by software, or may be realized by execution of encoding by hardware. The image / sound compression unit 260 is assumed to function as a codec, but can handle uncompressed images or sounds. Furthermore, it is assumed that the image / sound compression unit 260 also functions as a scalable codec. The scalable codec here means a codec that can be freely adapted in accordance with, for example, the resolution of the information processing apparatus (sink device) on the receiving side or the network environment.

  Based on the control of the control unit 240, the stream transmission unit 270 transmits the data (image data and audio data) compressed by the image / audio compression unit 260 from the antenna 210 via the wireless communication unit 220 as a stream. Perform transmission processing.

  Note that the information processing apparatus 200 can include a display unit, an audio output unit, an operation reception unit, and the like in addition to the above-described units, but these are not illustrated in FIG. 2 for the sake of simplicity. .

  A display unit (not shown) of the information processing apparatus 200 displays, for example, an image generated by the image / audio signal generation unit 250. Various display panels can be used as the display unit. For example, an organic EL (Electro Luminescence), a crystal LED (Light Emitting Diode) display, and an LCD (Liquid Crystal Display) can be used.

  The audio output unit (not shown) of the information processing apparatus 200 is a speaker, for example, and outputs the audio generated by the image / audio signal generation unit 250. The image can be output from both the transmitting device and the receiving device, but the sound is preferably output from only one of them.

  In addition, an operation receiving unit (not shown) of the information processing apparatus 200 is configured with, for example, a keyboard, a mouse, a game pad, a touch panel, a camera, and a microphone, and receives an operation input performed by an operator. Note that the operation reception unit and the display unit can be integrally configured using a touch panel that allows an operator to input an operation by touching or approaching the finger with the display surface.

  In this specification, an example in which the information processing apparatus 200 generates image data and sound data to be transmitted will be described. However, the information processing apparatus 200 acquires image data and sound data to be transmitted from an external device. You may do it. For example, the information processing apparatus 200 may acquire image data and audio data to be transmitted from a web camera with a microphone.

  In addition, the information processing apparatus 200 may transmit content (for example, content including image data and audio data) stored in a storage device (for example, a hard disk) (not shown) as a transmission target. The storage device may be either internal or external to the information processing apparatus 200. In this case, it may be assumed that the content stored in the storage device is a compressed content. When the compressed content is compressed by the encoding method defined by the standard adopted in the communication system 100, the compressed content is transmitted as it is without being decoded. Also good.

[Configuration of sink device]
FIG. 3 schematically illustrates a functional configuration example of the information processing apparatus 300 that operates as a sink device in the communication system 100 according to the first embodiment. It should be understood that the other information processing apparatuses 400 have the same functional configuration.

  The information processing apparatus 300 includes an antenna 310, a wireless communication unit 320, a stream reception unit 330, an image / sound development unit 340, an image / sound output unit 350, a user information acquisition unit 360, a control unit 370, A control signal transmission unit 380 and a management information holding unit 390 are provided. The image / sound output unit 350 includes a display unit 351 and a sound output unit 352. Hereinafter, each part will be described.

  The wireless communication unit 320 uses wireless communication to control each piece of information (for example, image data) with another information processing apparatus (for example, a source device such as the information processing apparatus 200) based on the control of the control unit 370. And audio data) are transmitted / received via the antenna 310. For example, when image data reception processing from another information processing apparatus is performed, image data received by the antenna 310 is expanded by the image / sound expansion unit 340 via the wireless communication unit 320 and the stream reception unit 330. (Decoded). The developed image data is supplied to the image / sound output unit 350, and an image corresponding to the developed image data is output from the image / sound output unit 350. That is, an image corresponding to the developed image data is displayed on the display unit 351. In addition, a sound corresponding to the expanded sound data is output from the sound output unit 352.

  Further, it is assumed that wireless communication unit 320 can transmit / receive each information to / from another information processing apparatus (for example, information processing apparatus 200) using a plurality of frequency channels. In the first embodiment, an example will be described in which the wireless communication unit 320 has a function capable of transmitting and receiving only frequencies at the same time or selected from three types of frequency channels of 2.4 GHz, 5 GHz, and 60 GHz. That is, it is assumed that the wireless communication unit 320 can perform communication using the first frequency band and communication using the second frequency band having a data transmission rate higher than that of the first frequency band. Further, the control unit 370 controls which frequency channel of the plurality of frequency channels is used for wireless communication with each source device.

  In the communication system 100 illustrated in FIG. 1, a link between the information processing device 200 and the information processing device 300 (or between the source device and the sink device), and between the information processing device 400 and the information processing device 300 (or the sink). The link between the devices may be the same frequency channel or different frequency channels.

  In the first embodiment, an example in which the wireless communication unit 320 has a function capable of transmitting and receiving three types of frequency channels of 2.4 GHz, 5 GHz, and 60 GHz will be described. However, the frequency channel to be used is not limited thereto. . For example, the wireless communication unit 320 may have only a function capable of transmitting and receiving other frequency channels and one or two frequency channels. Further, a function capable of transmitting and receiving one or more frequency channels may be provided.

  Based on the control of the control unit 370, the stream reception unit 330 receives information and a stream (for example, an image stream and an audio stream) exchanged with each source device from the pieces of information received by the wireless communication unit 320. To do. Then, the stream reception unit 330 outputs the received command information to the control unit 370, and outputs the received stream to the image / audio development unit 340 and the control unit 370.

  The information exchanged with each source device here is information transmitted from the source device (for example, the information processing apparatus 200), and includes, for example, an acquisition request for system performance information of the information processing apparatus 300. The system performance information is information indicating, for example, usable frequency channels, resolutions, TCP, UDP, encryption method support, SD / HD / 4K support, and low power consumption mode support.

  Further, the stream receiving unit 330 has a function of measuring a radio wave propagation situation (link radio wave propagation situation) when data is transmitted / received to / from a source device using wireless communication. Then, the stream receiving unit 330 outputs the measurement result (radio wave propagation measurement information) to the control unit 370.

  The image / sound development unit 340 develops (decodes) a stream (image data and sound data) transmitted from another information processing apparatus (for example, the information processing apparatus 200) based on the control of the control unit 370. Then, the image / sound development unit 340 outputs the developed data (image data and sound data) to the image / sound output unit 350. The image / sound decompression unit 340 may be realized by executing decoding by software, or may be realized by executing decoding by hardware. The image / sound developing unit 340 is assumed to function as a codec, but can handle uncompressed images or sounds. In addition, the image / sound development unit 340 also functions as a scalable codec.

  A display unit 351 in the image / sound output unit 350 displays each image based on the image data developed by the image / sound development unit 340. As the display unit 351, for example, a display panel such as an organic EL panel, a crystal LED display, or an LCD panel can be used. Further, as the display unit 351, a touch panel that allows an operation input by an operator touching or approaching the finger with the display surface may be used.

  The audio output unit 352 includes, for example, a speaker, and outputs various types of audio (such as audio related to the image displayed on the display unit 351) based on the audio data expanded by the image / sound expansion unit 340. As an audio output method, for example, a method is used in which only the sound of the source device assigned to the central channel (main image) is reproduced from the speaker and the sound of the source device assigned to the peripheral channel (sub-image) is not reproduced. be able to. Also, as another audio output method, for example, a method is used in which the sound volume of the source device assigned to the central channel is set as the main and the sound volume of the source device assigned to the peripheral channel is reduced and played back. Can do. Of course, other audio output methods may be used.

  The user information acquisition unit 360 acquires information (user information) related to the operator of the information processing apparatus 300 and outputs the acquired user information to the control unit 370. For example, the user information acquisition unit 360 acquires user information by receiving input from an operation reception unit (keyboard, mouse, remote controller, game pad, touch panel) that allows the operator to directly set the display method. Can do. The operation accepting unit is an operation member for designating an arbitrary region in the image displayed on the display unit 351, for example. Further, for example, the user information acquisition unit 360 accepts input from a device that can grasp the intention of the operator, such as a camera, a microphone, and various sensors (for example, a gyro sensor and a sensor that senses a human body). The user information can be acquired.

  For example, the user information acquisition unit 360 operates when information based on a stream received from another information processing apparatus (for example, the information processing apparatus 200) using wireless communication is output from the image / sound output unit 350. User information generated by user actions by a user is acquired. This user information is, for example, user information generated by a user operation relating to an image displayed on the display unit 351. For example, the user information is information generated based on a user operation regarding an image displayed on the display unit 351.

  Also, the user information acquisition unit 360 can acquire user data by acquiring image data generated by an imaging unit (not shown) attached to the display unit. Further, the user information acquisition unit 360 may acquire information (for example, position information and identification information) acquired by, for example, an external device (for example, each sensor or wearable device) and generate user information. .

  The control unit 370 holds each piece of information acquired by the stream reception unit 330 in the management information holding unit 390 and manages the source device based on the management information held in the management information holding unit 390. In addition, in the multi-source topology environment, the control unit 370 controls transmission of streams so that the stability of the streams transmitted from a plurality of source devices is improved in the entire system.

  For example, the control unit 370 performs stream transmission control based on the user information acquired by the user information acquisition unit 360 and the management information held in the management information holding unit 390. Specifically, control unit 370 generates a control signal for performing stream transmission control for each source device, based on the management information held in management information holding unit 390. Then, control unit 370 outputs the generated control signal to control signal transmission unit 380. For example, the control unit 370 changes the resolution of the image displayed on the display unit 351 based on the user information and management information, and generates a control signal for requesting each source device to transmit a transmission rate equivalent to this resolution. To do. For example, the control unit 370 generates a control signal for determining the image display area on the display unit 351 based on the user information and the management information. For example, the control unit 370 generates a control signal for determining the image size in the display unit 351 based on the user information and the management information.

  In addition, the control unit 370 performs control for setting the frequency channel to be used and the resolution based on the user information and management information. For example, the control unit 370 sets a frequency channel to be used for each source device for a plurality of frequency channels included in the wireless communication unit 320. In addition, when the power consumption mode is different for each frequency channel, the control unit 370 can grasp each mode and set a frequency channel that cares about the power consumption of the mobile device. That is, the control unit 370 separately sets the first power consumption mode related to the first frequency band and the second power consumption mode related to the second frequency band having a higher data transmission rate than the first frequency band. can do.

  The control signal transmission unit 380 performs transmission processing for transmitting the control signal output from the control unit 370 to another wireless communication device via the wireless communication unit 320 and the antenna 310.

[Example of communication using Wi-Fi CERTIFIED Miracast specification command]
FIG. 4 shows an example of a connection sequence between the source device 410 and the sink device 420 in the first embodiment. This figure is an example of communication processing (WFD capability negotiation) using the RTSP (Real Time Streaming Protocol) protocol. However, the source device 410 and the sink device 420 correspond to the information processing device 200 and the information processing device 300 in the communication system 100 illustrated in FIG. 1, respectively.

  First, as shown in a dotted-line rectangle 430 in FIG. 4, an “RTSP M3 Request” (RTSP GET_PARAMETER Request) message transmitted from the source device 410 to the sink device 420 and the sink device 420 in response to this message. An “RTSP M3 Response” (RTSP GET_PARAMETER Response) message transmitted to the source device 410 can be used.

  On the other hand, it may be appropriately transmitted from the source device 410 to the sink device 420. For example, the exchange of the “RTSP M3 Request” message and the “RTSP M3 Response” message is omitted, the management information is included in the message transmitted from the source device 410 to the sink device 420, and the sink device from the source device 410 Management information may be transmitted to 420, and the sink device 420 may select the information and hold it in the management information holding unit 390. Further, when performing content protection setting, link protection setup may be performed after the “RTSP M3 Response” message, and then communication may be performed while securing the secrecy of M4 messages and higher.

  The information of the sink device 420 received as RTSP M3 Response is as follows.

Audio format (s) supported by the WFD Sink.
Video format (s) support by the WFD Sink.
-3D formats supported by the WFD Sink.
The HDCP system 2.0 / 2.1 / 2.2 support / control port.
-Available EDID information of display attached to the WFD Sink.
-Coupled WFD Sink information.
RTP port (s) the WFD Sink (s) listen on.
Supports I2C commands and port number.
UIBC capability supported.
-WFD Source uses this parameter to obtain the connector type currently active on the WFD Sink.
・ Indicate the support for standby and resume control using RTSP.

  The connection sequence between the source device 410 and the sink device 420 illustrated in FIG. 4 includes the connection sequence between the information processing device 200 and the information processing device 300 in the communication system 100 according to the first embodiment, and the information processing device 200. This applies to the connection sequence with the information processing apparatus 400.

[Multi-sync topology environment with mixed accessibility modes]
Next, an environment in which an accessibility mode, which is a terminal mode for persons with disabilities, is mixed in the multi-sync topology environment as shown in FIG.

  The accessibility mode referred to here is a mode corresponding to browsing security (web accessibility) for people with physical disabilities and those whose physical functions have deteriorated due to aging. Accessibility and accessibility for each disability to be supported, such as “visual disability support” to support visual disabilities, “hearing disability support” to support hearing disabilities, and “body function (operation difficulty) support” to support cognitive or physical disabilities. The mode type (operation corresponding to the information processing apparatus) is different. The operation of the information processing apparatus in each accessibility mode type is exemplified below.

(1) Visual disabilities support-Visual support is provided by reading out the screen contents aloud. Specifically, there are terminals such as a function for reading out the application name aloud, a tap, and a tap once to select an icon, and a double tap to execute the icon. Furthermore, it is possible to notify that an icon has been selected by vibration, or to connect a braille display.
-You can enlarge the characters on the screen, enlarge / reduce the screen with the zoom function, and reverse the screen color.

(2) Deaf support ・ Change to subtitle display when watching movies and videos.
・ Adjust the audio volume.
・ Notify incoming calls with LED flash.

(3) Gestures that do not work well, such as physical function support and pinch, can be replaced with other gestures that can be operated by the operator.
・ Allow voice operation when gestures such as pinch cannot be performed.

  FIG. 8 summarizes a response request to the information processing apparatus for each type of obstacle and an operation example that the information processing apparatus actually corresponds to.

  Supplementing FIG. 8, TTS, notification sound, convex point, Braille, display, voice operation, and the like can be given as specific examples of the corresponding action “voice, tactile information presentation, operation”. In addition, specific examples of the corresponding action “Do not use a combination of colors that are difficult to see, and add characters to the information to be identified by color” include prohibiting the combination of red and green, and adding characters to the color key. it can. Further, specific examples of the corresponding action “a person with a visual acuity of 0.286 to 0.1 can operate” can include a character, a screen enlargement, a contrast change, and the like. Specific examples of the corresponding operation “visualization of voice information” include Speech to Text, notification light, vibration, gesture, sign language, and the like. A specific example of the corresponding operation “providing a mode that does not rely on utterance” is keyboard support. Further, specific examples of the corresponding action “providing a mode regardless of cognition, memory, language, and comprehension ability” can include plain text, TTS, symbols and illustrations. In addition, as a specific example of the corresponding operation “providing a mode that does not require a detailed operation or an operation to be performed at the same time”, keyboard shortcut support can be cited. Further, specific examples of the corresponding action “providing a mode that can be operated even by a person who has difficulty in stretching or applying power” include a touch panel, an external keyboard, and voice operation. In addition, specific examples of the corresponding action “can be operated without directly touching the body” include a large keyboard and voice operation.

  Even if the type of the obstacle is the same, the request for handling the information processing apparatus varies depending on the degree of the obstacle. Further, even if the types of obstacles are different, depending on the degree of obstacles, the correspondence request to the information processing apparatus and the correspondence operation of the information processing apparatus may be the same or similar. However, FIG. 8 is merely an example, and it should be understood that other response requests for the type of obstacle can be considered. Hereinafter, the accessibility mode corresponding to the type of disability is referred to as “accessibility mode type”.

[Assumed connection type]
An outline of the operation of the source device will be described in a multi-sync topology environment in which a plurality of sink devices have a mode for normal persons (normal mode) and a mode for disabled persons (accessibility mode).

  FIG. 5A illustrates a connection form in which both the source device and the sink device exchange in the accessibility mode. For example, in the communication system 100 shown in FIG. 1, the information processing device 200 and the information processing device 300 are operating in the accessibility mode and the information processing device 400 is operating in the normal mode.

  FIG. 5B illustrates a connection form in which the source device in the normal mode and the sink device in the accessibility mode exchange with each other. For example, in the communication system 100 shown in FIG. 1, the information processing apparatus 200 operates in the normal mode, and at least one of the information processing apparatuses 300 or 400 operates in the accessibility mode.

  In a multi-sync topology environment, it is assumed that sink devices in normal mode and accessibility mode are mixed. The source device checks whether it is in accessibility mode when connecting each sink device. For the sink device in accessibility mode, metadata for supporting disabilities (or for guaranteeing viewing of the transmitted image) when transmitting original image / audio data (see FIG. It is determined whether or not to add data necessary to realize the corresponding operation example shown in FIG. 8 (for example, voice, video, text, information visualizing voice information, size information, etc.). The source device changes the transmission method of metadata depending on whether the accessibility mode of each sink device is on or off. As a transmission method, both Multiple Unicast and Multicast can be used.

  For example, the source device transmits all metadata that needs to be supported to all connected sink devices. On the other hand, each sink device may determine whether or not to use metadata suitable for its accessibility mode type.

  In the multi-sync topology environment in which the normal mode and the accessibility mode are mixed as shown in FIGS. 5A and 5B, the following five approaches (1) to (5) are conceivable.

(1) In a multi-sync topology environment, the source device performs arbitration to unify the modes of the sink devices before starting image transfer. Specifically, the source device requests each sink device to match the mode of multiple sink devices (arbitration for unifying the mode of the sink device), and transfers the image after matching the mode of each terminal. To start.

(2) The source device performs transmission processing in the operation mode of its own terminal without performing arbitration for unifying the mode with the plurality of sink devices as in (1). Each sink device converts the data received by the source device into data that can be handled by the terminal itself and reproduces the data.

(3) In a multi-sync topology environment, the source device grasps that it is connected to the accessibility mode sink device, and support is based on the result of taking the logical sum of the accessibility mode types to be connected. Send all necessary metadata. The sink device extracts and reproduces only the metadata that can be reproduced by the terminal itself from the received plurality of metadata.

(4) In a multi-sync topology environment, the source device grasps the presence / absence of the accessibility mode among multiple sink devices, and if there is an accessibility mode terminal, selects it as the accessibility mode type. Gather all possible metadata from Capability Negotiation processing. The sink device extracts and reproduces only the metadata that can be reproduced by the terminal itself from the received plurality of metadata.

(5) In the multi-sync topology environment, the source device collects information on the accessibility mode type in which a plurality of sink devices are selected from the Capability Negotiation process. As a result of information collection, grouping is performed for each type of the same obstacle, and metadata corresponding to each group is transmitted.

  FIG. 6 shows an example of a connection sequence between the source device and each sink device by the above approach (1) in the topology environment shown in FIG. 5A.

  When the source device 200 is in the accessibility mode, an approach for converting the sink device 400 from the normal mode to the accessibility mode is required, and whether the sink device 300 matches the accessibility mode type.

  First, the processing of the connection sequence between the source device 200 and the sink device 300 will be described on the assumption that the accessibility modes of the source device 200 and the sink device 300 match in the topology environment shown in FIG. 5A.

  When the accessibility mode of the source device 200 is operated to select video or audio, in addition to the transmission of normal content data (content data shown in FIG. 6), metadata for accessibility mode (shown in FIG. 6) As metadata, one or more pieces of information among the visual support, the auditory support, and the physical function (operation difficulty) support are separately transmitted. However, the transmission procedure of the accessibility mode metadata and the content data is not limited to the communication procedure shown in FIG. Accessibility mode metadata is information for interpolating the content data (to support disabilities), and either one may be transmitted in addition to being transmitted simultaneously with the content data.

  On the other hand, the sink device 300 receives the content data from the source device 200, and at the same time, receives the metadata for accessibility mode, and appropriately displays the optimum content under the control of the sink device 300.

  On the other hand, when the sink device 300 operates the accessibility mode to remotely select the video or audio of the source device 200, the sink device 300 is transferred from the sink device 300 to the source device 200 separately from the content data and metadata. Is transmitted as accessibility mode operation information (operation metadata shown in FIG. 6). When the source device 200 receives the operation information of the sink device 300 as the accessibility mode operation information of the sink device 300 from the sink device 300, the source device 200 operates according to the received operation information. The operation of the source device 200 based on the accessibility mode operation information is a function equivalent to the conventional UIBC. However, in consideration of browsing security, the UIBC data type can be set to audio, video, text information, or Information that visualizes audio information is also included.

  Here, in case the operator of the source device 200 or the sink device 300 changes during connection, a mode exchange message signal for changing between the normal mode and the accessibility mode may be prepared. For example, when this message is exchanged between transmission and reception, it is possible to change the normal mode and accessibility mode, or the type of accessibility mode, without disconnecting the wireless video transmission currently being transmitted and received. (FIG. 11 (described later) shows an example of command exchange).

  Note that the source device may display a pop-up for confirming whether the change is acceptable to the operator on the sink device being operated before changing the type of the accessibility mode.

  Next, processing of the connection sequence between the source device 200 and the sink device 400 will be described on the assumption that the accessibility modes of the source device 200 and the sink device 400 do not match in the topology environment illustrated in FIG. 5A.

  When video or audio transmitted from the source device 200 to the sink device 400 is operated on the source device 200 side, the source device 200 is operated so that the accessibility mode types of the source device 200 and the sink device 400 match. Accessibility mode metadata linked to the created video, audio, and text is transmitted to the sink device 400. At this time, the source device 200 performs transmission processing after converting the accessibility mode metadata into content media that can be operated by the sink device 400. For example, in the case of accessibility for auditory support, the source device 200 transmits subtitle data as text or adjusts sound volume for movies and video viewing contents. Further, the source device 200 performs a process of transmitting control data to the sink device 400 so that the LED flash can be received on the sink device 400 side.

  Further, in the above approach (2), when the sink device 400 receives the accessibility mode content data or the accessibility mode metadata from the source device 200, the sink device 400 appropriately performs the optimal conversion on the content. Then display.

  The source device 200 receives the remote operation information of the source device 200 (converted into operation information receivable by the source device 200) from the sink device 400, and uses the received operation metadata as it is in the accessibility mode. Process as operation data. Although it is a function equivalent to the conventional UIBC, in consideration of browsing security, UIBC data types may include audio, video, text information, or information that visualizes audio information. .

  Note that the operation for realizing approach (2) is not limited to the above. For example, data conversion suitable for the accessibility mode may be performed on either the source device 200 or the sink device 400 side. In addition, for example, the source device 200 may be able to convert to the normal mode. In particular, since the operation data is handled differently in the normal mode and the accessibility mode, the conversion operation is essential for transmission between devices having different modes. For example, when the operator of the source device 200 registers a unique user operation in support of physical functions (difficult operations), the sink device 300 and 400 cannot interpret the operation content, and therefore the source device 200 is accessible by the source device 200. It is suitable to grasp the source device 200 so that the mode and the normal mode operation of the sink device 400 can be understood.

  Next, a method in which the source device 410 performs Capability Negotiation with the sink device 420 using the Miracast processing in the above approach (3) will be described with reference to FIG. By this capability negotiation method, it is possible to confirm whether the accessibility mode selected from the auditory, visual, and physical functions (difficult to operate) can be determined on the source device 410 side.

  As shown in a dotted-line rectangle 430 in FIG. 7, the sink device 420 returns the accessibility capability set on the sink device 420 side in a GET_PARAMETER Response in accordance with RTSP GET_PARAMETER Request from the source device 410. FIG. 7 shows an example of the reply contents of this GET_PARAMETER Response.

  Note that the following is used as an example of a message when the source device 410 exchanges accessibility capabilities with the sink device 420 according to the Miracast standard.

[Accessibility mode setting example]
10 and 11 show a sequence example in which Capability Negotiation communication processing is performed between the source device and the sink device in the communication system 100 according to the first embodiment. FIGS. 10 and 11 show a communication example (accessibility mode metadata setting example) for setting accessibility mode metadata in Capability Negotiation. This accessibility mode metadata setting method is an example in the case of adapting to the Wi-Fi CERTIFIED Miracast standard.

  When setting the metadata for accessibility mode, the Wfd-accessibility-capability is exchanged between the source device 410 and the sink device 420 by the M3 message (RTSP GET_PARAMETER). The purpose of this exchange is for the source device 410 to check whether or not the sink device 420 has a built-in accessibility mode metadata function.

  When this purpose is realized by the above approach (3), the bit assignment of the accessibility support request in FIG. 9 is simultaneously started up in each message of Capability Negotiation RTSP M3 in S601 and S602 in the communication sequence shown in FIG. The source device 200 determines that the sink device 300, 400 is a unified accessibility mode that can be set.

  For example, on the assumption that bit 0 is “1” (accessibility mode On), the Wfd-accessibility-capability Response information [14: 5] indicated by one sink device 300 in the message of S601 becomes 0000000101, and the message of S602 If the Wfd-accessibility-capabilityResponse information [14: 5] indicated by the other sink device 400 is 0000000100, the bit OR operation is performed on all the operation contents, and the first bit (“no vision” in the bit assignment shown in FIG. 9) 3 ”(corresponding to“ can be operated with low vision and softness ”in the bit assignment shown in FIG. 9) is“ 1 ”. For example, if the example of handling the handling request for the type of obstacle shown in FIG. 16 is followed, the source device 200 determines which of the following two can be generated as accessibility mode metadata.

・ Notification voice, Braille display data, operation voice / character size, screen enlargement level, information on contrast change

  When the source device 200 can determine that one of the two accessibility mode metadata can be generated, the source device 200 sets the corresponding accessibility mode as Wfd-accessibility-setting. Like that.

  In the Capability Negotiation communication method between the source device and the sink device shown in FIG. 10A, when the source device 410 receives the capability information of the sink device 420 from the sink device 420, the accessibility mode is sent to the sink device 420 by an M4 message. A metadata line setting request is made. Note that the connection request may be made with the M14 message after the image transmission is started without sending the accessibility mode metadata line setting request with the M4 message. Further, when transmitting the M14 message, not only the request from the source device 410 but also the metadata line setting request for accessibility mode may be made from the sink device 420 to the source device 410 as shown in FIG. 11A.

  Also, when w4d-accessibility-setting and wfd-accessibility-setting can be exchanged with M4, the metadata for accessibility mode becomes Enable. However, when M14 message is exchanged, M15 message The metadata for accessibility mode does not become enabled until is received. For this reason, as shown in FIGS. 10B and 11B, the source device 410 or the sink device 420 exchanges the M15 message after receiving a response to the accessibility mode metadata line setting request. As a result, the accessibility mode metadata line is enabled.

  In the example of the operation sequence illustrated in FIG. 11, after the video transmission is started between the source device 410 and the sink device 420, the accessibility mode metadata line is requested from the sink device 420. In addition, about FIG. 11A, as shown to FIG. 10A, it shall also be set at the time of Capability Negotiation before video transmission start.

  As described above, whether or not the source device and the sink device are compatible with the accessibility mode can be confirmed by the Capability Negotiation at the time of connection. Furthermore, in this embodiment, the accessibility mode type is determined by Capability Negotiation, but other types may be used. For example, Device Discovery or Service Discovery may be used. As an example of the control method of the Device Discovery or the Service Discovery, a P2P IE (Information Element), a WFD IE (Information Element), or a standard (PlP of the Application Service Platform), or Lu (PnP of the Application Service Platform) is used. .

  In addition, a channel for transmitting accessibility mode metadata (data necessary for realizing the corresponding operation example of FIG. 8, such as voice, video, text, and vibration) from the source device to the sink device should be newly established. You can also. Also, determine whether to make the accessibility mode metadata meaningful, or according to the setting mode, whether the UIBC transmitted from the sink device to the source device can be played even if the connection setting is not enabled. And can choose.

  FIG. 12 summarizes the operation examples of the sink device in each accessibility mode type by setting the accessibility mode metadata line to Enable.

  In the figure, the “Accessibility” (visual support, auditory support, operational difficulty support, cognitive support) column indicates the mode type set between the source device and the sink device, and the “media data” column is This indicates a medium of content data, and corresponds to data transmitted by RTP in the Miracast standard.

  “Accessibility data” indicates media (accessibility mode metadata) transmitted using the accessibility metadata line. For example, RTSP or TCP is used as the protocol of the metadata line for accessibility.

  Each column of “Accessibility mode support” or “Normal mode support” shows an example of the difference in display mode between when the accessibility mode is selected on the sink device and when the normal mode is selected. ing.

  For example, when the accessibility mode type is “visually impaired” and “operable without visual acuity” (see FIG. 8), voice and tactile operations are possible. In order to make this possible, audio is transmitted as media data, and operation audio is transmitted from the source device to the sink device as accessibility mode metadata.

  The sink device that has received the audio for media data and the operation audio as metadata for accessibility mode has an accessibility operation button on the screen of the sink device shown in FIG. In the case of ON), the media sound is muted or the volume is lowered, and the operation sound is reproduced mainly. On the other hand, as a response in the normal mode (for example, when the default operation button is on in the sink device screen shown in FIG. 17), processing is performed such that media sound is given priority and operation sound is not reproduced.

  FIG. 13A and FIG. 13B show a processing procedure for the source device to select a common accessibility mode from a plurality of accessibility mode types for sink devices in the form of a flowchart.

  First, the control unit 240 of the source device 200 starts Capability Negotiation with each sink device to be connected (exists in the information transmission range 101) (step S902).

  Subsequently, after confirming the accessibility mode that can be connected to each sink device, the control unit 240 determines whether or not to request an operation in the accessibility mode (step S903). For a sink device that does not require an operation in the accessibility mode (No in step S903), the source device 200 starts communication in the normal mode.

  Further, the control unit 240 of the source device 200 registers and holds the sink device that requests the operation in the accessibility mode (Yes in Step S903) in the metadata management list (Step S904).

  When confirmation to all sink devices to be connected has not been completed (No in step S905), the process returns to step S902 to repeat the same processing as described above. When the confirmation to all the sink devices to be connected is completed (Yes in step S905), the control unit 240 proceeds to the process in the next step S906.

  The source device 200 checks whether it is possible to group each sink device having the same accessibility mode type (step S906).

  When the sink devices are not grouped for each accessibility mode type (No in step S906), all accessibility mode metadata is transmitted to all sink devices to be connected (step S907), and this processing is performed. Exit.

  On the other hand, when the sink device is grouped and managed for each accessibility mode type (Yes in step S906), the source device 200 confirms the accessibility mode type (step S908). If it is not necessary to consider the type of accessibility mode or the type information is not included (No in step S909), the mode with the highest priority is selected from among the accessibility modes. FIG. 13 shows a case where visual support has the highest priority.

  In steps S910, S911, and S912, the source device 200 determines whether the accessibility mode of the group is visual support, auditory support, or physical (operation difficulty) support from the capability information received from the sink device.

  When the accessibility mode of the group is visual support (Yes in step S910), the source device 200 provides audio synchronized with the content media transmission as the metadata for the accessibility mode (for example, the contents of the application are read aloud). Process), or the name of the selected icon selected when the operator taps is changed to a process of providing by voice (step S913) and transmitted to each sink device allocated to the group (step S916). ). In addition, examples of the audio providing method include a method of providing as a part of audio of accessibility mode content media and a method of inserting audio as accessibility mode metadata, but are not limited thereto. .

  When the accessibility mode of the group is auditory support (Yes in step S911), the source device 200 provides subtitles synchronized with the content / media transmission or set by the operator as accessibility mode metadata. The voice notification is converted into an LED flash (step S914) and transmitted to each sink device assigned to the group (step S916). As a method for providing subtitles and LED flash, there are a method of providing as a part of content media and a method of providing as metadata for accessibility mode, but are not limited thereto. In addition, the packet type when wirelessly transmitting content media and accessibility mode metadata may be either a multicast packet or a unicast packet.

  When the accessibility mode of the group is physical (operation difficulty) support (Yes in step S912), the source device 200 converts the gesture operation data created independently into pointing that can be understood by the sink device ( In step S915), a process of transmitting to each sink device assigned to the group is performed (step S916). Note that the packet type used when wirelessly transmitting the content media to be transmitted and the metadata for accessibility mode may be either a multicast packet or a unicast packet.

  Step S917 is processing for ending this operation flow.

  When the terminal mode or accessibility type is different in the transmission or reception terminal, the operation means may be converted by transmission or reception to match the modes.

  As described above, the wireless communication unit 220 of the source device 200 performs real-time image transmission with the sink device according to the Wi-Fi CERTIFIED Miracast specification. In addition, the control unit 240 displays an image based on the image information transmitted from the source device on a display unit (not shown in FIG. 2), and exchanges accessibility mode metadata regarding the image with the sink device. Control at ports used for a plurality of communication methods (UIBC, WebSocket). In this case, for example, when the port is set on the TCP session generated with the sink device, the control unit 240 specifies information specifying a plurality of communication methods using the port (for example, FIG. 15 (described later). )) To the sink device.

  The control unit 240 can set a plurality of ports for each of a plurality of communication methods as ports used for a plurality of communication methods. In this case, the control unit 240 can set a plurality of ports in response to a request from the sink device.

  Further, the control unit 240 can set one port for a plurality of communication methods as a port used for a plurality of communication methods.

  For example, the control unit 240 of the information processing device 200 transmits image information for displaying an image on the display unit on the sink device side to the sink device. In addition, the control unit 240 can perform control to set ports used in a plurality of communication methods for exchanging control information regarding the image with the sink device.

[Second Embodiment]
[Communication system configuration]
FIG. 16 schematically illustrates a configuration example of a communication system 1100 according to the second embodiment of the technology disclosed in this specification. FIG. 16 shows an example of a communication system that can perform wireless communication by P2P direct communication.

  The communication system 1100 includes a plurality of information processing apparatuses 1200, 1300, and 1400. The communication system 1100 is configured such that the information processing apparatus 1300 receives data (for example, image data and audio data) transmitted from at least one of the information processing apparatus 1200 or the information processing apparatus 1400. . Each of the information processing apparatuses 1200, 1300, and 1400 is a transmission / reception device having a wireless communication function.

  Hereinafter, the operation of the communication system 1100 will be described by taking as an example a case where wireless communication using a wireless LAN is performed between the information processing device 1200 and the information processing device 1300 or between the information processing device 1400 and the information processing device 1300.

  The illustrated communication system 1100 is assumed to perform wireless connection in a multi-source topology environment using P2P direct communication. For example, the information processing apparatuses 1200 and 1400 are source devices, and the information processing apparatus 1300 is a sink device. Specifically, image data generated by the imaging operation of the information processing device 1200 is transmitted to the information processing device 1300, and an image 111 based on the image data is displayed on the display unit 1351 of the information processing device 1300. In addition, content stored in a storage unit (for example, a hard disk) of the information processing device 1400 is transmitted to the information processing device 1300, and an image 112 based on the content is displayed on the display unit 1351 of the information processing device 1300. The information processing apparatus 1300 may include an imaging unit 1361 that images an operator or the like.

  In FIG. 16, a range in which the information processing apparatus 1300 can directly communicate using wireless communication by P2P direct connection is illustrated as an information transmission range 1101. This information transmission range 1101 is an information transmission range (service range) when the information processing apparatus 1300 is used as a reference.

  In such a multi-source topology environment, it is assumed that terminals in normal mode and terminals in accessibility mode are mixed. FIG. 14A illustrates a connection form in which both the accessibility mode source device and the normal mode source device interact with one accessibility mode sink device. FIG. 14B illustrates a connection form in which both the normal mode and accessibility mode source devices communicate with one normal mode sink device.

  The multi-source topology environment is different from the multi-sink topology environment described as the first embodiment. In the multi-source topology environment, the following approaches (11) and (12) are possible as in the first embodiment.

(11) In multi-source, the sink device responds to each source device so as to match the modes of the plurality of source devices, prompts the source device to match the modes of each terminal, and then starts image transfer.

(12) The sink device does not perform arbitration to unify the mode with the plurality of source devices as in (11) above, and converts the data into data that can be handled by the terminal based on the data received from the source device Let it play.

  On the other hand, the difference from the first embodiment is that UIBC data is transmitted from a sink device to a plurality of source devices in a multi-source topology environment. The following (21) to (23) can be given as the UIBC packet format exchanged between the source device and the sink device.

(21) A method of transmitting after converting the UIBC data so as to match the mode of each source device

(22) A method in which the sink device transmits UIBC data regardless of the mode of the source device, and converts each source device into data understandable by its own terminal for internal processing

(23) A method of transmitting and receiving after switching to a common mode in which all devices including the source device and sink device can be switched

  In addition, as a use case in a multi-source topology environment, a case where a large screen display display is shared by the owner of the source device can be considered. In this use case, there is a possibility that a case where a normal person operates a large screen display and a case where a person with a disability operates time division or simultaneously occur. Therefore, when accessibility mode is connected as the source device, it is possible to switch or use it simultaneously from the display screen of the sink device, whether to operate the accessibility mode or to operate the default display device. Shall. Note that before changing the accessibility mode type, the sink device may display a pop-up for confirming whether the change is acceptable to the operator on the operated sink device.

  FIG. 17 illustrates a large screen display 1700 including buttons for designating accessibility operations and default operations. The large screen display 1700 operates as a sink device in a multi-source topology environment, and includes an accessibility operation button 1711 and a default operation button 1712 on a touch panel screen, for example. The operator selects one of the accessibility operation button 1711 and the default operation button 1712 to instruct whether to perform an operation for the accessibility mode or to perform an operation of the default display device. Can do.

  Furthermore, when the operator changes the mode of the sink device during connection by switching or simultaneously using the accessibility operation and the default operation, a message message for mode exchange that changes the normal mode and accessibility mode in the middle ( By transmitting or receiving the above) during transmission, even in an environment changed by the operator, an efficient operation can be performed without returning to the initial operation.

  Further, since the sink device transmits information related to the display position of the sink device based on a request from the source device, the UIBC control channel can be switched to the requested source device during connection. Further, since the source device transmits the display position information of the source device by an operation from the sink device, the UIBC control channel can be switched to the requested sink device during connection.

[Transmission example of metadata transmission for accessibility mode using WebSocket]
It is also possible to send and receive image data, audio data, and text data using a WebSocket control line as the accessibility mode metadata line. Therefore, an example in which image data and audio data are transmitted and received using a WebSocket control line will be described below.

  FIG. 15 illustrates a communication processing example of image data and audio data using WebSocket between the source device 410 and the sink device 420 in the first embodiment. FIG. 15A shows an example of communication processing between the source device 410 and the sink device 420. 15B and 15C show examples of the frame format of WebSocket exchanged between the source device 410 and the sink device 420. FIG.

  15A, the Wi-Fi CERTIFIED Miracast source device 410 functions as a WebSocket client, and the sink device 420 functions as a WebSocket server. It is assumed that one source device and one sink device are connected to each other, so either server or client may be used. However, in an environment where multiple devices are connected, a GO (group It is desirable for the owner to operate as a WebSocket server function.

  For example, the source device 410 transmits Host, Sec-WebSocket-Key, and Sec-WebSocket-Version to the sink device 420 as indicated by reference numeral 441. Host designates the connection destination of WebSocket. Sec-WebSocket-Key is for obtaining a handshake response. Sec-WebSocket-Version is for designating the version of WebSocket.

  In addition, the sink device 420 creates a value of Sec-WebSocket-Accept based on the value of Sec-WebSocket-Key received from the source device 410. Then, the sink device 420 transmits the created Sec-WebSocket-Accept to the source device 410 as indicated by reference numeral 442. Here, Sec-WebSocket-Accept is for maintaining a connection with the client.

  Further, the source device 410 transmits the frame format shown in FIG. 15B to the sink device. In this frame format, data to be transmitted can be specified by a WebSocket Type ID. FIG. 15C shows an example of this WebSocket Type ID.

  Further, as indicated by reference numeral 443 in FIG. 15A, image data and audio data are exchanged as binary data of WebSocket between the source device 410 and the sink device 420.

  For example, when FIN is 0x0, since the payload continues, the sink device 420 connects to the immediately preceding data. When FIN is 0x1, the sink device 420 processes the payload because the payload is a completed flag.

  For example, RSV is an extension bit.

  Further, for example, when the opcode is 0x0, since the payload continues, the sink device 420 connects to the immediately preceding data. When the opcode is 0x1, it means that the payload is text, and when the opcode is 0x2, it means that the payload is binary.

  For example, MASK is a bit indicating whether a message is encoded. Note that the message from the source device 410 to the sink device 420 is always encoded.

  As described above, when opcode binary is used, image data and audio data can be exchanged as binary data of WebSocket between the source device 410 and the sink device 420. That is, image data and audio data can be transmitted using the WebSocket standard format. For this reason, pre-authentication can be performed using image data or audio data. Moreover, it can respond to various other use cases.

  Further, the exchange of image data and audio data between the source device and the sink device can be performed using WebSocket while performing image transmission of the Wi-Fi CERTIFIED Miracast standard. In addition, WebSocket may be used for exchanging operation metadata.

  As described above, the control unit 370 of the information processing apparatus 300 as the sink device can acquire the image information and the audio information transmitted from the source device using WebSocket. Further, the control unit 370 can display an image based on the image information on the display unit 351, and can output a sound based on the sound information from the sound output unit 352.

  As described above, in the first embodiment, the accessibility mode operation of the source device can be accepted by the sink device. That is, in an environment in which images, audio, text, and auxiliary operation information from the source device are displayed on the display unit or speaker on the sink device side, the operation can be stably performed, and connection control operations with real-time properties are performed be able to.

  In addition, by notifying the sink device in real time of the display position of each image on the source device, the display position can be controlled from the source device using the accessibility mode metadata. In addition, information regarding the display position of the accessibility mode metadata can be exchanged simultaneously using UIBC and WebSocket.

  As described above, the technology disclosed in this specification has been described in detail with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the scope of the technology disclosed in this specification. That is, the technology disclosed in the present specification has been described in the form of exemplification, and the description content of the present specification should not be interpreted in a limited manner. In order to determine the gist of the technology disclosed in this specification, the claims should be taken into consideration.

  According to the technology disclosed in this specification, in a multi-sink topology environment, a source device manages the modes of a plurality of sink devices, and makes a request for matching in some cases. Can be effectively transmitted.

  In addition, by confirming the mode change to the operator in advance when the mode is changed, the operator can understand the operation change, and the operator can grasp the difference in operation associated with the mode change as much as possible.

  Also, according to the technology disclosed in this specification, in a multi-sync topology environment, the source device arbitrates between the modes of a plurality of sink devices, so that each terminal manages and exchanges with the mode of its own terminal. It can be done to the minimum.

  Further, according to the technique disclosed in this specification, bandwidth control using a multicast packet or a unicast packet can be performed.

  Further, according to the technology disclosed in this specification, in a multi-sync topology environment, the source device can minimize the data conversion of the sink device.

  Further, according to the technology disclosed in this specification, even if the operator who operates the sink device is a disabled person, it is possible to perform a remote operation by providing a mode selection UI that can operate the source device from the sink device. .

  Further, according to the technology disclosed in this specification, when a connection is made with a source device having a different operation method in a multi-source topology environment, conversion processing for each source device is performed from a common operation from the sink device. Remote operation can be performed.

  Further, according to the technology disclosed in this specification, even when the operator of the connected source device or sink device changes, an efficient operation can be performed without returning to the initial operation. In a multi-source topology environment, the sink device can perform Capability Negotiation management so that the modes of a plurality of source devices match.

Note that the technology disclosed in the present specification can also be configured as follows.
(1) a wireless communication unit that wirelessly communicates as a source device with a plurality of sink devices;
A control unit for controlling connection with the sink device and transmission of content data to the sink device;
Comprising
Each of the plurality of sink devices is operable in one of a normal mode and one or more types of accessibility modes that guarantee browsing corresponding to an operator's physical disability or reduced functionality,
The control unit manages types of accessibility modes of the plurality of sink devices;
Wireless communication device.
(2) The control unit arbitrates accessibility modes of the plurality of sink devices, and then starts transmission of content data.
The wireless communication device according to (1) above.
The wireless communication device according to the above (1) or (2) manages the modes of a plurality of sink devices in a multi-sync topology environment, and makes a request for matching in some cases, so that the meta-data for accessibility mode is obtained. Data can be transmitted effectively.
(3) The control unit displays a pop-up to confirm to the operator whether there is no problem with the change to the sink device to change to the accessibility mode.
The wireless communication device according to (2) above.
According to the wireless communication device described in (3) above, when the mode is changed, by confirming the mode change to the operator in advance, the operator can understand the change of operation, and the user can minimize the difference in operation associated with the mode change. I can figure it out.
(4) The control unit grasps that it is connected to the sink device in the accessibility mode, and based on the result of taking the logical sum of the accessibility mode type to be connected, the necessary accessibility mode meta Send data,
The wireless communication device according to (1) above.
According to the wireless communication device described in (4) above, in a multi-sync topology environment, the mode of a plurality of sink devices is arbitrated so that each terminal can manage and exchange with its own terminal mode to a minimum. Can do.
(5) The control unit causes accessibility mode metadata to be transmitted in either a multicast packet or a unicast packet.
The wireless communication device according to (4) above.
(6) The control unit groups the plurality of sink devices according to the same accessibility mode type, and transmits accessibility mode metadata corresponding to each group.
The wireless communication device according to (1) above.
According to the wireless communication device described in (6) above, data conversion of the sink device can be minimized.
(7) The control unit causes accessibility mode metadata to be transmitted in either a multicast packet or a unicast packet.
The wireless communication device according to (6) above.
(8) a wireless communication unit that performs wireless communication with the source device as a sink device;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
The control unit extracts and reproduces only reproducible metadata among a plurality of accessibility mode metadata received from the source device,
Wireless communication device.
(9) a wireless communication unit that performs wireless communication as a source device to the same sink device together with other source devices;
A control unit for controlling connection with the sink device and transmission of content data to the sink device;
Comprising
The sink device can operate in either a normal mode and one or more types of accessibility modes that ensure browsing in response to a physical disability or reduced functionality of the operator,
The control unit causes the sink device to be notified of connection in accessibility mode.
Wireless communication device.
According to the wireless communication device described in (9) above, it is possible to provide a UI that can operate the source device from the sink device even if the operator who operates the sink device is a disabled person.
(10) a wireless communication unit that performs wireless communication as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
A user interface that selects between accessibility and normal operations,
A wireless communication apparatus comprising:
According to the wireless communication device described in (10) above, it is possible to provide a UI that can operate the source device from the sink device even if the operator who operates the sink device is a disabled person.
(11) a wireless communication unit that performs wireless communication as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
The control unit causes UIBC data to be transmitted to the plurality of source devices.
Wireless communication device.
(12) The control unit causes the UIBC data to be transmitted after being converted to match the mode of each source device.
The wireless communication device according to (11) above.
(13) The control unit causes the UIBC data to be transmitted regardless of the mode of the source device.
The wireless communication device according to (11) above.
(14) The control unit transmits UIBC data after switching to a common mode in which all devices can be switched.
The wireless communication device according to (11) above.
According to the wireless communication device described in the above (11) to (14), even when connecting to a source device having a different operation method, conversion processing for each source device is performed from a common operation from the sink device, and remote operation is performed. Can be done.
(15) a wireless communication unit that performs wireless communication as a source device to the same sink device together with other source devices;
A control unit for controlling connection with the sink device and transmission of content data to the sink device;
Comprising
The sink device can operate in either a normal mode and one or more types of accessibility modes that ensure browsing in response to a physical disability or reduced functionality of the operator,
When the operator changes the mode of the sink device during connection, the control unit causes a mode exchange message signal to be changed in the middle of transmission between the normal mode and the accessibility mode.
Wireless communication device.
According to the wireless communication device described in (15) above, even in an environment changed by the operator, an efficient operation can be performed without returning to the initial operation.
(16) a wireless communication unit that performs wireless communication as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
The control unit causes each source device to request that the accessibility modes of the plurality of source devices match.
Wireless communication device.
According to the wireless communication device described in (16) above, in multi-source, the sink device can perform Capability Negotiation management so that the modes of a plurality of source devices are matched.
(17) a wireless communication unit that performs wireless communication as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
The control unit grasps that it is connected to the source device in the accessibility mode, and causes the source device to transmit based on the result of taking the logical sum of the accessibility mode type to be connected.
Wireless communication device.
According to the wireless communication device described in (17) above, in the multi-source, the sink device can perform the capability negotiation management so that the modes of the plurality of source devices are matched.
(18) a wireless communication step of performing wireless communication as a source device with a plurality of sink devices;
A control step for controlling connection to the sink device and transmission of content data to the sink device;
Have
Each of the plurality of sink devices is operable in one of a normal mode and one or more types of accessibility modes that guarantee browsing corresponding to an operator's physical disability or reduced functionality,
In the control step, the type of accessibility mode of the plurality of sink devices is managed.
Wireless communication method.
(19) a wireless communication step of performing wireless communication as a sink device with a plurality of source devices;
A control step for controlling connection with the source device and transmission of content data from the source device;
Have
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
In the control step, UIBC data is transmitted to the plurality of source devices.
Wireless communication method.
(20) a wireless communication step of performing wireless communication as a source device to the same sink device together with other source devices;
A control step for controlling connection to the sink device and transmission of content data to the sink device;
Have
The sink device can operate in either a normal mode and one or more types of accessibility modes that ensure browsing in response to a physical disability or reduced functionality of the operator,
In the control step, when the mode of the sink device is changed while the operator is connected, a message message for mode exchange that changes the normal mode and the accessibility mode in the middle is transmitted in the middle of transmission.
Wireless communication method.

DESCRIPTION OF SYMBOLS 100 ... Communication system, 101 ... Information transmission range 200 ... Information processing apparatus, 210 ... Antenna, 220 ... Wireless communication part 230 ... Control signal receiving part, 240 ... Control part 250 ... Image / audio signal generation part, 260 ... Image / audio Compression unit 270 ... Stream transmission unit 300 ... Information processing device, 310 ... Antenna, 320 ... Wireless communication unit 330 ... Stream reception unit, 340 ... Image / sound development unit 350 ... Image / sound output unit, 351 ... Display unit 352 ... Audio Output unit, 360 ... User information acquisition unit 370 ... Control unit, 380 ... Control signal transmission unit, 390 ... Management information holding unit 1100 ... Communication system 1200, 1300, 1400 ... Information processing device 1351 ... Display unit, 1361 ... Imaging unit

Claims (20)

A wireless communication unit that wirelessly communicates as a source device with a plurality of sink devices;
A control unit for controlling connection with the sink device and transmission of content data to the sink device;
Comprising
Each of the plurality of sink devices is operable in one of a normal mode and one or more types of accessibility modes that guarantee browsing corresponding to an operator's physical disability or reduced functionality,
The control unit manages types of accessibility modes of the plurality of sink devices;
Wireless communication device. The control unit arbitrates accessibility modes of the plurality of sink devices, and then starts transmission of content data.
The wireless communication apparatus according to claim 1. The control unit displays a pop-up to confirm to the operator whether there is no problem with the change to the sink device to change to the accessibility mode.
The wireless communication apparatus according to claim 2. The control unit grasps that it is connected to an accessibility mode sink device, and transmits necessary accessibility mode metadata based on the result of ORing the accessibility mode types to be connected. Let
The wireless communication apparatus according to claim 1. The control unit causes accessibility mode metadata to be transmitted in either a multicast packet or a unicast packet.
The wireless communication apparatus according to claim 4. The control unit groups the plurality of sink devices for the same accessibility mode type, and transmits accessibility mode metadata corresponding to each group.
The wireless communication apparatus according to claim 1. The control unit causes accessibility mode metadata to be transmitted in either a multicast packet or a unicast packet.
The wireless communication apparatus according to claim 6. A wireless communication unit that wirelessly communicates with the source device as a sink device;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
The control unit extracts and reproduces only reproducible metadata among a plurality of accessibility mode metadata received from the source device,
Wireless communication device. A wireless communication unit that performs wireless communication as a source device to the same sink device together with other source devices;
A control unit for controlling connection with the sink device and transmission of content data to the sink device;
Comprising
The sink device can operate in either a normal mode and one or more types of accessibility modes that ensure browsing in response to a physical disability or reduced functionality of the operator,
The control unit causes the sink device to be notified of connection in accessibility mode.
Wireless communication device. A wireless communication unit that wirelessly communicates as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
A user interface that selects between accessibility and normal operations,
A wireless communication apparatus comprising: A wireless communication unit that wirelessly communicates as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
The control unit causes UIBC data to be transmitted to the plurality of source devices.
Wireless communication device. The control unit causes the UIBC data to be transmitted after being converted to match the mode of each source device,
The wireless communication apparatus according to claim 11. The control unit transmits UIBC data regardless of the mode of the source device.
The wireless communication apparatus according to claim 11. The control unit causes UIBC data to be transmitted after switching to a common mode in which all devices can be switched.
The wireless communication apparatus according to claim 11. A wireless communication unit that performs wireless communication as a source device to the same sink device together with other source devices;
A control unit for controlling connection with the sink device and transmission of content data to the sink device;
Comprising
The sink device can operate in either a normal mode and one or more types of accessibility modes that ensure browsing in response to a physical disability or reduced functionality of the operator,
When the operator changes the mode of the sink device during connection, the control unit causes a mode exchange message signal to be changed in the middle of transmission between the normal mode and the accessibility mode.
Wireless communication device. A wireless communication unit that wirelessly communicates as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
The control unit causes each source device to request that the accessibility modes of the plurality of source devices match.
Wireless communication device. A wireless communication unit that wirelessly communicates as a sink device with a plurality of source devices;
A control unit for controlling connection with the source device and transmission of content data from the source device;
Comprising
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
The control unit grasps that it is connected to the source device in the accessibility mode, and causes the source device to transmit based on the result of taking the logical sum of the accessibility mode type to be connected.
Wireless communication device. A wireless communication step of wirelessly communicating as a source device with a plurality of sink devices;
A control step for controlling connection to the sink device and transmission of content data to the sink device;
Have
Each of the plurality of sink devices is operable in one of a normal mode and one or more types of accessibility modes that guarantee browsing corresponding to an operator's physical disability or reduced functionality,
In the control step, the type of accessibility mode of the plurality of sink devices is managed.
Wireless communication method. A wireless communication step of performing wireless communication as a sink device with a plurality of source devices;
A control step for controlling connection with the source device and transmission of content data from the source device;
Have
Each of the plurality of source devices is operable in either a normal mode and one or more types of accessibility modes that ensure browsing in response to an operator's physical disability or reduced functionality,
In the control step, UIBC data is transmitted to the plurality of source devices.
Wireless communication method. A wireless communication step of performing wireless communication as a source device to the same sink device together with other source devices;
A control step for controlling connection to the sink device and transmission of content data to the sink device;
Have
The sink device can operate in either a normal mode and one or more types of accessibility modes that ensure browsing in response to a physical disability or reduced functionality of the operator,
In the control step, when the mode of the sink device is changed while the operator is connected, a message message for mode exchange that changes the normal mode and the accessibility mode in the middle is transmitted in the middle of transmission.
Wireless communication method.