TWI686067B - Information processing apparatus, communication system, information processing method, and program - Google Patents

Abstract

Provided is an information processing apparatus that includes circuitry configured to receive a multiplexed image signal and sound signal from another information processing apparatus using a moving picture experts group (MPEG) 2-transport stream (TS); and perform control to cause reduction of data contained in a packetized elementary stream (PES) payload in a PES packet packed in a transport (TS) packet specified by a packet identifier (PID) specifying sound data transmission described in a program map table (PMT) and to extract a presentation time stamp (PTS) contained in a PES header portion of the PES packet after requesting the another information processing apparatus to reduce a sound data amount.

Description

Information processing equipment, communication system, information processing method, and program [Cross-reference to related applications]

This application claims the priority of the Japanese patent application JP 2015-049332 filed on March 12, 2015, the entire contents of which are incorporated herein by reference.

This disclosure relates to information processing equipment. More specifically, the present disclosure relates to information processing equipment, communication systems, information processing methods that can use wireless communication to exchange information, and programs that cause computers to execute the methods.

In recent years, information processing equipment that uses a wireless local area network (LAN) to perform wireless communication has been widely used. As for this wireless LAN, for example, one of the wireless LANs represented by the Institute of Electrical and Electronics Engineers (IEEE) 802.11 has been widely used.

Furthermore, as for audio visual (AV) transmission communication, for example, wireless fidelity (Wi-Fi) Fi) CERTIFIED Miracast has been proposed (see for example PTL 1).

[PTL 1]JP 2014-96074A

In the aforementioned related fields, real-time image transmission can be performed between two information processing devices according to the specifications of Wi-Fi CERTIFIED Miracast. For example, an image based on the image data transmitted from the information processing device at the transmitting end can be displayed on the display of the information processing device at the receiving end.

Here, it is assumed that a plurality of images with sound are displayed on the display of the information processing device at the receiving end. It is also assumed that in at least one image set to a main image and other images set to a secondary image, only the sound of the main image is output. In this case, the sound of the image is processed differently based on the display form of the image, and therefore it is important to properly process the sound data exchanged between the devices.

In consideration of the situation described above, it is expected that sound materials can be appropriately processed.

The first embodiment of the present technology is directed to an information processing device, its information processing method, and a non-transitory computer-readable medium storing a program that causes a computer to execute the method. The information processing device includes: a circuit, which It is configured to use a Motion Picture Experts Group (MPEG) 2 transport stream (TS) to receive a multiplexed video signal and audio signal from another information processing device; and is requesting the other information processing device to reduce one After the volume of sound data, control is performed to cause the transmission of a transport stream (TS) packet included in the sound data specified in a program map table (PMT) specified by a packet identifier (PID). The reduction of data in a PES payload of a PES packet and extraction of a presentation time stamp (PTS) contained in a PES header portion of the PES packet.

In this first embodiment, the controller can make a request to the other information processing device based on a first trigger. In this way, an embodiment of the present disclosure exerts the effect of making a request to another information processing device based on the first trigger.

In this first embodiment, when making the request to another information processing device, the circuit specifies one of a first sound data reduction method and a second sound data reduction method, including The MPEG2-TS multiplex signal of the PES packet is generated and transmitted in the first sound data reduction method, and the MPEG2-TS multiplex signal of the PES packet that does not include the sound data is generated and transmitted Transmitted in the second sound data reduction method. In this way, an embodiment of the present disclosure exerts the following effect: when a request is made to another information processing device, the circuit indicates one of a first sound data reduction method and a second sound data reduction method, including The MPEG2-TS multiplex signal of the PES packet carrying sound data is generated and transmitted to the first sound In the data reduction method, the MPEG2-TS multiplex signal that does not include the PES packet carrying sound data is generated and transmitted in the second sound data reduction method.

In this first embodiment, when the other information processing device is requested to restore an original volume of sound data based on a second trigger, the controller can perform control to extract the sound data contained in the PES payload while receiving The volume of the audio data is reduced to the MPEG2-TS multiplex signal and the MPEG2-TS multiplex signal received from the other information processing device after the request is demultiplexed. In this way, an embodiment of the present disclosure exerts the following effect: when another information processing device is requested to restore an original sound data amount based on the second trigger, the controller can perform control to extract the sound data contained in the PES payload At the same time, the amount of MPEG2-TS multiplex signal received at the same time is reduced and the MPEG2-TS multiplex signal received from another information processing device after the request is demultiplexed.

In the first embodiment, the first trigger and the second trigger can be caused by user operations. In this way, an embodiment of the present disclosure exerts the following effect: using the first trigger and the second trigger caused by user operation.

In the first embodiment, the first trigger may be caused by detection of the user leaving the information processing device or movement of the user. In this way, an embodiment of the present disclosure exerts the following effect: the user triggers the first trigger caused by detection around the information processing device or movement of the user.

In the first embodiment, the controller may retain a sound volume setting of the sound output before the request is made, request the other information processing device to restore an original sound data volume, and when included in the PES The sound data in the payload is the set value of the sound volume retained at a timing setting extracted after the request. In this way, an embodiment of the present disclosure exerts the following effects: it can retain a sound volume setting of the sound output before the request is made, request the other information processing device to restore an original sound data volume, and include The sound data in the PES payload is the sound volume setting value retained at a timing setting extracted after the request.

The second embodiment of the present technology is directed to an information processing device, its information processing method, and a non-transitory computer-readable medium storing a program that causes a computer to execute the method. The information processing device includes: an information processing device including a circuit , Which is configured to use a Motion Picture Experts Group (MPEG) 2 transport stream (TS) to transmit a multiplexed video signal and audio signal to another information processing device; based on the information from the other information processing device A request, without changing the packet identifier (PID), to reduce the transmission of a TS packet contained in the sound data stored in the program map table (PMT) specified by the PID One of the PES packets is divided into the data in the payload of the basic stream (PES) payload; and multiplex processing and transmission of PES packets for the transmission of audio signals in which the data volume is reduced and PES for the transmission of video signals Packet.

In the second embodiment, the controller can perform control to reduce the amount of sound data when requested from the other information processing device. By making the PES payload data amount substantially zero or completely zero, a data amount is reduced. In this way, one embodiment of the present disclosure exerts the following effect: when a volume of sound data is requested to be reduced from another information processing device, a volume of data is reduced by making the amount of PES payload data substantially zero or completely zero .

In the second embodiment, the controller can perform control to send PES packets for transmission of sound signals in which the amount of data is reduced by the reduction of the first sound data after multiplexing processing and for When a PES packet for video signal transmission is requested to restore an original volume of audio data from the other information processing device, the general audio data stored in the PES payload, and multiplex processing and transmission of the PES packet for audio signal transmission And PES packets for video signal transmission. In this way, an embodiment of the present disclosure exerts the following effect: when after multiplexing processing, the PES packet used for the transmission of the audio signal in which the amount of data is reduced by the reduction of the first audio data and the image When a PES packet for signal transmission is requested to restore an original volume of sound data from another information processing device, the general sound data stored in the PES payload, and multiplex processing and transmission of the PES packet for sound signal transmission and for PES packet for video signal transmission.

In the second embodiment, the controller may perform control to reduce the amount of sound data when requested from the other information processing device to reduce the amount of sound data using a sound data reduction other than a first sound data reduction. In the case of sound data in the PID information of the PMT, the PES packet carrying the sound data is not reduced by the second sound data generated therein, and there is no multiplexing to process a PES packet for sound signal transmission The next transmission is used to transmit one PES packet of the video signal. Yu Caidi In the second embodiment, the controller may perform control to reduce the amount of sound data when requested to use a sound data reduction other than a first sound data reduction from the other information processing device to perform In the case of the PID information in the PMT, the PES packet carrying sound data is not used for the transmission of the second sound data generated therein, and there is no multiplexing for one PES packet used for sound signal transmission. Send a PES packet in the video signal.

In the second embodiment, the controller can perform control to transmit PES for video signal transmission without multiplexing the PES packet for audio signal transmission after the second audio data reduction is performed Packets, using the PID specified for the transmission of the audio data contained in the PMT to store the general audio data in the PES payload, and multiplexing and transmitting PES packets for audio signal transmission and video signal transmission PES packet. In this way, an embodiment of the present disclosure exerts the following effect: when the second audio data reduction is performed, the PES packet for video signal transmission is transmitted without multiplexing the PES packet for audio signal transmission Utilize the PID that specifies the transmission of the audio data contained in the PMT to store the general audio data in the PES payload, and multiplex the processing and transmission of PES packets for audio signal transmission and PES packets for video signal transmission.

The third embodiment of the present technology is a communication system, its information processing method, and a program that causes a computer to execute the method. The communication system includes: a first information processing device configured to use an MPEG2-TS to transmit a Multiplexed video signal and audio signal to a second information processing device Based on a request from the second information processing device, without changing the PID, reduce the inclusion of a PES packet contained in a TS packet specified by the PID specified in the voice data transmission specified in the PMT A PES payload data volume is reduced by the first sound data generated by PES packets containing PTS in the PES header; and multiplexing and transmission of PES for transmission of sound signals in which the data volume is reduced Packets and PES packets for the transmission of video signals; and second information processing equipment, which is configured to discard the transmission of the audio data contained in the PMT specified in the PMT and stored in the TS packet in the PES The data in the packaged PES payload, and the use of MPEG2-TS to receive the multiplexed video and audio signals from the first information processing equipment and perform demultiplexing on the MPEG2 received from the first information processing equipment after the request -At the same time as the TS multiplex signal, when requesting the first information processing device to reduce the amount of sound data, extract the PTS contained in the header portion of the PES. In this way, one embodiment of the present disclosure exerts the following effect: based on a request from the second information processing device, the first information processing device, without changing the PID, reduces the content contained in the designation specified by the PID The amount of PES payload data of a PES packet of a TS packet, which is the transmission of voice data described in PMT; the first voice data generated by a PES packet that contains PTS in the PES header is reduced; and multiple processing And transmitting PES packets for the transmission of sound signals in which the amount of data is reduced and PES packets for the transmission of image signals; and the second information processing device discards the sound contained in the PMT specified by the PID The amount of data stored in the PES payload of the PES packet stored in the TS packet for data transmission; The information processing device reduces the amount of sound data and performs demultiplexing to extract the PTS included in the PES header portion when receiving the MPEG2-TS multiplexing signal received from the first information processing device after the request.

An embodiment of the present disclosure exerts an excellent effect of appropriately processing sound data. The effects described here are not necessarily limited, and any effects described in this disclosure can be exerted.

10‧‧‧Communication system

11‧‧‧ Broadcasting Station

21‧‧‧Image

22‧‧‧Image

23‧‧‧ video

50‧‧‧Source device

60‧‧‧Receiving device

71‧‧‧Process

72‧‧‧Process

73‧‧‧Handling

74‧‧‧Handling

75‧‧‧ group owner

76‧‧‧Customer

77‧‧‧Process

78‧‧‧Process

79‧‧‧Process

80‧‧‧Process

81‧‧‧Process

82‧‧‧Process

100‧‧‧ Information processing equipment

110‧‧‧Communication unit

111‧‧‧Data processing unit

112‧‧‧Transmission processing unit

113‧‧‧Wireless interface unit

114‧‧‧ Antenna

120‧‧‧Sensor

130‧‧‧Operation receiving unit

140‧‧‧Monitor

150‧‧‧Sound output unit

160‧‧‧Memory

170‧‧‧Controller

200‧‧‧ Information processing equipment

201‧‧‧Monitor

210‧‧‧ Information processing equipment

211‧‧‧Monitor

900‧‧‧smartphone

901‧‧‧ processor

902‧‧‧Memory

903‧‧‧storage

904‧‧‧External connection interface

906‧‧‧Camera

907‧‧‧sensor

908‧‧‧Microphone

909‧‧‧Input device

910‧‧‧Display device

911‧‧‧speaker

913‧‧‧Wireless communication interface

914‧‧‧antenna switch

915‧‧‧ Antenna

917‧‧‧Bus

918‧‧‧Battery

919‧‧‧ auxiliary controller

920‧‧‧Car navigation device

921‧‧‧ processor

922‧‧‧Memory

924‧‧‧GPS (Global Positioning System) module

925‧‧‧Sensor

926‧‧‧Data interface

927‧‧‧Content player

928‧‧‧ Storage media interface

929‧‧‧Input device

930‧‧‧Display device

931‧‧‧speaker

933‧‧‧Wireless communication interface

934‧‧‧ Antenna switch

935‧‧‧ Antenna

938‧‧‧Battery

940‧‧‧In-vehicle system

941‧‧‧Car network

942‧‧‧Carrier side module

950‧‧‧Wireless access point

951‧‧‧Controller

952‧‧‧Memory

954‧‧‧Input device

955‧‧‧Display device

957‧‧‧Web interface

958‧‧‧Wired communication network

963‧‧‧Wireless communication interface

964‧‧‧antenna switch

965‧‧‧ antenna

FIG. 1 is a diagram showing a configuration example of a communication system 10 according to the first embodiment of the present disclosure.

FIG. 2 is a block diagram showing a functional configuration example of an information processing device 100 according to an embodiment of the present disclosure.

FIG. 3 is a sequence diagram showing a communication processing example of the source device 50 and the sink device 60 based on an embodiment of the present disclosure.

FIG. 4 is a sequence diagram showing a communication processing example of the source device 50 and the sink device 60 according to an embodiment of the present disclosure.

FIG. 5 is a sequence diagram showing a communication processing example of the source device 50 and the sink device 60 based on an embodiment of the present disclosure.

FIG. 6 is a diagram showing an example of an RTSP message exchanged between the source device 50 and the sink device 60 according to an embodiment of the present disclosure.

FIG. 7 is a diagram showing an example of an RTSP message exchanged between the source device 50 and the sink device 60 according to an embodiment of the present disclosure.

FIG. 8 is a diagram showing an example of information exchanged between information processing apparatuses according to the first embodiment of the present disclosure.

FIG. 9 is a diagram showing an example of information exchanged between information processing apparatuses according to the first embodiment of the present disclosure.

FIG. 10 is a diagram showing an example of information exchanged between information processing apparatuses according to the first embodiment of the present disclosure.

FIG. 11 is a diagram showing an example of information exchanged between information processing apparatuses according to the first embodiment of the present disclosure.

FIG. 12 is a diagram showing a transformation example of a screen displayed from the display 140 in the information processing apparatus 100 and the sound information output from the sound output unit 150 according to the first embodiment of the present disclosure.

FIG. 13 is a diagram showing a transformation example of information exchanged between the information processing apparatus 100 and the information processing apparatus 200 according to the first embodiment of the present disclosure.

FIG. 14 is a flowchart showing an example of a processing procedure of reception processing by the information processing apparatus 100 according to the first embodiment of the present disclosure.

FIG. 15 is a flowchart showing an example of a processing procedure of transmission processing by the information processing device 200 according to the first embodiment of the present disclosure.

FIG. 16 is a diagram showing an example of information exchanged between information processing apparatuses according to the second embodiment of the present disclosure.

FIG. 17 is a diagram showing information processing according to the second embodiment of the present disclosure An example of a pattern of information exchanged between devices.

FIG. 18 is a diagram showing an example of information exchanged between information processing apparatuses according to the second embodiment of the present disclosure.

FIG. 19 is a diagram showing a transformation example of a screen displayed from the display 140 in the information processing apparatus 100 and the sound information output from the sound output unit 150 according to the second embodiment of the present disclosure.

FIG. 20 is a diagram showing a transformation example of information exchanged between the information processing apparatus 100 and the information processing apparatus 200 according to the second embodiment of the present disclosure.

21 is a flowchart showing an example of a processing procedure of reception processing by the information processing apparatus 100 according to the second embodiment of the present disclosure.

FIG. 22 is a flowchart showing an example of a processing procedure of transmission processing by the information processing device 200 according to the second embodiment of the present disclosure.

FIG. 23 is a sequence diagram showing a communication processing example of the source device 50 and the sink device 60 according to the second embodiment of the present disclosure.

FIG. 24 is a diagram showing an example of information exchanged between information processing apparatuses according to the second embodiment of the present disclosure.

FIG. 25 is a diagram showing an example of information exchanged between information processing apparatuses according to the second embodiment of the present disclosure.

FIG. 26 is a diagram showing an example of information exchanged between information processing apparatuses according to the second embodiment of the present disclosure.

FIG. 27 is a diagram showing a transformation example of a screen displayed from the display 140 in the information processing apparatus 100 and the sound information output from the sound output unit 150 according to the third embodiment of the present disclosure.

FIG. 28 is a diagram showing a transformation example of information exchanged between the information processing apparatus 100 and the information processing apparatus 200 according to the third embodiment of the present disclosure.

FIG. 29 is a flowchart showing an example of a processing procedure of reception processing by the information processing apparatus 100 according to the third embodiment of the present disclosure.

FIG. 30 is a flowchart showing an example of a processing procedure of reception processing by the information processing apparatus 100 according to the third embodiment of the present disclosure.

Fig. 31 is a block diagram showing an example of the outline configuration of a smartphone.

Fig. 32 is a block diagram showing an example of a car navigation device.

Fig. 33 is a block diagram showing an example of the outline configuration of a wireless access point.

Hereinafter, preferred embodiments of the present disclosure (hereinafter referred to as embodiments) will be described. The embodiments are described in the following order.

1. The first embodiment (in the example where the reduction of audio data is requested during streaming)

2. Second embodiment (in the example where the audio data is reduced at the beginning of the stream, in the example where the use of the second audio data reduction method is requested)

3. The third embodiment (an example in which the receiving device detects the movement or presence of a person and requests the reduction of audio data based on the detection result)

4. Application examples

<1. First embodiment> Communication system configuration example

FIG. 1 is a diagram showing a configuration example of the communication system 10 according to the first embodiment of the present disclosure.

The communication system 10 includes an information processing device 100, an information processing device 200, and an information processing device 210.

The information processing device 100, the information processing device 200, and the information processing device 210 have a wireless communication function, and are information processing devices that can transmit and receive various types of information by connecting to each other using wireless communication. Furthermore, these information processing devices are information processing devices that support IEEE 802.11 specifications that enable peer-to-peer (P2P) connections. That is, these information processing devices form a communication group and can directly communicate with each other without any access point intervention. In this case, the operation of the information processing equipment as the group owner and the information processing equipment as the customer may or may not be determined during manufacturing. When it is not decided during the manufacturing period, the operation of the information processing equipment operating as the group owner and the information processing equipment operating as the customer can be determined through negotiation between a plurality of information processing equipment. For example, when the information processing device 100 and the information processing device 200 form a communication group, the information processing device 100 can directly transmit data (eg, moving image content) to the information processing device 200. In this case, the information processing devices 100 and 200 use wireless communication to connect to each other, and the moving image content stored in the information processing device 100 can be displayed on the information processing device 200. It should be noted that Wi-Fi Direct (Wi-Fi Direct) is an example of a communication standard known as direct communication between information processing equipment.

The information processing device 100 is, for example, an image viewing device that records or displays moving image content (for example, a TV receiver with a built-in hard disk). For example, the information processing device 100 may receive the broadcast wave from the broadcast station 11 and display the image 21 on the display 140 based on the broadcast wave. Furthermore, for example, the information processing device 100 may display the image 22 on the display 140 based on the image content transmitted from the information processing device 200. For example, the information processing device 100 may also display the image 23 on the display 140 based on the image content transmitted from the information processing device 210.

For example, the information processing device 200 is a portable information processing device (such as a smart phone with a call function and a data communication function). The information processing device 210 is an information processing device that performs various types of information processing (for example, a tablet computer terminal, a laptop type personal computer (PC)).

It should be noted that the information processing device 100, the information processing device 200, and the information processing device 210 can transmit and receive various types of information by connecting to an access point (not shown) using wireless communication. Here, for example, the access point is an access point that supports IEEE 802.11a, 11b, 11g, and 11n wireless LAN standards. That is, the router and the access point (for example, the access point is a product embedded in the router) are standardized by the wireless LAN by IEEE 802.11a, 11b, 11g, and 11n.

It should be noted that the information processing device shown in Figure 1 It is prepared as an example, and this embodiment can also be applied to another information processing device. For example, this embodiment can be applied to image devices with wireless communication functions (such as digital still cameras, digital video cameras (such as camera integrated type recorders)) and sound output devices with wireless communication functions (such as (Portable music player with display panel). For example, the embodiment can also be applied to display devices with wireless communication function (such as digital photo frames) and electronic book display devices with wireless communication function. Furthermore, for example, this embodiment can be applied to another information processing device with wireless communication function. For example, information processing equipment with wireless communication functions includes domestically used image processing devices (digital multi-functional disc (DVD) recorders, video recorders, etc.), personal digital assistants (PDAs), home game consoles, home appliances, and Portable image processing devices, portable game machines, and the like. For example, this embodiment can also be applied to an information processing device (for example, a personal computer without a wireless communication function) that can perform wireless communication by providing a wireless communication device with a wireless communication function.

Information processing equipment configuration example

FIG. 2 is a block diagram showing a functional configuration example of the information processing apparatus 100 according to an embodiment of the present disclosure. It should be noted that the functional configuration of the information processing device 200 and the information processing device 210 is the same as the functional configuration of the information processing device 100, and therefore, the description thereof is omitted here.

The information processing device 100 includes a communication unit 110, an antenna 114, a sensor 120, an operation receiving unit 130, a display 140, a sound The output unit 150, the memory 160, and the controller 170. The communication unit 110 includes a data processing unit 111, a transmission processing unit 112, and a wireless interface unit 113.

The material processing unit 111 processes various types of materials based on the control of the controller 170. For example, in the transmission operation, the data processing unit 111 forms various data frames and data packets according to the request from the upper layer and provides them to the transmission processing unit 112. For example, in the receiving operation, the data processing unit 111 processes and analyzes various data frames and data packets provided from the transmission processing unit 112.

The transmission processing unit 112 performs various types of transmission processing based on the control of the controller 170. For example, in the transmission operation, the transmission processing unit 112 executes for various data headers, frame check sequences (FCS), and the like to add an error detection code to the packet generated by the data processing unit 111 Of processing. Then, the transmission processing unit 112 supplies the processed data to the wireless interface unit 113. For example, in the receiving operation, the transmission processing unit 112 analyzes various data frames provided from the wireless interface unit 113. Then, once the transmission processing unit 112 confirms that the data frame does not have any errors based on the error detection code, it supplies various information frames to the data processing unit 111.

The wireless interface unit 113 is an interface for transmitting and receiving various types of information by connecting to another information processing device. For example, in the transmission operation, the wireless interface unit 113 generates a modulation signal in a frequency band of the carrier based on the data received from the transmission processing unit 112 Signal, and the modulated signal transmitted through the antenna 114 is like a wireless signal. Furthermore, in the receiving operation, the wireless interface unit 113 will down-convert a wireless signal received through the antenna 114 into a bit string, thereby decoding various data frames.

Furthermore, for example, the communication unit 110 uses a moving picture expert group (MPEG) 2 transmission stream (TS) to receive multiplexed video signals and audio signals from another information processing device. The communication unit 110 uses MPEG2-TS to wirelessly transmit the multiplexed image signal and audio signal to another information processing device.

The sensor 120 is a sensor that detects the presence of a person, the movement or gesture of a person (for example, the action of a person waving a hand, or the action of a person performing some operation) and the like, and provides the detected information to the controller 170. Here, for example, as for the sensor 120, an image sensor that images an object, generates image data, and performs various types of image recognition on the image data can be used. For example, regarding the sensor 120, an infrared sensor that uses infrared rays to obtain various types of information can be used. Furthermore, as for the sensor 120, a human body sensor that uses infrared rays, ultrasonic waves, visible light, and the like to detect the presence of a person (for example, a person approaching the information processing apparatus 100) may be used. One of these sensors may be used as the sensor 120, or a plurality of them may be combined and used as the sensor 120. Another sensor can also be used.

The operation receiving unit 130 is an operation receiving unit that receives user's operation input and outputs operation information to the controller 170 according to the received operation input. Regarding the operation receiving unit 130, for example, it is possible to make Use a mouse, keyboard, touchpad, buttons, microphone, switcher, or joystick. The operation receiving unit 130 also receives operations for transmitting and receiving various types of information from another information processing device.

The display 140 is a display that displays various types of information (image information, character information, time information, and the like) based on the control of the controller 170. For example, the display 140 displays an image transmitted from another information processing device (such as the image shown in FIG. 1). It should be noted that, for example, regarding the display 140, a display panel such as an organic electroluminescence (EL) panel and a liquid crystal display (LCD) panel may be used. It should be noted that the operation receiving unit 130 and the display 140 can be integrated using a touchpad that enables a user to perform operation input by placing a finger above or close to the display surface.

The sound output unit 150 is a sound output unit (such as a speaker) that outputs various types of sound information based on the control of the controller 170.

The memory 160 functions as a work area for data processing by the controller 170 and as a storage medium for storing various types of data. The memory 160 stores various types of information (as a connection partner) included in the data to be transmitted to the information processing device. As for the memory 160, for example, recording media such as non-volatile memory, magnetic disks, optical disks, and magneto-optical (MO) disks may be used. It should be noted that, for example, non-volatile memory, electrically erasable programmable read-only memory (EEPROM) or erasable programmable ROM (EPROM) can be used. For example, regarding a magnetic disk, a hard disk or a disk-shaped disk may be used. For example, regarding the optical disc, a compact disc (CD), a recordable digital versatile disc (DVD-R), and a Blu-ray disc (BD) (registered trademark) can be used.

The controller 170 controls the units of the information processing apparatus 100 based on the control program in the memory 160. For example, the controller 170 controls the reception operation and the transmission operation of each of the data processing unit 111, the transmission processing unit 112, and the wireless interface unit 113. For example, the controller 170 performs operations to determine the frequency of use, command the formation or transmission of control messages, and interpret control messages. It should be noted that the control message is notification information, such as a beacon, an indicator receiving response, an exploration request, and an exploration response. As described above, the information processing apparatus 100 can receive broadcast waves and display images on the display 140 based on the broadcast waves. Therefore, the controller 170 can obtain a broadcast wave through a broadcast receiving unit (not shown) and display an image on the display 140 based on the broadcast wave.

Wi-Fi CERTIFIED Miracast release-1

The Wi-Fi CERTIFIED Miracast release-1 developed by the Wi-Fi Alliance will be described here.

FIGS. 3 to 5 are sequence diagrams showing examples of communication processing of the source device 50 and the sink device 60 according to an embodiment of the present disclosure.

Figures 6 and 7 show an RTSP message exchanged between the source device 50 and the sink device 60 according to an embodiment of the present disclosure An example of a diagram. In Figures 6 and 7, the name of the RTSP message is displayed in the Message ID section. Furthermore, whether the RTSP message is a request or a response is displayed in the req/res (direction) part. In addition, the RTSP message from the source device to the sink device is displayed as (src->snk), and the RTSP message from the sink device to the source device is displayed as (snk->src).

Figures 3 and 4 show the flow from the discovery of a device to the beginning of AV streaming via Wi-Fi CERTIFIED Miracast. Figure 5 shows the flow from the determination of the AV format to the start of AV streaming by RTSP in the processing shown in Figures 3 and 4.

Here, in Wi-Fi Direct, multiple devices detect each other's presence (device discovery, service discovery). Then, after devices connected to each other are selected, device authentication is performed between the selected devices using Wi-Fi protected setup (Wi-Fi protected setup; WPS), so that a direct connection system is established therebetween. Furthermore, in Wi-Fi Direct, multiple devices determine whether they function as group owners or customers and form a communication group.

In the communication processing examples shown in FIGS. 3 and 4, part of the packet transmission and reception is omitted. For example, for using WPS, packet exchange at the initial connection is required, as described above, and packet exchange is also required for transmission and reception of authentication requests/responses and the like. However, Figures 3 and 4 omit the display of this packet exchange, and only show the second or subsequent connections.

It should be noted that Figures 3 and 4 show the source device The communication processing example between 50 and the receiving device 60 can also be applied to the communication processing between other devices.

First, device discovery is performed between the source device 50 and the sink device 60 (71). For example, the source device 50 transmits an exploration request (response request signal) and receives from the receiving device 60 an exploration response (response signal) corresponding to the exploration request. In this way, the source device 50 and the sink device 60 can discover each other's existence. Furthermore, device discovery allows the acquisition of device names, types (TV, PC, smartphones, and the like), partner's source support devices/reception support devices, and the like.

Next, service discovery is performed between the source device 50 and the sink device 60 (72). For example, the source device 50 transmits a service discovery query to inquire about the services supported by the sink device 60 discovered during the discovery of the host device. Next, the source device 50 receives the service discovery response from the sink device 60, thereby obtaining the service supported by the sink device 60. That is, the source device 50 can obtain the services performed by the partner, the detailed available capabilities of the partner, and the like through service discovery. For example, services performed by partners include services and agreements (digital living network alliance (DLNA), digital media renderer (DMR), etc.).

Next, the user performs an operation of selecting a partner to connect (connection partner selection operation) (73). The connection partner selection operation may only occur on one of the source device 50 and the sink device 60. For example, the connection partner selection screen is displayed on the display of the source device 50, and the receiving device 60 is operated by the user on the connection partner selection screen. Select as connection partner.

After the user performs the connection partner selection operation (73), the group owner negotiation is performed between the source device 50 and the sink device 60 (74). Figures 3 and 4 show an example of the result of the group owner negotiation that the source device 50 becomes the group owner 75 and the receiving device 60 becomes the customer 76.

Next, various types of processing (77 to 81) are executed between the source device 50 and the receiving device 60, so that a direct connection system is established. That is, the combination (Layer 2 (L2) link establishment) (77), secure link establishment (78), and IP address designation (79) are sequentially performed. Furthermore, TCP connection establishment (80) and streaming control by RTSP (81) are executed sequentially.

Next, the AV data is transmitted from the source device 50 to the sink device 60 in MPEG2-TS (82).

In this way, in Wi-Fi CERTIFIED Miracast release-1, real-time AV data streaming from the source device 50 to the sink device 60 is possible. However, according to this standard, whether audio data is transmitted from the receiving device 60 to the source device 50 cannot be specified.

Furthermore, for example, in the case of a receiving device having a relatively large display (information processing apparatus 100), as shown in FIG. 1, a plurality of screens may be displayed on the display. In this case, it is considered that the sound of one of the multiple screens is output. For example, it is possible to arrange that the sound designated by the user as the content of the home screen is output, but the sound not designated as the content of the home screen is not Is output.

Also in this case, the image data and audio data (sub-screen) not designated as the main screen are also transmitted using Wi-Fi CERTIFIED Miracast, which indicates that the audio data that is not output is transmitted. In this case, the audio data that is not output is transmitted or pressed out of a band of a wireless section.

Furthermore, the transmission of audio data that is not output increases the power consumption of the source device 50 beyond what is needed. In addition, the reception of audio data that is not output will also increase the power consumption of the receiving device 60 beyond what is needed.

Next, the embodiment of the present disclosure describes an example in which the audio data exchanged between the source device 50 and the sink device 60 is properly processed. It should be noted that in the embodiment of the present disclosure, the screen that outputs sound is referred to as the main screen, and the screen that does not output sound is referred to as the sub-screen. Furthermore, the main screen and the sub-screen can be defined based on the display size or display position.

RTSP message configuration example

FIG. 8 is a diagram showing an example of information exchanged between information processing apparatuses according to the first embodiment of the present disclosure. Figure 8 shows an example of the definition of the grammar with the added backus-naur form (ABNF) when the new parameter (WFD-audio-stream-control) is defined in RTSP.

FIG. 9 is a diagram showing an example of information exchanged between information processing apparatuses according to the first embodiment of the present disclosure. Figure 9 shows an example configuration of RTSP M3 request and RTSP M3 response. In FIG. 9, the information system surrounded by dotted rectangles 401 to 404 is different from the example shown in FIG. 6.

10 and 11 are diagrams showing examples of information exchanged between information processing apparatuses according to the first embodiment of the present disclosure. Figures 10 and 11 show examples of newly defined RTSP M110 requests and RTSP M110 responses.

Here, the specific functions represent the functions played in the embodiments of the present disclosure. For example, the first embodiment of the present disclosure describes an example in which a specific function is to reduce the amount of audio data in a stream to be transmitted to a receiving device and the function of transmitting the stream based on a request from the receiving device.

For example, when the source device (information processing device 200) is adaptable to the specific function, the source device (information processing device 200) transmits a dotted line containing "wfd_audio_stream_control" (shown in FIG. 9 Rectangle 402) RTSP M3 request message. In this way, the source device (information processing device 200) indicates that the source device is adaptable to the specific function of the receiving device (information processing device 100), and asks whether the receiving device (information processing device 100) is Specific functions are adaptable.

Here, the receiving device that is unsuitable for the specific function sends a "none" response in the RTSP M3 response message. Therefore, after receiving this response, the source device (information 200) Do not use this specific function.

Furthermore, an adaptable bit is set to 1 for the receiving device (information processing device 100) that is adaptable for the specific function, and it is transmitted to the source device (information processing device 200) in the RTSP M3 response message.

Here, in the dotted rectangle 404 shown in FIG. 9, the value (C) of “wfd_audio_stream_control” is expressed in hexadecimal numbers. This value is 12 in decimal numbers and 1100 in 4-bit binary numbers. It indicates that the value (C) of "wfd_audio_stream_control" corresponds to "(re-)include audio" at bit position B3 and "reduce audio" at bit position B2 shown in Fig. 8.

Furthermore, for example, when the reduction of audio data is triggered in the receiving device (information processing device 100), the receiving device (information processing device 100) sends a message indicating this fact to the source data. For example, the receiving device (information processing apparatus 100) transmits the RTSP M110 request message (specified by the information surrounded by the dotted rectangle 405) shown in FIG.

In this way, the receiving device may request the source device to reduce audio data based on the first trigger. Here, for example, the first trigger can be caused by a user operation (such as a switching operation between the main screen and the sub-screen).

Furthermore, the source device that has received the RTSP M110 request message shown in Figure 10 returns the RTSP M110 shown in Figure 10 Send "OK" to the message. Next, the source device reduces the amount of audio data in a stream to be transmitted to the receiving device and transmits the stream to the receiving device according to a command from the receiving device.

The method of reducing audio data will be described here.

For example, the source device can perform the first audio data reduction so that the amount of PES payload data is stored in the TS packet specified by the PID without changing the PID specified in the PMT to specify the transmission of the audio data The number of PES packets is reduced, and PES packets containing PTS in the PES header are generated almost periodically. The source device multiplexes and transmits PES packets for audio signal transmission (in which the amount of data is reduced) and PES packets for video signal transmission. In this case, the source device can reduce the amount of data by making the amount of PES payload data substantially zero or completely zero.

In this case, the data may not contain any PES payload, or the data may contain stuff bytes instead of actual audio data.

Furthermore, it is preferable to set the PES packet generation cycle to about 100 msec. Therefore, even a receiving device that reproduces video in synchronization with the audio clock can regenerate the audio clock and regenerate the video with proper clock synchronization while avoiding free running of the clock.

Furthermore, for example, when the receiving device performs demultiplexing on the MPEG2-TS multiplexing signal, the receiving device discards the data contained in the PES payload and extracts the sound stored in the description specified in the PMT PTS in the PES header portion of the PES packet of the TS packet specified by the PID of the data transfer.

In this way, the first embodiment of the present disclosure describes an example of a first audio data reduction method (as an audio data reduction method) used in which PTS is retained.

Furthermore, for example, when re-inclusion of audio data is triggered in the receiving device (information processing device 100), the receiving device (information processing device 100) sends a message indicating this fact to The source device. For example, the receiving device (information processing apparatus 100) transmits the RTSP M110 request message (specified by the information surrounded by the dotted rectangle 406) shown in FIG. 11.

Furthermore, the source device that has received the RTSP M110 request message shown in Figure 11 sends "OK" in the RTSP M110 response message shown in Figure 11. Next, the source device restores the original amount of audio data in a stream to be transmitted to the receiving device and transmits the stream to the receiving device according to a command from the receiving device.

In this way, the controller of the receiving device can request the source device to restore the original volume of audio data based on the second trigger while receiving the MPEG2-TS multiplex signal in which the volume of audio data is reduced. Here, for example, the second trigger can be caused by a user operation (for example, a switching operation between the main screen and the sub-screen).

Furthermore, when the PES packet used for the transmission of the audio signal in which the amount of data is reduced by the reduction of the first audio data and the PEG packet used for the transmission of the video signal are transmitted through the receiving device after the multiplexing process When it is requested to restore an original volume of sound data, the controller of the source device performs control to store general sound data in the PES payload, and more Process and transmit PES packets for audio signal transmission and PES packets for video signal transmission.

The controller of the receiving device performs control to extract the audio data contained in the PES payload when performing demux on the MPEG2-TS multiplexing signal received from the source device after the request.

Home screen display switching example

The following will describe an example in which the receiving device receives a broadcast wave and redisplays the screen based on the broadcast wave when the screen of the source device is displayed on the receiving device.

FIG. 12 is a diagram showing a transformation example of a screen displayed from the display 140 in the information processing apparatus 100 and the sound information output from the sound output unit 150 according to the first embodiment of the present disclosure.

FIG. 13 is a diagram showing a transformation example of information exchanged between the information processing apparatus 100 and the information processing apparatus 200 according to the first embodiment of the present disclosure. Figure 13 shows an example of the exchange of information corresponding to Figure 12.

FIG. 12A shows an example of a situation where the image from the information processing device 200 and the sound of the image are output from the information processing device 100. That is, FIG. 12A shows an example in which streaming is performed from the information processing apparatus 200 to the information processing apparatus 100 using Wi-Fi CERTIFIED Miracast using settings including audio (311, 312 shown in FIG. 13). In this case, the sound 301 of the image from the information processing device 200 is output from the sound output unit 150.

FIG. 12B is shown in the state shown in FIG. 12A, through which the user operates the user to perform settings to receive broadcast waves and display an image on the display 140 based on the broadcast waves (example shown in FIG. 13) 313). Furthermore, it is assumed that through the user's operation, the user also performs settings for displaying two screens of the image 306 and the image 305 from the information processing device 200 on the display 140 based on the broadcast wave. In addition, through this user operation, it is assumed that the user sets the image 305 as the home screen based on the broadcast wave.

In this case, the information processing device 100 transmits an RTSP M110 request message (shown in FIG. 10) to the information processing device 200 and requests a reduction in audio data (314, 315 shown in FIG. 13). Furthermore, the information processing device 200 transmits an "OK" request message (shown in Figure 10) to the RTSP M110 to the information processing device 100 (316, 317 shown in Figure 13).

Next, the information processing device 200 transmits a stream in which the audio data is reduced to the information processing device 100 (318, 319 shown in FIG. 13). In this case, the sound of the image 306 from the information processing device 200 is not output, as shown in FIG. 12B. That is, when the image 305 based on the broadcast wave is set as the home screen, the sound of the image 306 from the information processing device 200 is not output. Next, the audio 302 of the video 305 based on the broadcast wave is output from the audio output unit 150.

Fig. 12C is shown in the state shown in Fig. 12B, the video 307 based on the broadcast wave is set as a sub-screen and comes from The image 308 of the information processing device 200 is an example of a case where the home screen is set by user operation (320 shown in FIG. 13).

In this case, the information processing device 100 transmits an RTSP M110 request message (shown in FIG. 11) to the information processing device 200 and requests an increase in audio data (321, 322 shown in FIG. 13). Furthermore, the information processing device 200 sends an "OK" request message (shown in Figure 11) to the RTSP M110 to the information processing device 100 (323, 324 shown in Figure 13).

Next, the information processing device 200 transmits a stream containing audio data to the information processing device 100 (325, 326 shown in FIG. 13). In this case, the sound of the video 307 based on the broadcast wave is not output, as shown in Figure 12C. Furthermore, the sound 303 of the image 308 from the information processing device 200 is output from the sound output unit 150.

Here, as shown in FIG. 12A, when an image from the information processing device 200 is displayed as the main image on the display 140, the sound 301 of the image is output from the sound output unit 150. In this case, it is assumed that the user viewing the image adjusts the sound volume setting value (amount) of the sound 301 output from the sound output unit 150 according to the sound pressure of the sound 301. Next, when the sound volume setting value of the sound 301 output from the sound output unit 150 is adjusted, the controller 170 of the information processing device 100 stores the adjusted value in the memory 160.

Furthermore, as shown in Figure 12B, when based on the broadcast When the image 305 of the wave is displayed as the main image on the display 140, the sound 302 of the image 305 is output from the sound output unit 150. Similarly, in this case, it is assumed that when the broadcast wave is being received, the user viewing the image 305 adjusts the sound 302 output from the sound output unit 150 according to the sound pressure of the sound 302 Sound volume setting value (volume).

As shown in FIG. 12C, when the image 308 from the information processing device 200 is set as the main image, the controller 170 of the information processing device 100 sets a sound volume corresponding to a value stored in the memory 160. That is, when the image 308 from the information processing device 200 is set as the main image, the controller 170 of the information processing device 100 discards the sound volume set when the sound 302 of the image 305 based on the broadcast wave is output Set the value (amount) and reset the sound volume used when the image from the information processing device 200 is set as the main image.

In this way, the controller 170 of the information processing apparatus 100 stores the sound volume setting value of the sound output before the request is made to the source device in the memory 160. Then, the controller 170 of the information processing apparatus 100 requests the source device to restore the original volume of sound data and set the time point when the sound data included in the PES payload is extracted after the request is stored in the memory 160 Sound volume setting value.

In this way, for example, it is possible to prevent the output of unintended sound volume when the home screen is switched.

Operation example of receiving device

FIG. 14 is a flowchart showing an example of a processing procedure of reception processing by the information processing apparatus 100 according to the first embodiment of the present disclosure. Figure 14 shows an example of a situation where both the source device and the sink device are adapted to specific functions.

First, the controller 170 of the information processing device 100 executes the process of starting Wi-Fi CERTIFIED Miracast (step S801).

Next, the controller 170 starts a session of Wi-Fi CERTIFIED Miracast (step S802). In this case, the exchanged streams contain audio data.

Then, the controller 170 performs processing of receiving, decoding, and regenerating the stream containing the audio data, for example (step S803). In this way, the image based on the image data contained in the received stream is displayed on the display 140, and the sound based on the audio data contained in the received stream is output from the sound output unit 150.

Next, the controller 170 determines whether an image that does not require audio data exists in the image displayed on the display 140 (based on the received stream image) (step S804). For example, as shown in Fig. 12A and Fig. 12B, when the user is used to receive the broadcast wave and display it as the main screen, it is determined that the audio data from the source device is unnecessary (step S804) .

When an image that does not require audio data does not exist (step S804), the operation returns to step S803. When an image that does not require audio data exists (step S804), the controller 170 will be used to request audio information One of the messages of reduced data is sent to the source device that sends the stream corresponding to the image that does not require audio data (step S805). For example, the controller 170 transmits the RTSP M110 request message shown in FIG.

After the message is transmitted, the communication unit 110 receives a response to the message from the source device (step S806). For example, the source device sends "OK" in the RTSP M110 response message shown in Figure 10.

After the response to the message for requesting the reduction of the audio data is received (step S806), the controller 170 performs processing of receiving, decoding, and regenerating a stream in which the audio data is reduced (step S807) . In this case, the image based on the image data contained in the received stream is displayed on the display 140. However, the received stream does not contain audio data, and therefore no sound is output.

Next, the controller 170 determines whether an image requiring audio data exists in the image displayed on the display 140 (based on the received stream image) (step S808). For example, as shown in FIGS. 12B and 12C, when a user operation for displaying an image from the source device as the main screen is performed, it is determined that the audio data from the source device is required (Step S808).

When the image requiring audio data (the image based on the received stream) does not exist (step S808), the operation returns to step S807. When an image requiring audio data exists (step S808), the controller 170 transmits a message for requesting an increase in audio data to the transmission The source device corresponding to the stream of the image requiring audio data (step S809). For example, the controller 170 transmits the RTSP M110 request message shown in FIG. 11.

After the message is transmitted, the communication unit 110 receives a response to the message from the source device (step S810). For example, the source device sends "OK" in the RTSP M110 response message shown in Figure 11.

After the response to the message for requesting the addition of audio data is received (step S810), the controller 170 determines whether the completion of the reception process has been commanded (step S811). For example, when the completion of the reception process has not been commanded (step S811), the operation returns to step S803, and the controller 170 performs the process of receiving, decoding, and regenerating the stream containing the audio data (step S803). In contrast, when the completion of the reception process has been commanded (step S811), the operation of the reception process is completed. It should be noted that steps S803 and S807 are examples of communications described in the scope of patent application. It should be noted that steps S805 to S807 are examples of the control described in the patent application scope.

Source device operation example

FIG. 15 is a flowchart showing an example of a processing procedure of transmission processing by the information processing device 200 according to the first embodiment of the present disclosure. Figure 15 shows an example of a situation where both the source device and the sink device are adapted to specific functions.

First, the controller of the information processing device 200 (corresponding to The controller 170 in FIG. 2) executes the process of starting Wi-Fi CERTIFIED Miracast (step S821).

Next, the controller of the information processing device 200 starts a session of Wi-Fi CERTIFIED Miracast (step S822). In this case, the exchanged streams contain audio data.

Next, the controller of the information processing apparatus 200 generates a stream containing audio data and transmits the generated stream to the receiving device (step S823). In this way, the image based on the image data contained in the transmitted stream is displayed on the display of the receiving device, and the sound based on the audio data contained in the received stream is output from the sound output unit of the receiving device .

Next, the controller of the information processing apparatus 200 determines whether a message for requesting reduction of audio data has been received from the receiving device (step S824). For example, the controller of the information processing device 200 determines whether the RTSP M110 request message shown in FIG. 10 has been received. When the message is not received from the receiving device (step S824), the operation returns to step S823.

When the message has been received from the receiving device (step S824), the controller of the information processing apparatus 200 transmits one of the messages to the receiving device (step S825). For example, the information processing device 200 transmits "OK" in the RTSP M110 response message shown in Figure 10.

Then, the controller of the information processing device 200 generates a stream in which the audio data is reduced and transmits the generated stream to the interface Receiver (step S826). In this case, the image based on the image data contained in the received stream is displayed on the display of the receiving device. However, no sound is output.

Next, the controller of the information processing apparatus 200 determines whether the message for requesting the increase of audio data has been received from the receiving device (step S827). For example, the controller of the information processing device 200 determines whether the RTSP M110 request message shown in FIG. 11 has been received. When the message is not received from the receiving device (step S827), the operation returns to step S826.

When the message has been received from the receiving device (step S827), the controller of the information processing apparatus 200 transmits one of the messages to the receiving device (step S828). For example, the information processing device 200 transmits "OK" in the RTSP M110 response message shown in Figure 11.

Next, the controller of the information processing apparatus 200 determines whether the completion of the transfer process has been commanded (step S829). When the completion of the transfer process has not been commanded (step S829), the operation returns to step S823. In contrast, when the completion of the transfer process has been commanded (step S829), the operation of the transfer process is completed. It should be noted that steps S823 and S826 are examples of communications described in the scope of patent application. Furthermore, step S826 is an example of the control described in the scope of patent application.

<2. Second embodiment>

The second embodiment of the present disclosure illustrates the use of Wi-Fi CERTIFIED Miracast is an example where audio data is reduced at the beginning of streaming. Furthermore, the second embodiment of the present disclosure describes an example in which the use of a sound data reduction method (second sound data reduction method) different from that requested by the sound data reduction method described in the first embodiment of the disclosure . That is, in the second embodiment of the present disclosure, the method of reducing audio data can be specifically described. Therefore, in the second embodiment of the present disclosure, further extensions are introduced into the structure of the newly provided RTSP parameters.

It should be noted that the configuration of the information processing device in the second embodiment of the present disclosure is substantially the same as the configuration of the information processing devices 100 and 200 shown in FIG. 1 and the like. Therefore, elements common to the first embodiment of the present disclosure are denoted by element symbols of the first embodiment of the present disclosure, and the description of these elements is partially omitted.

RTSP message configuration example

FIG. 16 is a diagram showing an example of information exchanged between information processing apparatuses according to the second embodiment of the present disclosure. Figure 16 shows an example of a situation in which further extensions are introduced to the structure of the newly provided RTSP parameters.

FIG. 17 is a diagram showing an example of information exchanged between information processing apparatuses according to the second embodiment of the present disclosure. Figure 17 shows an example configuration of RTSP M3 request and RTSP M3 response. In FIG. 17, the information system surrounded by dotted rectangles 411 to 414 is different from the example shown in FIG. 6.

FIG. 18 is a diagram showing the capital according to the second embodiment of the present disclosure A schematic diagram of an example of information exchanged between information processing devices. Figure 18 shows an example configuration of the newly defined RTSP M110 request and RTSP M110 response.

For example, when the source device (information processing device 200) is adaptable to the specific function, the source device (information processing device 200) transmits a dotted line including "wfd_audio_stream_control" (shown in FIG. 17 Rectangular 412) RTSP M3 request message. In this way, the source device (information processing device 200) indicates that the source device is adaptable to the specific function of the receiving device (information processing device 100), and asks whether the receiving device (information processing device 100) is Specific functions are adaptable.

Here, the receiving device that is unsuitable for the specific function sends a "none" response in the RTSP M3 response message. Therefore, after receiving this response, the source device (information processing device 200) does not use the specific function.

Furthermore, an adaptable bit is set to 1 for the receiving device (information processing device 100) that is adaptable for the specific function, and it is transmitted to the source device (information processing device 200) in the RTSP M3 response message.

Here, in the dotted rectangle 414 shown in FIG. 17, the value (F) of "wfd_audio_stream_control" is expressed in hexadecimal numbers. This value is 15 in decimal numbers and 1111 in 4-bit binary numbers. It means: the value (F) of "wfd_audio_stream_control" corresponds to any bit position B3 to B1 (shown on page 16 (Figure) And the reduction of audio data is requested from the beginning of the stream. That is, it means that the value (F) of "wfd_audio_stream_control" corresponds to the "(re-)include audio" at bit position B3 shown in Figure 16 and the "reduce audio" at bit position B2 ". Furthermore, it means: the value (F) of "wfd_audio_stream_control" corresponds to the reduction of audio data using a reduction method different from "reduce audio" at bit position B, and the reduction of audio data is from the beginning of streaming Be requested.

That is, Figure 17 shows an example of an RTSP M3 response message requesting the reduction of audio data from the beginning of the stream. The source device that has received the RTSP M3 response message uses any data reduction method (where the receiving device is adaptable) to reduce the data, and then sends the stream to the receiving device. For example, the reduction method that is the same as the first embodiment of the present disclosure (first audio data reduction method) can be used.

Furthermore, for example, when the re-containment of audio data is triggered, the receiving device transmits a newly provided RTSP M110 request message (shown in FIG. 11). The source device that has received the RTSP M110 request message sends "OK" in the RTSP M110 response message (shown in Figure 11). Then, the source device restores the original volume of audio data and transmits it according to the command from the receiving device. It should be noted that the configuration of the RTSP M110 request message and the RTSP M110 response message are the same in Figure 11, and therefore their detailed descriptions are omitted.

Here, it is assumed that the user operates the receiving device to set an image as the main screen based on the broadcast wave and sets one from the source device The image serves as a sub-screen. In this case, the receiving device may decide that the audio data of the image from the source device has become unnecessary. Therefore, using this operation as a trigger, the receiving device requests the source device to reduce the audio data. In this case, it is assumed that the receiving device is designed and implemented without audio PTS to smoothly display an image from the source device. Therefore, the receiving device may specify another reduction method (for example, a reduction method in which no audio PES packet is transmitted to further reduce the amount of data).

Figure 18 shows an example configuration of the RTSP M110 request and RTSP M110 response sent in this situation.

The source device that has received the RTSP M110 request message shown in Figure 18 sends "OK" in the RTSP M110 response message. Then, the source device reduces the data about the stream transmitted after this "OK" transmission by the second audio data reduction method, and transmits the audio data without multiplexing the PES packets for the audio data Data other than PES packets. Here, the second audio data reduction method is a reduction method in which PES packets that do not carry audio data are generated in a stream without including information specifying the PID of the audio data.

In this way, the source device performs the second sound data reduction in which PES packets that do not carry audio data are generated without including the information specifying the PID of the audio data, and is used for audio signals without multiplexing In the case of transmitted PES packets, PES packets used for image signal transmission can be transmitted.

The receiving device receives the audio data through the second sound The audio data reduction method is reduced by one stream and decoding the video data contained in the received stream.

It is assumed that in this state, the user cancels both screens and only indicates that the display is from the source device's image. In this case, the receiving device may decide that the audio data of the image from the source device has become necessary. Therefore, using this operation as a trigger, the receiving device requests the source device to restore the original audio data in the RTSP M110 request message.

The configuration example of the RTSP M110 request and RTSP M110 response sent in this situation is the same as in Figure 11.

The source device sends "OK" to the RTSP M110 response (shown in Figure 11) and sends a stream including audio data. That is, the source device includes information specifying the PID of the audio data in the PMT in the stream and using the TS packet specified by the PID to generate the PES packet carrying the audio data. Then, the source device multiplexes the generated PES packet carrying audio data and the PES packet carrying video data, and transmits the result to the receiving device.

In this way, when the second audio data is reduced, the receiving device is requested to restore an original sound when the PES packet for image information transmission is transmitted without multiplexing the PES packet for audio signal transmission The amount of data, the source device includes the PID that specifies the transmission of audio data in the PMT, the storage of general audio data in the PES payload, and the multiplex processing and transmission of PES packets for audio signal transmission and PES packets for video signal transmission.

For example, the receiving device receives a string containing audio data Stream, perform demultiplexing on the video data and audio data contained in the received stream, and decode and reproduce the data.

Home screen display switching example

The following will describe an example of a case in which when a screen based on broadcast waves is displayed on a receiving device, the receiving device newly displays an image from the source device.

FIG. 19 is a diagram showing a transformation example of a screen displayed from the display 140 in the information processing apparatus 100 and the sound information output from the sound output unit 150 according to the second embodiment of the present disclosure.

FIG. 20 is a diagram showing a transformation example of information exchanged between the information processing apparatus 100 and the information processing apparatus 200 according to the second embodiment of the present disclosure. Figure 20 shows an example of the exchange of information corresponding to Figure 19.

FIG. 19A shows an example of a case where a broadcast wave is received and an image based on the broadcast wave and the sound of the image are output from the information processing device 100.

FIG. 19B shows in the state shown in FIG. 19A that the transmission of a stream from the information processing device 200 to the information processing device 100 has been operated by the user of the information processing device 200 to use Wi-Fi CERTIFIED Miracast. An example of the situation being started.

In this case, the controller 170 of the information processing device 100 uses Wi-Fi CERTIFIED Miracast to detect the connection. Then, the controller 170 of the information processing device 100 automatically sets the base for display on a screen An image of the broadcast wave is sent to the main screen and the newly added image from the information processing device 200 is set to the sub-screen. In this way, the controller 170 of the information processing device 100 can determine that the image from the information processing device 200 is set as a sub-screen (331 shown in FIG. 20).

Furthermore, using this decision as a trigger, the controller 170 of the information processing device 100 requests the information processing device 200 to reduce the audio data and transmit it. In this case, the information processing device 100 sets the least significant bit B0 of the control bitmap (control-bitmap) to 1 (shown in FIG. 17) in the RTSP M3 response message (which is a request message for the received RTSP M3 Response), and send the message to the information processing device 200, thereby requesting the reduction of audio data (332, 333 shown in Figure 20). At that time, the controller 170 of the information processing apparatus 100 does not specify the reduction method. Therefore, the information processing device 200 can automatically or manually determine the reduction method. Next, the information processing device 200 transmits a stream in which the audio data is reduced to the information processing device 100 (334, 335 shown in FIG. 20).

FIG. 19C is shown in the state shown in FIG. 19B, where the broadcast wave-based image 327 is set as the sub-screen and the image 328 from the information processing device 200 is set as the main screen by user operation Example (336 shown in Figure 20).

In this case, the information processing device 100 transmits an RTSP M110 request message (shown in FIG. 11) to the information processing device 200 and requests an increase in audio data (337, 338 shown in FIG. 20). Furthermore, the information processing device 200 transmits "OK" to RTSP The M110 requests a message (shown in FIG. 11) to the information processing device 100 (339, 340 shown in FIG. 20).

Next, the information processing device 200 transmits a stream containing audio data to the information processing device 100 (341, 342 shown in FIG. 20). In this case, the sound of the video 327 based on the broadcast wave is not output, as shown in Figure 19C. Furthermore, the sound 323 of the image 328 from the information processing device 200 is output from the sound output unit 150.

Here, when the sound 323 of the image 328 from the information processing device 200 starts to be output from the sound output unit 150, if the volume of the sound 323 is the same as the sound 322 of the image based on the broadcast wave, it may suddenly output a lot the sound of. Thus, when the sound 323 of the image 328 from the information processing device 200 starts to be output from the sound output unit 150, the sound volume setting value (amount) of the sound 322 smaller than the image based on the broadcast wave can be reset.

Furthermore, it is assumed that in the state shown in FIG. 19, the image 327 based on the broadcast wave is set as the main screen and the image 328 from the information processing device 200 is set as the sub-screen by user operation An example of the situation (343 shown in Figure 20).

In this case, the information processing device 100 transmits an RTSP M110 request message (shown in FIG. 18) to the information processing device 200 and requests a reduction in audio data (344, 345 shown in FIG. 20). Here, it is assumed that the method in which the audio PES packet is not included (second audio data reduction method) is specified to be used for decreasing audio data Methods.

In addition, the information processing device 200 transmits an "OK" RTSP M110 request message (shown in FIG. 18) to the information processing device 100 (346, 347 shown in FIG. 20).

Next, the information processing device 200 transmits a stream in which the audio data is reduced to the information processing device 100 (348, 349 shown in FIG. 20).

Operation example of receiving device

21 is a flowchart showing an example of a processing procedure of reception processing by the information processing apparatus 100 according to the second embodiment of the present disclosure. Figure 21 shows an example of a situation where both the source device and the sink device are adapted to specific functions.

First, the controller 170 of the information processing device 100 determines whether the start trigger of Wi-Fi CERTIFIED Miracast has occurred (step S831). When the start trigger does not occur (step S831), the monitoring system continues.

When the start trigger has occurred (step S831), the controller 170 determines whether audio data is needed at the start (step S832). When the audio data is not needed at the beginning (step S832), the controller 170 sends a message requesting the reduction of the audio data to the source device to start sending a stream (step S833). For example, when sending the RTSP M3 response message shown in Figure 17 as a response to the received RTSP M3 request message shown in Figure 17 At this time, the controller 170 sets the minimum effective bit B0 of the control bitmap to 1, and requests the reduction of audio data.

Next, the controller 170 starts a session of Wi-Fi CERTIFIED Miracast (step S834). In this case, the exchanged streams do not contain audio data.

Then, the controller 170 performs processing of receiving, decoding, and regenerating a stream in which the audio data is reduced, for example (step S835). In this case, the image based on the image data contained in the received stream is displayed on the display 140. However, the received stream does not contain audio data, and therefore no sound is output.

Next, the controller 170 determines whether the image displayed on the display 140 (based on the received stream image) requires audio data (step S836). When the audio data is unnecessary (step S836), the operation returns to step S835.

When audio data is needed (step S836), the controller 170 sends a message requesting the increase of audio data to the source device to send a stream (step S837). For example, the controller 170 transmits the RTSP M110 request message shown in FIG. 11.

After the message is transmitted, the communication unit 110 receives a response to the message from the source device (step S838). The source device sends "OK" in the RTSP M110 response message shown in Figure 11.

Meanwhile, when the audio data is needed at the beginning (step S832), the controller 170 transmits a request to increase the audio data Add a message to the source device to send a stream (step S839). For example, the controller 170 transmits the RTSP M3 response message shown in FIG. 6.

Next, the controller 170 starts a session of Wi-Fi CERTIFIED Miracast (step S840). In this case, the exchanged streams contain audio data.

Next, the controller 170 performs processing of receiving, decoding, and regenerating a stream containing audio data, for example (step S841). In this case, the image based on the image data included in the received stream is displayed on the display 140, and the sound based on the audio data included in the received stream is output from the sound output unit 150.

Next, the controller 170 determines whether the image displayed on the display 140 (based on the received stream image) does not require audio data (step S842). When audio data is needed (step S842), the operation returns to step S841.

When the audio data is unnecessary (step S842), the controller 170 determines the audio data reduction method (step S843). For example, the controller 170 may determine the audio data reduction method based on the number of connected source devices, the performance of the source devices, the surrounding communication environment, and the like.

For example, when the first method (first audio data reduction method) (step S843), the controller 170 transmits a message requesting the reduction of audio data to the source device to transmit a stream (step S844). For example, the controller 170 transmits the data shown in Figure 10 RTSP M110 request message.

After the message is transmitted, the communication unit 110 receives a response to the message from the source device (step S845). For example, the source device sends "OK" in the RTSP M110 response message shown in Figure 10.

For example, when the second method (second audio data reduction method) (step S843), the controller 170 transmits a message requesting the deletion of audio data to the source device to transmit a stream (step S846). For example, the controller 170 transmits the RTSP M110 request message shown in FIG. 18.

After the message is transmitted, the communication unit 110 receives a response to the message from the source device (step S847). The source device sends "OK" in the RTSP M110 response message shown in Figure 18.

Next, the controller 170 performs processing to receive, decode, and reproduce a stream in which the audio data is deleted, for example (step S848). In this case, the image based on the image data contained in the received stream is displayed on the display 140. However, the received stream does not contain audio data, and therefore no sound is output.

Next, the controller 170 determines whether the image displayed on the display 140 (based on the received stream image) requires audio data (step S849). When the audio data is unnecessary (step S849), the operation returns to step S848. When audio data is needed (step S849), the operation returns to step S837.

In this way, when making a request to the source device, the controller 170 may specify: a first method for reducing sound data (the MPEG2-TS multiplex signal containing PES packets carrying sound data is generated and transmitted) One of the second audio data reduction method (in which MPEG2-TS multiplex signal without PES packet carrying audio data is generated and transmitted).

Source device operation example

FIG. 22 is a flowchart showing an example of a processing procedure of transmission processing by the information processing device 200 according to the second embodiment of the present disclosure. Figure 22 shows an example of a situation where both the source device and the sink device are adapted to specific functions.

First, the controller of the information processing device 200 determines whether the start trigger of Wi-Fi CERTIFIED Miracast has occurred (step S851). When the start trigger does not occur (step S851), the monitoring system continues.

When the start trigger occurs (step S851), the controller of the information processing device 200 determines whether the received RTSP M3 response message indicates that the audio data is unnecessary (step S852).

When the audio data is designated as unnecessary (step S852), the controller of the information processing device 200 starts a Wi-Fi CERTIFIED Miracast session (step S853). In this case, the exchanged streams do not contain audio data.

Then, the controller of the information processing device 200 is generated from its The middle audio data is reduced by one stream and the generated stream is transmitted to the receiving device (step S854). In this case, the image based on the image data contained in the received stream is displayed on the display of the receiving device. However, no sound is output.

Next, the controller of the information processing apparatus 200 determines whether the message for requesting the increase of audio data has been received from the receiving device (step S855). For example, the controller of the information processing device 200 determines whether the RTSP M110 request message shown in FIG. 11 has been received. When the message is not received from the receiving device (step S855), the operation returns to step S854.

When the message has been received from the receiving device (step S855), the controller of the information processing apparatus 200 transmits a response to one of the messages to the receiving device (step S856). For example, the information processing device 200 transmits "OK" in the RTSP M110 response message shown in Figure 11.

When the audio data is specified as necessary (step S852), the controller of the information processing device 200 starts a Wi-Fi CERTIFIED Miracast dialogue (step S857). In this case, the exchanged streams contain audio data.

Next, the controller of the information processing apparatus 200 generates a stream containing audio data and transmits the generated stream to the receiving device (step S858). In this way, the image based on the image data contained in the transmitted stream is displayed on the display of the receiving device, and the sound based on the audio data contained in the transmitted stream is output from the sound of the receiving device Output unit output.

Next, the controller of the information processing apparatus 200 determines whether a message for requesting reduction of audio data has been received from the receiving device (step S859). For example, the controller of the information processing device 200 determines whether the RTSP M110 request message shown in FIG. 10 or FIG. 18 has been received. When the message is not received from the receiving device (step S859), the operation returns to step S858.

When the message has been received from the receiving device (step S859), the controller of the information processing apparatus 200 confirms the audio data reduction method specified by the message (step S860). Next, when the reduction method is the first method (in the case of the RTSP M110 request message shown in FIG. 10) (step S860), the controller of the information processing apparatus 200 transmits one of the messages to the receiving device (step S861). For example, the controller of the information processing device 200 transmits "OK" in the RTSP M110 response message shown in Figure 10.

Furthermore, when the reduction method is the second method (in the case of the RTSP M110 request message shown in FIG. 18) (step S860), the controller of the information processing device 200 determines whether the information processing device 200 is adapted to the second method (Step S862). Next, when the information processing apparatus 200 is not adapted to the second method (step S862), the operation proceeds to step S861.

When the information processing device 200 is adapted to the second method (step S862), the controller of the information processing device 200 sends a response to one of the messages to the receiving device (step S863). For example, information processing equipment Prepare 200 to send "OK" in the RTSP M110 response message shown in Figure 18.

Next, the controller of the information processing apparatus 200 generates a stream in which the audio data is deleted and transmits the generated stream to the receiving device (step S864). In this case, the image based on the image data contained in the received stream is displayed on the display of the receiving device. However, no sound is output.

Next, the controller of the information processing apparatus 200 determines whether the message for requesting the increase of audio data has been received from the receiving device (step S865). When the message is not received from the receiving device (step S865), the operation returns to step S864. In contrast, when the message has been received from the receiving device (step S865), the operation proceeds to step S856.

Example of using another message

The above has explained an example in which the change of the order of RTSP message exchange added to Wi-Fi CERTIFIED Miracast release-1 is avoided as much as possible. That is, the above has been described in the example in which a request for starting without including any audio data is included in the M3 GET_PARAMETER response.

The following will explain the situation in which the order of RTSP message exchange can be changed (for example, successor standard or own implementation). For example, M3R request/response messages and M4R request/response messages are defined. By reversing M3 request/response message and M4 request/response message are obtained. Then, capability notification, parameter setting, and the like between the source device and the receiving device are performed using the M3R request/response message or the M4R request/response message.

FIG. 23 is a sequence diagram showing a communication processing example of the source device 50 and the sink device 60 according to the second embodiment of the present disclosure. It should be noted that Figure 23 shows that the newly defined RTSP message is added to the example shown in Figure 5. That is, RTSP M3R request, RTSP M3R response, RTSP M4R request, and RTSP M4R response are added.

Here, M3 GET_PARAMETER and M4 SET_PARAMETER are used to send a request from the source device to the sink device. Figure 23 shows the sequence of RTSP messages when the inverted message is newly defined.

As shown in FIG. 23, the receiving device 60 uses the M3R request message (501) to inquire about the adaptability of the source device 50 (eg, whether the source device 50 can adapt to the reduction in audio data). The source device 50 uses a response to the M3R request message (M3R response message) (502) to notify the receiving device 60 about the adaptability of the source device 50.

Furthermore, the receiving device 60 uses the M4R request message to set, so that the source device 50 transmits a stream without including audio data (503). The source device 50 uses the response to the M4R request message (M4R response message) (504) to notify the receiving device 60 of "OK".

24 to 26 are diagrams showing examples of information exchanged between information processing apparatuses according to the second embodiment of the present disclosure.

Figure 24 shows an example of bit definition when "wfd_audio_stream_control" is used in the newly defined RTSP message. In this example, the receiving device 60 can explicitly notify the source device 50 of the fact that the audio system is not included in SET_PARAMETER. Therefore, in the example shown in Figure 16, the "Initial audio-off" of the least significant bit (at bit position B0) can be ignored.

Figure 25 shows the configuration examples of RTSP M3R request, RTSP M3R response, RTSP M4R request, and RTSP M4R response.

Next, a configuration example of specific messages will be explained with reference to FIG. 25.

The receiving device 60 includes "wfd_audio_stream_control" (dashed rectangle 420 shown in FIG. 25) in the RTSP M3R request message and transmits the message to the source device 50 (501 shown in FIG. 23). Therefore, the receiving device 60 may ask whether the source device 50 can adapt to the specific function.

In response to the inquiry, the source device 50 sends an RTSP M3R response message to the receiving device 60, and notifies the receiving device 60 that the source device 50 can adapt to the specific function (502 shown in FIG. 23). This example illustrates a case where the source device 50 is adapted to all (re)functions including audio data, functions of reducing audio data, and functions of reducing audio data by another method. Therefore, the source device 50 sets the binary number 1110 with the hexadecimal number "E (dotted rectangle 421 in Figure 25)" And send the response (502 shown in Figure 23).

Next, the receiving device 60 transmits the RTSP M4R request message to the source device 50, and sets (SET) so that the source device 50 reduces the audio data and transmits it (503 shown in FIG. 23). In this example, the RTSP M3R request message confirms that the source device 50 has the function of reducing audio data by another method. Therefore, the receiving device 60 only sets the bit position B1 shown in FIG. 24 to 1, and can specify the transmission if any audio data is included. In response, the source device 50 sends "OK" to the RTSP M4R response message (shown in Figure 25) to the receiver device 60 (504 shown in Figure 23).

In this way, the receiving device 60 only sets the bit position B1 shown in FIG. 24 to 1. Therefore, the binary number 0010 and the hexadecimal number "2 (dotted rectangle 422 in Figure 25)" are used as the RTSP parameter setting values.

Therefore, the source device 50 sets a format for transmitting RTSP M4 requests. In this case, the source device 50 may perform the setting including the audio format described above, or perform the setting not including the audio format. Figure 26 shows an example in which the settings are performed without including the audio format.

Figure 26 shows an example configuration of RTSP M4 request and RTSP M4 response.

Here, it is assumed that the transmission without audio data from the beginning is specified, the method in which no audio basic stream is included is selected as the audio reduction method, and the RTSP M4 request message is not It is transmitted if there is any information including audio format. In this case, when the receiving device 60 uses the RTSP M110 request message to request the source device 50 to add audio data, the source device 50 needs to use the RTSP M110 response message to notify the receiving device 60 of "OK" and further add a parameter for Specify the audio format and use RTSP M4 request message to send it. The configuration example of the RTSP M4 request message used in this case is the same as the example shown in FIG. 7, and therefore its description is omitted here.

<3. Third embodiment>

The third embodiment of the present disclosure describes an example in which the receiving device detects the movement or presence of a person and requests the reduction of audio data based on the detection result.

It should be noted that the configuration of the information processing device in the third embodiment of the present disclosure is substantially the same as the configuration of the information processing devices 100 and 200 shown in FIG. 1 and the like. Therefore, elements common to the first embodiment of the present disclosure are denoted by element symbols of the first embodiment of the present disclosure, and description of these elements is omitted.

Home screen display switching example

The following is an example of a case where when the screen of the source device is displayed on the receiving device, the output of the sound is controlled based on whether the user is watching the screen.

FIG. 27 is a diagram showing the third embodiment according to the present disclosure. A diagram of a transformation example of one of the screen displayed on the display 140 in the information processing apparatus 100 and the sound information output from the sound output unit 150.

FIG. 28 is a diagram showing a transformation example of information exchanged between the information processing apparatus 100 and the information processing apparatus 200 according to the third embodiment of the present disclosure. Figure 28 shows an example of the exchange of information corresponding to Figure 27.

FIG. 27A shows an example of a situation where the image from the information processing device 200 and the sound of the image are output from the information processing device 100. That is, FIG. 27A shows an example in which streaming is performed from the information processing apparatus 200 to the information processing apparatus 100 using Wi-Fi CERTIFIED Miracast using settings including audio (361, 362 shown in FIG. 28). In this case, the sound 351 of the image from the information processing device 200 is output from the sound output unit 150.

FIG. 27B is shown in the state shown in FIG. 27A, assuming that the user has not viewed the screen displayed on the display 140 (363 shown in FIG. 28).

For example, it is possible to detect the presence or movement of a person through the sensor 120. Furthermore, the controller 170 of the information processing device 100 may determine whether the user is viewing the screen of the display 140 based on the detection result of the sensor 120. Next, when it is determined that the user has not viewed the screen of the display 140 for a certain period of time, the controller 170 of the information processing device 100 turns off the sound output from the sound output unit 150.

In this case, the information processing device 100 transmits RTSP M110 requests a message (shown in Figure 10) to the information processing device 200 and requests a reduction in audio data (364, 365 shown in Figure 28). The information processing device 200 sends an "OK" RTSP M110 request message (shown in Figure 10) to the information processing device 100 (366, 367 shown in Figure 28).

Next, the information processing device 200 transmits a stream in which the audio data is reduced to the information processing device 100 (368, 369 shown in FIG. 28). In this case, the sound of the image from the information processing device 200 is not output, as shown in FIG. 27B.

Furthermore, in this case, the controller 170 of the information processing apparatus 100 may display a notification message indicating that “the sound has been deleted after determining that there is no viewer” on the display 140, as shown in FIG. 27B. Furthermore, for example, the controller 170 of the information processing device 100 can attract the user's attention by outputting a warning sound. For example, after the output of the notification or warning sound, the user performs some type of action (such as a waved motion) or a given response from the sensor 120 is observed within a given period of time, information The processing device 100 may not request the reduction of sound effect data.

Furthermore, when it is determined that the user has not watched the screen of the display 140 for a certain period of time, the controller 170 of the information processing device 100 may gradually decrease the sound volume setting value (volume) of the sound output from the sound output unit 150 value). In this case, after the sound volume setting value (volume value) of the sound output from the sound output unit 150 is sufficiently reduced, the controller 170 of the information processing apparatus 100 may request The reduction of audio data.

Furthermore, when it is determined that the user has not watched the screen of the display 140 for a certain period of time, the controller 170 of the information processing apparatus 100 may transmit a request for reducing power consumption to the information processing apparatus 200. For example, the controller 170 of the information processing device 100 may send a request to the information processing device 200 to reduce the resolution of the image or reduce the update rate.

FIG. 27C shows an example of the situation in which the user viewing the screen displayed on the display 140 has been detected in the state shown in FIG. 27B (370 shown in FIG. 28). For example, assume that a user who has viewed the message shown in FIG. 27B returns to the information processing device 100. Furthermore, the presence of the user can be detected when the user performs a given motion (such as a wave motion, etc.).

In this case, the information processing device 100 transmits an RTSP M110 request message (shown in FIG. 11) to the information processing device 200 and requests an increase in audio data (371, 372 shown in FIG. 28). The information processing device 200 transmits "OK" in the RTSP M110 request message (shown in FIG. 11) to the information processing device 100 (373, 374 shown in FIG. 28).

Next, the information processing device 200 transmits a stream containing audio data to the information processing device 100 (375, 376 shown in FIG. 28). In this case, the sound 352 of the image from the information processing device 200 is output from the sound output unit 150, as shown in FIG. 27C As shown in.

In this way, the third embodiment of the present disclosure is substantially the same as the first embodiment of the present disclosure, except for the trigger for requesting the reduction of audio data.

Operation example of receiving device

FIGS. 29 and 30 are flowcharts showing an example of a processing procedure of reception processing by the information processing apparatus 100 according to the third embodiment of the present disclosure. Figures 29 and 30 show examples of situations where both the source device and the sink device are adapted to specific functions. Furthermore, the processing procedures shown in FIGS. 29 and 30 (steps S871, S872, S874, S884 to S886, S888 to S890) correspond to the processing procedures shown in FIG. 14 (steps S801 to S803, S805 to S807, S809 to S811). Therefore, its description is partially omitted.

Furthermore, the dotted rectangle 601 shown in FIG. 29 corresponds to the processing for determining whether audio data is necessary in FIG. 14 (step S808). Furthermore, the dashed rectangle 602 shown in FIG. 30 corresponds to the processing for determining whether audio data is unnecessary in FIG. 14 (step S804).

After starting the session of Wi-Fi CERTIFIED Miracast (step S872), the controller 170 resets the first timer (step S873). Here, the first timer is a timer used to determine whether notification information for a user is to be output. Then, the controller 170 performs processing of receiving, decoding, and regenerating a stream containing audio data, for example (Step S874).

Next, the controller 170 determines whether the presence or movement of the user has been detected by the sensor 120 (step S875). When the user's presence or movement has been detected (step S875), the operation returns to step S873.

When the presence or movement of the user is not detected (step S875), the controller 170 increases the value of the first timer (step S876). Next, the controller 170 determines whether the first timer has exceeded the upper limit value (step S877). When the first timer does not exceed the upper limit value (step S877), the operation returns to step S874.

When the first timer has exceeded the upper limit value (step S877), the controller 170 resets the second timer (step S878). Here the second timer is a timer used to determine whether the reduction of audio data is to be requested to the source device. Then, the controller 170 outputs (for example, display or sound output) notification information to the user (step S879).

Next, the controller 170 determines whether the presence or movement of the user has been detected by the sensor 120 (step S880). When the presence or movement of the user has been detected by the sensor 120 (step S880), the controller 170 resets the second timer (step S881), and the operation returns to step S873.

When the presence or movement of the user is not detected (step S880), the controller 170 increases the value of the second timer (step S882). Next, the controller 170 determines whether the second timer has exceeded the upper limit value (step S883). When the second timer does not exceed the upper limit (step S883), the operation returns to step S879. When the second timer has exceeded the upper limit value (step S883), the operation proceeds to step S884.

Furthermore, the controller 170 determines whether the presence or movement of the user has been detected by the sensor 120 (step S887). When the user's presence or movement has been detected (step S887), the operation proceeds to step S888. In contrast, when the user's presence or movement has been detected (step S887), the operation returns to step S886.

In this way, the trigger for requesting the source device to reduce the audio data and transmit the data (first trigger) may be a trigger caused by the detection of the user's presence around the information processing device 100 or the user's movement.

It should be noted that the operation example of the source device is the same as in FIG. 15, and therefore its description is omitted here.

As described above, in the embodiment of the present disclosure, in the transmission method using MPEG2-TS, when the fact that "audio data is unnecessary" is notified by the connected receiving device, the source device reduces the audio data and transmits the data. For example, the source device can reduce the audio data stored in the payload of the PES packet and transmit the data without changing the PID. In this way, the source device can appropriately adjust the data volume of the basic audio stream according to the request from the connected sink device.

Therefore, in the embodiment of the present disclosure, when audio is not needed, the amount of data required for the wireless part can be reduced, which can avoid collision or congestion. Furthermore, it is possible to reduce the power consumption of both the source device and the sink device.

Furthermore, when the receiving device operates on the audio master, it is possible to perform synchronous regeneration based on PTS.

A receiving device capable of regenerating video in free run of the audio reproduction clock can request the source device to perform transmission without including even PTS. In this way, it is possible to avoid collision or congestion by further reducing the amount of data and reducing the power consumption of both the source device and the sink device.

Furthermore, it is possible to increase and decrease the audio data according to the switching operation of the main screen and the sub-screen (for example, user operation), automatic movement (for example, increase of the display screen), or whether the user is watching the screen. In this case, it is not necessary to perform complicated user operations to increase and decrease the audio data.

It should be noted that the first and second embodiments of the present disclosure illustrate an example in the case where an image based on broadcast waves and an image from the source device are simultaneously displayed on the receiving device. However, the first and second embodiments of the present disclosure can be similarly applied when images from two or more source devices are simultaneously displayed on the receiving device, or when multiple images from one source device are simultaneously Shown on the receiving device. Furthermore, the third embodiment of the present disclosure describes an example in the case where an image from a source device is displayed on the receiving device. However, the third embodiment of the present disclosure can also be similarly applied when images from two or more source devices are simultaneously displayed on the receiving device.

<4. Application example>

The technology according to the present disclosure can be applied to various products. For example, the information processing devices 100, 200, and 210 may be implemented as mobile terminals (such as smartphones, tablet PCs (personal computers), notebook PCs, portable game terminals, or digital cameras), stationary terminals (Such as a TV receiver, printer, digital scanner, or network storage), or a car terminal (such as a car navigation device). In addition, the information processing devices 100, 200, and 210 can be implemented as terminals that perform M2M (machine-to-machine) communication (also known as MTC (machine type communication) terminals), such as smart meters, vending machines, and remote control monitoring Device, or POS (point of sale) terminal. Furthermore, the information processing devices 100, 200, and 210 may be wireless communication modules (for example, integrated circuit modules composed of one die) provided in the terminal.

On the other hand, for example, the information processing devices 100, 200, and 210 may be implemented as wireless LAN access points (also referred to as wireless base stations) with or without router functions. The information processing devices 100, 200, and 210 may be implemented as mobile wireless LAN routers. The information processing devices 100, 200, and 210 may also be wireless communication modules (for example, integrated circuit modules configured with one die) provided in the device.

4-1. First application example

FIG. 31 is a block diagram showing an example of a schematic configuration of a smartphone 900 in which the disclosed technology can be applied. The smartphone 900 includes a processor 901, a memory 902, a storage 903, and an external connection Interface 904, camera 906, sensor 907, microphone 908, input device 909, display device 910, speaker 911, wireless communication interface 913, antenna switch 914, antenna 915, bus 917, battery 918, and auxiliary controller 919 .

For example, the processor 901 may be a CPU (Central Processing Unit) or SoC (System On Chip) and control functions of the application layer and other layers of the smartphone 900. The memory 902 includes RAM (random access memory) and ROM (read only memory), and stores programs and data executed by the processor 901. The storage 903 may include a storage medium such as a semiconductor memory or a hard disk. The external connection interface 904 is an interface for connecting an external attachment device such as a memory card or a USB (Universal Serial Bus) device to the smartphone 900.

The camera 906 has an image sensor (for example, CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor)) to generate captured images. The sensor 907 may include a sensor group including, for example, a positioning sensor, a gyro sensor, a geomagnetic sensor, an acceleration sensor, and the like. The microphone 908 converts the sound input to the smartphone 900 into an audio signal. The input device 909 includes, for example, a touch sensor (which detects a touch on the screen of the display device 910), a key pad, a keyboard, buttons, switches, and the like, to receive manipulations from the user or News. The display device 910 has a screen, such as a liquid crystal display (LCD) or an organic light emitting diode (OLED) display, to display the output image of the smartphone 900. The speaker 911 converts the audio signal output from the smartphone 900 into sound.

The wireless communication interface 913 supports one or more wireless LAN standards of IEEE 802.11a, 11b, 11g, 11n, 11ac, and 11ad to perform wireless LAN communication. The wireless communication interface 913 can communicate with another device via a wireless LAN access point in infrastructure mode. In addition, the wireless communication interface 913 can directly communicate with another device in an ad hoc mode or a direct communication mode in Wi-Fi Direct (registered trademark). Wi-Fi Direct is different from the specific mode, and therefore one of the two terminals operates as an access point. However, the communication system is directly executed between the terminals. The wireless communication interface 913 may typically include a baseband processor, RF (radio frequency) circuit, power amplifier, and the like. The wireless communication interface 913 may be a single-chip module in which a communication control program is stored in memory, a processor executes the program, and related circuits are integrated. In addition to the wireless LAN solution, the wireless communication interface 913 can support another type of wireless communication solution, such as a cellular communication solution, a short-range wireless communication solution, or a proximity wireless communication solution (proximity wireless communication). The antenna switch 914 switches the connection purpose of the antenna 915 for a plurality of circuits (for example, circuits used in different wireless communication schemes) included in the wireless communication interface 913. The antenna 915 has a single or a plurality of antenna elements (for example, a plurality of antenna elements constitute a MIMO antenna), and is used for transmission and reception of wireless signals from the wireless communication interface 913.

It should be noted that the smartphone 900 may include a plurality of antennas (for example, an antenna for a wireless LAN or an antenna for a proximity wireless communication scheme, or the like), and is not limited to the example in FIG. 31. in In this case, the antenna switch 914 can be omitted from the configuration of the smartphone 900.

The bus 917 connects the processor 901, memory 902, storage 903, external connection interface 904, camera 906, sensor 907, microphone 908, input device 909, display device 910, speaker 911, wireless communication interface 913, and auxiliary The controllers 919 are connected to each other. The battery 918 supplies electric power to each of the blocks of the smartphone 900 shown in FIG. 31 through a power supply line (represented by a broken line in the figure). For example, the auxiliary controller 919 causes the minimum function required for the smartphone 900 operated in the sleep mode.

In the smart phone 900 shown in FIG. 31, the control unit 170 described with reference to FIG. 2 may be provided in the wireless communication interface 913. At least some functions may be provided to the processor 901 or the auxiliary controller 919.

It should be noted that when the processor 901 performs the function of an access point at the application level, the smartphone 900 can operate as a wireless access point (software AP). In addition, the wireless communication interface 913 can have the function of a wireless access point.

4-2. Second application example

FIG. 32 is a block diagram showing an example of a schematic configuration of a car navigation device 920 in which the disclosed technology can be applied. The car navigation device 920 includes a processor 921, a memory 922, a GPS (Global Positioning System) module 924, a sensor 925, a data interface 926, a content player 927, storage media interface 928, input device 929, display device 930, speaker 931, wireless communication interface 933, antenna switch 934, antenna 935, and battery 938.

The processor 921 may be, for example, a CPU or SoC that controls the navigation function and other functions of the car navigation device 920. The memory 922 includes RAM and ROM that store programs and data executed by the processor 921.

The GPS module 924 uses GPS signals received from GPS satellites to measure the position of the car navigation device 920 (for example, latitude, longitude, and altitude). The sensor 925 may include a sensor group including, for example, a gyro sensor, a geomagnetic sensor, a barometric pressure sensor, and the like. For example, the data interface 926 is connected to the in-vehicle network 941 through a terminal that is not shown to obtain data generated on the car side.

The content player 927 reproduces the content stored in the storage medium (for example, CD or DVD) inserted in the storage medium interface 928. The input device 929 includes, for example, a touch sensor (which detects a touch on the screen of the display device 930), buttons, switches, and the like to receive manipulation or information from the user. The display device 930 has a screen (for example, LCD or OLED) to display the navigation function or the image of the reproduced content. The speaker 931 outputs the sound of the navigation function or the reproduced video.

The wireless communication interface 933 supports one or more wireless LAN standards of IEEE 802.11a, 11b, 11g, 11n, 11ac, and 11ad to perform wireless LAN communication. The wireless communication interface 933 can be connected to another device via a wireless LAN access point in infrastructure mode Set communication. In addition, the wireless communication interface 933 can directly communicate with another device in an ad hoc mode or a direct communication mode in Wi-Fi Direct. The wireless communication interface 933 may typically have a baseband processor, RF circuit, power amplifier, and the like. The wireless communication interface 933 may be a single-chip module in which a communication control program is stored in memory, a processor executes the program, and related circuits are integrated. In addition to the wireless LAN solution, the wireless communication interface 933 can support another type of wireless communication solution, such as a short-range wireless communication solution, a proximity wireless communication solution, or a cellular wireless communication solution. The antenna switch 934 switches the connection purpose of the multiple antennas 935 included in the wireless communication interface 933. The antenna 935 has a single or multiple antenna elements and is used to transmit and receive wireless signals from the wireless communication interface 933.

It should be noted that the car navigation device 920 may include a plurality of antennas, and is not limited to the example in FIG. 32. In this case, the antenna switch 934 can be omitted from the configuration of the car navigation device 920.

The battery 938 supplies electric power to each of the blocks of the car navigation device 920 shown in FIG. 32 through a power supply line (represented by a broken line in the figure). In addition, the battery 938 accumulates the electric power supplied from the vehicle.

In the car navigation device 920 shown in FIG. 32, the control unit 170 described with reference to FIG. 2 may be provided in the wireless communication interface 933. At least some functions may be provided to the processor 921.

The wireless communication interface 933 is operable as the information processing devices 100, 200, and 210 described above to provide wireless connection to the user Board the terminal owned by the vehicle.

The disclosed technology can be implemented as an in-vehicle system (or vehicle) 940 including one or more blocks of the car navigation device 920, the vehicle intranet 941, and the vehicle-side module 942 described above. The vehicle-side module 942 generates vehicle-side data, such as vehicle speed, engine revolutions, or error information, and outputs the generated data to the vehicle's intranet 941.

4.3. Third application example

FIG. 33 is a block diagram showing an example of a schematic configuration of a wireless access point 950 in communication with an information processing device in which the disclosed technology can be applied. The wireless access point 950 includes a controller 951, a memory 952, an input device 954, a display device 955, a network interface 957, a wireless communication interface 963, an antenna switch 964, and an antenna 965.

The controller 951 may be, for example, a CPU or a digital signal processor (DSP) and operates various functions of the Internet Protocol (IP) layer of the wireless access point 950 at higher layers (for example, access restriction, routing, encryption, Firewall, and records management). The memory 952 includes RAM and ROM and stores programs executed by the controller 951 and various types of control data (for example, terminal lists, routing tables, encryption keys, security settings, and records).

The input device 954 includes, for example, buttons or switches, and receives manipulation from the user. The display device 955 includes LED lights and displays the operation status of the wireless access point 950.

Network interface 957 is for connecting wireless access point 950 to wired Wired communication interface of communication network 958. The network interface 957 may include a plurality of connection terminals. The wired communication network 958 may be a LAN, such as Ethernet (registered trademark) or may be a wide area network (WAN).

The wireless communication interface 963 supports one or more wireless LAN standards of IEEE 802.11a, 11b, 11g, 11n, 11ac, and 11ad to provide wireless communication to connect to nearby terminals as access points. The wireless communication interface 963 may typically include a baseband processor, an RF circuit, and a power amplifier. The wireless communication interface 963 may be a single-chip module in which a communication control program is stored in a memory, a program is executed by a processor, and related circuits are integrated. The antenna switch 964 switches the connection purpose of the plurality of circuits included in the wireless communication interface 963 to the antenna 965. The antenna 965 includes an antenna element or a plurality of antenna elements and is used to transmit and receive wireless signals through the wireless communication interface 963.

In the wireless access point 950 shown in FIG. 33, the control unit 170 described with reference to FIG. 2 may be provided in the wireless communication interface 963. At least some functions may be provided to the controller 951.

Similarly, the above embodiment illustrates an example for realizing the technology, and indicates that the embodiment and the subject matter in the technology have a corresponding relationship with the subject matter within the scope of the patent application. Similarly, the target technology specified in the scope of the patent application and the target in the embodiments of the present technology (where the same name is appended) have a corresponding relationship. However, the present technology is not limited to this embodiment, and can be realized by various modifications of the embodiment without exceeding the scope of the present technology.

Similarly, the processing procedures described in the above embodiments can It is regarded as a method having a series of procedures, or it can be regarded as a program for causing a computer to execute a series of procedures or a non-transitory computer-readable recording medium having the program recorded on it. As for the recording medium, for example, a compact disc (CD), a compact disc (MD), a digital versatile disc (DVD), a memory card, and a Blu-ray (R) disc or the like are usable.

The effects described in this description are only examples, and the effects are not limited thereto, and may have other effects.

In addition, the present technology can also be structured as follows.

(1)

An information processing device includes: a circuit configured to receive a multiplexed video signal and audio signal from another information processing device using a moving picture expert group (MPEG) 2 transmission stream (TS); and upon request After the other information processing device reduces the amount of sound data, control is performed to cause one of the sound data transmissions included in the package specified by a packet identifier (PID) to be specified in a program map (PMT) The reduction of the data in the packetized elementary stream (PES) payload of one of the PES packets of the transport stream (TS) packet, and extraction of a presentation time stamp contained in a PES header portion of the PES packet ( PTS).

(2)

The information processing device according to (1), wherein the circuit makes the request to another information processing device based on a first trigger.

(3)

The information processing device according to (1), wherein when the request is made to another information processing device, the circuit specifies one of a first sound data reduction method and a second sound data reduction method, including carrying sound The MPEG2-TS multiplex signal of the PES packet of the data is generated and transmitted in the first sound data reduction method, and the MPEG2-TS multiplex signal of the PES packet that does not include the sound data is generated And is transmitted in the second sound data reduction method.

(4)

The information processing device according to (1), wherein the circuit makes the request to another information processing device based on a first trigger to reduce the amount of sound data, and wherein the circuit performs control to control when based on a second trigger After receiving the amount of sound data reduced to the MPEG2-TS multiplex signal in it, requesting the other information processing device to restore an original amount of sound data, extracting the sound data contained in the PES payload to An information processing device receives one of the MPEG2-TS multiplex signals for demultiplexing.

(5)

The information processing device according to (4), wherein the first trigger and the second trigger are caused by user operations.

(6)

The information processing device according to (2), wherein the first trigger is caused by detecting a user leaving the area around the information processing device or the user's movement.

(7)

The information processing device according to (1), wherein the circuit retains a sound volume setting value of the sound output before the request is made, requests the other information processing device to restore an original sound data volume, and when included in the PES The sound data in the payload is the sound volume setting value retained at a timing setting extracted after the request to restore the original sound data amount.

(8)

The information processing device according to (1), wherein the circuit requests the other information processing device to restore an original sound data volume, and when the sound data included in the PES payload is after the request to restore the original sound data volume Adjust the volume of a sound when it is extracted from the other information processing device.

(9)

An information processing device including: a circuit configured to use a moving picture expert group (MPEG) 2 transmission stream (TS) to transmit a multiplexed video signal and an audio signal to another information processing device; and perform control Based on a request from one of the other information processing devices, without changing the packet identifier (PID), reducing the sound data stored in the program map table (PMT) specified by the PID One TS packet, one PES packet, one sub-encapsulated elementary stream (PES) payload data volume, and multiplexing and transmitting PES packets used for audio signal transmission in which the data volume is reduced and for video PES packet for signal transmission.

(10)

The information processing device according to (9), wherein the circuit performs control to reduce the amount of PES payload data when the amount of sound data is requested to be reduced from substantially zero or completely zero when requested from the other information processing device One amount of information.

(11)

The information processing device according to (9), wherein the circuit performs control to send a PES packet for transmission of a sound signal in which the amount of data is reduced after multiplex processing and is requested to restore a from the other information processing device When the amount of original audio data, the general audio data stored in the PES payload, and multiplex processing and transmission of PES packets for audio signal transmission and PES packets for video signal transmission.

(12)

The information processing device according to (9), wherein the circuit performs control to use the MPEG2-TS multiplexing signal including the PES packet carrying the sound data other than the first sound data reduction when requested from the other information processing device When the sound data that is generated and transmitted in it is reduced to reduce the amount of sound data, the PES packet carrying the sound data is not generated without the information indicating the PID indicating the sound data in the PMT In the case where the second audio data is reduced, and there is no multiplexing processing, one PES packet for video signal transmission is transmitted.

(13)

The information processing device according to (12), wherein the circuit performs control To: when it is required to restore the original sound data amount from the other information processing device and when the second sound data reduction is performed without the multiplex processing of the PES packet for sound signal transmission, it is sent for the video signal The transmitted PES packet uses the PID specified for the audio data included in the PMT to store the general audio data stored in the PES payload, and multiplex processing and transmission of the PES packet for audio signal transmission and for video PES packet for signal transmission.

(14)

A communication system includes: a first information processing device having a circuit configured to use a moving picture expert group (MPEG) 2 transmission stream (TS) to transmit a multiplexed video signal and audio signal To a second information processing device; based on a request from the second information processing device, without changing the packet identifier (PID), reduce the number stored in the program map table (PMT) specified by the PID ) The voice data transmission described in) is one of the TS packets, one of the PES packets is a packetized basic stream (PES) payload data volume; and multiplexing and transmission are used for voice signal transmission in which the data volume is reduced PES packet and PES packet for video signal transmission; and a second information processing device with a circuit configured to cause the sound contained in the packet specified by the PID to be specified in the PMT Reduction of the data in the TS packet of the data packet that is sub-encapsulated in the PES payload of the PES packet, and extracting the header included in the PES after sending the request to the other information processing device to reduce the amount of sound data The PTS in the section.

(15)

An information processing method, implemented by an information processing device, including: performing communication through a circuit of the information processing device to use a dynamic image expert group (MPEG) 2 transmission stream (TS) from another information processing The device receives a multiplexed video signal and an audio signal; and performs control by the circuit of the information processing device to request the other information processing device to reduce the amount of audio data, and then executes the control to cause the packet to be included in the identification by a packet (PID) indicates a reduction in the data in the payload of a packetized elementary stream (PES) of one of the TS packets and one of the PE packets of the TS packets specified in a program map table (PMT), and Extract a PTS contained in a PES header part.

(16)

An information processing method including: performing communication through a circuit of the information processing device to transmit a multiplexed video signal and audio signal to another information using a moving picture expert group (MPEG) 2 transmission stream (TS) Processing equipment; and control performed by the circuit of the information processing equipment to reduce the storage of instructions specified by the PID without changing the packet identifier (PID) based on a request from the other information processing equipment The amount of data in a packetized elementary stream (PES) payload, and the multiplex processing and transmission used in the transmission of audio data described in a program map table (PMT), one of TS packets, one of PES packets Reduced sound signal transmission PES packets and PES packets used for video signal transmission.

(17)

A non-transitory computer-readable medium that stores a program that, when executed by an information processing device, causes the information processing device to execute a method that includes: performing communication to use a group of motion picture experts (MPEG ) 2 Transmission Stream (TS) receives a multiplexed video signal and audio signal from another information processing device; and performs control to request the other information processing device to reduce the amount of audio data before performing control to cause inclusion in the packet In a packetized elementary stream (PES) payload of a PES packet that transmits a TS packet specified in a program map table (PMT) of the sound data specified by a packet identifier (PID) Data reduction, and extraction of a PTS contained in a PES header section.

(18)

A non-transitory computer-readable medium that stores a program that, when executed by an information processing device, causes the information processing device to execute a method that includes: performing communication to use a group of motion picture experts (MPEG ) 2 Transmission Stream (TS) transmits a multiplexed video signal and audio signal to another information processing device; and performs control to change the packet identifier (PID) without changing the packet identifier based on a request from the other information processing device In the case of, reduce the sound stored in the program map table (PMT) specified by the PID. One of the TS packets, one of the PE packets, and one of the PES packets of the packetized basic stream (PES) payload data volume, and multiplexing and transmission. The PES packets and the use of the voice signals in which the data volume is reduced PES packets transmitted in the video signal.

100‧‧‧ Information processing equipment

110‧‧‧Communication unit

111‧‧‧Data processing unit

112‧‧‧Transmission processing unit

113‧‧‧Wireless interface unit

114‧‧‧ Antenna

120‧‧‧Sensor

130‧‧‧Operation receiving unit

140‧‧‧Monitor

150‧‧‧Sound output unit

160‧‧‧Memory

170‧‧‧Controller

Claims (17)

A first information processing device includes: a circuit configured to receive a first image signal and a first image signal from a second information processing device based on a moving picture expert group (MPEG) 2 transmission stream (TS) An audio signal; receiving a second image signal and a second audio signal from a third information processing device; transmitting first control information to the second information processing device, wherein the first control information indicates the first audio signal Reduction of the amount; sending second control information to the third information processing device, wherein the second control information indicates a reduction of the amount of the second sound signal; controlling the second information processing device or the third information processing device At least one of them is to stop the output of the data in at least one of the plural packed basic stream (PES) payloads, wherein the output of the data is based on at least one of the first control information or the second control information While being stopped, each of the plural PES packets includes a corresponding PES payload of one of the plural PES payloads, each of the plural PES payloads is in a corresponding TS packet of one of the plural TS packets, and the plural TS packets Each of them is specified by a corresponding packet identifier (PID) of a plurality of packet identifiers (PID), and each of the plurality of PIDs is in a program mapping table (PMT) specifies at least one of the first sound signal or the second sound signal; and extracts a presentation time stamp (PTS) included in a PES header portion of each of the plural PES packets. A first information processing device as claimed in item 1 of the patent scope, wherein the circuit is further configured to transmit at least one of the first control information or the second control information to the second information processing based on a first trigger Equipment or the third information processing equipment. For example, the first information processing device of claim 1, wherein the circuit is further configured to indicate one of a first sound data reduction method or a second sound data reduction method to the second information processing device or At least one of the third information processing equipment, in the first sound data reduction method, a first MPEG 2-TS multiplex signal is provided by the at least one of the second information processing equipment or the third information processing equipment Generated, the first MGEG 2-TS multiplex signal includes the at least one of the one sound signal or the second sound signal. In the second sound data reduction method, a second MPEG 2-TS The industrial signal is generated by the at least one of the second information processing device or the third information processing device, the second MGEG 2-TS multiplexing signal lacks the at least one of the one sound signal or the second sound signal By. For example, the first information processing device of the first item of patent scope, in which the circuit is further structured to: Based on a first trigger to transmit a first request to the at least one of the second information processing device or the third information processing device, wherein the first request corresponds to an original amount of the first sound signal or the Reduction of at least one of the original quantities of one of the second sound signals; receiving at least one MPEG 2-TS multiplexed signal including the reduced original quantity of the first sound signal or the reduced original quantity of the second sound signal At least one of the quantity; based on a second trigger to send a second request to the at least one of the second information processing device or the third information processing device, wherein the second request corresponds to the first sound signal At least one of the reduced original amount to the restoration of the original amount of the first sound signal or the reduced original amount of the second sound signal to the restoration of the original amount of the second sound signal; and After the transmission of the second request, the at least one of the first sound signal or the second sound signal is extracted from the at least one MPEG 2-TS multiplex signal. For example, in the first information processing device of claim 4, the first trigger and the second trigger are based on a user operation. For example, the first information processing device of claim 2 of the patent scope, wherein the first trigger is based on one of a user leaving from an area or a user moving, and the area is located in a specific area of the first information processing device Nearby. For example, the first information processing device of claim 1, wherein the circuit is further structured to: retain one sound of each of the first sound signal and the second sound signal Sound volume setting value; sending a first request to the at least one of the second information processing device or the third information processing device, wherein the first request corresponds to an original amount of the first sound signal or the first Reduction of at least one of the original quantities of one of the two sound signals; sending a second request to the second information processing device, wherein the second request corresponds to the reduced original quantity of the first sound signal to the first At least one of the restoration of the original quantity of the sound signal or the reduced original quantity of the second sound signal to the restoration of the original quantity of the second sound signal; and resetting the first quantity after the transmission of the second request The reserved sound volume setting value of each of an audio signal and the second audio signal. A first information processing device includes: a circuit configured to transmit a video signal and an audio signal to a second information processing device based on a Moving Picture Experts Group (MPEG) 2 transmission stream (TS); Receiving first control information from the second information processing device; stopping the output of data in a packetized elementary stream (PES) payload, wherein the output of the data is stopped based on the received first control information, A first PES packet includes the PES payload, and a TS packet includes the first PES packet, The TS packet is indicated by a packet identifier (PID), which specifies the transmitted audio signal in a program map table (PMT), and the PID is independent of the data in the PES payload Output; multiplex processing the first PES packet and a second PES packet to generate a first multiplexed PES packet, wherein the second PES packet includes the image signal; and transmitting the first multiplexed PES packet to the second Information processing equipment. For example, the first information processing device of claim 8, wherein the circuit is further configured to reduce the amount of the data to substantially zero or completely zero, and the reduction is based on the received first control information. For example, the first information processing device of claim 8, wherein the circuit is further configured to: receive second control information from the second information processing device to restore an original amount of the data in the PES payload; Storing the original amount of the data in the PES payload; multiplexing a third PES packet and the second PES packet to generate a second multiplex PES packet; and transmitting the second multiplex PES packet to the A second information processing device, the third PES packet includes the PES with the original amount of the data The payload and the data correspond to the sound signal. For example, the first information processing device of claim 8, wherein the circuit is further configured to: receive second control information from the second information processing device; and stop the first in the first multiplexed PES packet Transmission of a PES packet, wherein the transmission of the first PES packet is stopped based on the received second control information, the second control information is different from the first control information, and the PMT is lacking corresponding to the first The PID of a PES packet. For example, in the first information processing device of claim 11, the circuit is further configured to: receive third control information from the second information processing device to restore an original amount of the data in the PES payload; Storing the original amount of the data in the PES payload; multiplexing a third PES packet and the second PES packet to generate a second multiplex PES packet; and transmitting the second multiplex PES packet to the In a second information processing device, the third PES packet includes the PES payload with the original amount of the data, and the data corresponds to the sound signal. A communication system, including: A first information processing device including a first circuit, which is configured to: transmit an image signal and an audio signal based on a moving picture expert group (MPEG) 2 transmission stream (TS); 2. The information processing equipment receives control information; stops the output of data in a packetized elementary stream (PES) payload, where the output of the data is stopped based on the received control information, and a first PES packet includes the PES payload, a TS packet includes the first PES packet, the TS packet is specified by a packet identifier (PID), the PID specifies the transmitted audio signal in a program mapping table (PMT), and the PID is independent of the output of the data in the PES payload; multiplex processing the first PES packet and a second PES packet to generate a first multiplexed PES packet, wherein the second PES packet includes the image signal ; And transmitting the first multiplexed PES packet to the second information processing device; and the second information processing device including a second circuit, which is configured to: transmit the control information to the first information processing device to Reduce this The amount of sound signals; and controlling the first information processing device to stop the output of the data in the PES payload, where the output of the data is stopped based on the control information. An information processing method includes: in a first information processing device: receiving a first image signal and a first image signal from a second information processing device based on a moving picture expert group (MPEG) 2 transmission stream (TS) A first sound signal; receiving a second image signal and a second sound signal from a third information processing device; transmitting first control information to the second information processing device, wherein the first control information indicates the first sound Reduction in the amount of signals; sending second control information to the third information processing device, wherein the second control information indicates a reduction in the amount of the second sound signal; controlling the second information processing device or the third information processing At least one of the devices stops the output of data in at least one of the multiple packed basic stream (PES) payloads, wherein the output of the data is based on at least one of the first control information or the second control information Is stopped, each of the plural PES packets includes a corresponding PES payload of one of the plural PES payloads, and each of the plural PES payloads is encapsulated in a corresponding TS packet of one of the plural TS packets, Each of the plurality of TS packets is specified by a corresponding packet identifier (PID) of one of the plurality of packet identifiers (PID), and each of the plurality of PID is specified in a program map table (PMT) At least one of an audio signal or the second audio signal; and extracting a presentation time stamp (PTS) included in a PES header portion of each of the plurality of PES packets. An information processing method, including: in a first information processing device: based on a moving picture expert group (MPEG) 2 transmission stream (TS) to send an image signal and an audio signal to a second information processing Equipment; receiving control information from the second information processing equipment; stopping the output of data in a packetized elementary stream (PES) payload, where the output of the data is stopped based on the received control information, a first A PES packet includes the PES payload, a TS packet includes the first PES packet, the TS packet is indicated by a packet identifier (PID), and the PID specifies the transmitted in a program map table (PMT) The audio signal, and the PID are independent of the output of the data included in the PES payload; Multiplex processing the first PES packet and a second PES packet to generate a multiplexed PES packet, wherein the second PES packet includes the image signal; and transmitting the multiplexed PES packet to the second information processing device. A non-transitory computer-readable medium on which computer-executable instructions are stored, which when executed by a computer of a first information processing device, causes the computer to perform operations, such operations include: based on a dynamic image expert Group (MPEG) 2 transmission stream (TS) to receive a first image signal and a first audio signal from a second information processing device; a second image signal and a second from a third information processing device Sound signal; sending first control information to the second information processing device, wherein the first control information indicates a reduction in the amount of the first sound signal; sending second control information to the third information processing device, wherein the first The second control information indicates a reduction in the amount of the second sound signal; at least one of the second information processing device or the third information processing device is controlled to stop at least one of the multiple packetized elementary stream (PES) payload The output of the data in which the output of the data is stopped based on at least one of the first control information or the second control information, each of the plural PES packets includes a corresponding PES of the plural PES payload Payload, each of these plural PES payloads is encapsulated in plural TS packets In a corresponding TS packet, each of the plural TS packets is indicated by a corresponding packet identifier (PID) of the plural packet identifier (PID), and each of the plural PIDs is in a program mapping table (PMT) specifies at least one of the first sound signal or the second sound signal; and extracts a presentation time stamp (PTS) included in a PES header portion of each of the plural PES packets. A non-transitory computer-readable medium on which computer-executable instructions are stored, which when executed by a computer of a first information processing device, causes the computer to perform operations, such operations include: based on a dynamic image expert Group (MPEG) 2 transmission stream (TS) to send an image signal and an audio signal to a second information processing device; receiving control information from the second information processing device; stopping a packetized basic stream (PES) ) The output of the data in the payload, where the output of the data is stopped based on the received control information, a first PES packet includes the PES payload, and a TS packet includes the first PES packet, the TS packet It is indicated by a packet identifier (PID) which specifies the transmitted audio signal in a program map table (PMT), and The PID is independent of the output of the data in the PES payload; multiplex processing the first PES packet and a second PES packet to generate a multiplexed PES packet, wherein the second PES packet includes the image signal; And transmitting the multiplexed PES packet to the second information processing device.

Images