JP6057004B2 - Information processing apparatus, information processing method, and program - Google Patents

Description

  The present technology relates to an information processing apparatus. Specifically, the present invention relates to an information processing apparatus, an information processing method, and a program for exchanging various types of information using wireless communication.

  Conventionally, there is a wireless communication technology for exchanging various data using wireless communication. For example, an information exchange device that exchanges various types of data by short-range wireless communication between two wireless communication devices has been proposed (see, for example, Patent Document 1).

JP 2008-278388 A

  According to the above-described prior art, various types of data can be exchanged between two wireless communication devices by short-range wireless communication without being connected by a wired line. However, when various data are exchanged between the two wireless communication devices as described above, a delay due to short-range wireless communication may occur.

  Here, for example, in the information processing apparatus on the receiving side displaying the image based on the image data transmitted from the information processing apparatus on the transmission side on the display unit, the user performs an operation input for performing an operation related to the image. Assume a case. In this case, it is assumed that the time until the image based on the image data reflecting the operation input is displayed on the display unit is delayed by the delay. In this way, if the time until the image reflecting the operation input is displayed becomes long, the user cannot quickly grasp whether or not the operation input has been correctly received, and the operation input again. As such, there is a risk of misoperation.

  The present technology has been created in view of such a situation, and an object of the present technology is to allow a user to easily grasp a reception state of a user operation.

  The present technology has been made to solve the above-described problems. The first aspect of the present technology is that an image based on image data received from another information processing apparatus using short-range wireless communication is displayed on the display unit. An operation accepting unit that accepts a user operation related to the image when displayed, and notification information for notifying that the user operation is accepted when the user operation is accepted. An information processing apparatus including a control unit that performs control to transmit corresponding control data to the other information processing apparatus, an information processing method thereof, and a program that causes a computer to execute the method. Thus, when an image based on image data received from another information processing apparatus using short-range wireless communication is displayed on the display unit, the user operation regarding the image is accepted. The notification information for notifying that it has been accepted is output, and control data corresponding to the user operation is transmitted to another information processing apparatus.

  In this first aspect, the control unit outputs the notification information by displaying an additional image indicating that the user operation has been received over the image displayed on the display unit. May be. This brings about the effect that an additional image indicating that a user operation has been accepted is displayed superimposed on the image displayed on the display unit.

  In the first aspect, the operation receiving unit and the display unit are configured by a touch panel, the user operation is a contact operation or a proximity operation on the touch panel, and the control unit is a contact operation on the touch panel. Alternatively, when the proximity operation is accepted, the additional image may be displayed with reference to the position on the touch panel where the contact operation or the proximity operation is accepted. As a result, when a touch operation or a proximity operation is received on the touch panel, an additional image is displayed based on the position on the touch panel where the contact operation or the proximity operation is received.

  In the first aspect, when a pressing operation for pressing an operation image is received on the touch panel, the control unit displays a ripple image centered on a position on the touch panel where the pressing operation is received. When an operation for tracing the operation image is accepted on the touch panel, an image representing a locus on the touch panel for the tracing operation may be displayed as the additional image. As a result, when a pressing operation for pressing an operation image on the touch panel is accepted, a ripple image centered on the position on the touch panel where the pressing operation is accepted is displayed, and an operation for tracing the operation image on the touch panel is accepted. In such a case, an image representing the locus on the touch panel of the tracing operation is displayed.

  In the first aspect, the control unit may delete the additional image at a predetermined timing. This brings about the effect that the additional image is erased at a predetermined timing.

  In the first aspect, the control unit may determine the predetermined timing based on a delay time of the short-range wireless communication that occurs with the other information processing apparatus. This brings about the effect that the predetermined timing is determined based on the delay time of the short-range wireless communication that occurs with another information processing apparatus.

  In the first aspect, the control unit may set the predetermined timing as a timing at which data indicating that the control data has been received is received from the other information processing apparatus. This brings about the effect that the timing at which the data indicating that the control data has been received is received from another information processing apparatus is set as the predetermined timing.

  In the first aspect, the control unit obtains image data corresponding to image data identification information included in data indicating that the other information processing apparatus has received the control data from the other information processing apparatus. The reception timing or the timing for displaying an image based on the image data may be set as the predetermined timing. As a result, the timing at which the image data corresponding to the image data identification information included in the data indicating that the other information processing apparatus has received the control data is received from the other information processing apparatus, or an image based on the image data is displayed. This brings about the effect that the timing to perform is a predetermined timing.

  In the first aspect, the information processing device further includes a posture detection unit that detects a posture of the information processing device, and the user operation is an operation performed by changing the posture of the information processing device, and the control When the posture of the information processing apparatus is changed, the unit may output the notification information for notifying that the user operation based on the change has been received. As a result, when the attitude of the information processing apparatus is changed, the notification information for notifying that the user operation based on the change is accepted is output.

  In the first aspect, when the attitude of the information processing apparatus is changed, the control unit displays an additional image that visually represents the change superimposed on the image displayed on the display unit. By doing so, the notification information may be output. Thereby, when the attitude of the information processing apparatus is changed, an additional image that visually represents the change is displayed so as to be superimposed on the image displayed on the display unit.

  In addition, according to a second aspect of the present technology, an image based on image data received from the second information processing apparatus using short-range wireless communication is displayed on the first display unit, and a user operation regarding the image is accepted. In order to display the image on the first information processing apparatus using the short-range wireless communication and the first information processing apparatus that transmits control data corresponding to the user operation to the second information processing apparatus. When the control data is received from the first information processing apparatus, the data indicating that the control data has been received is displayed in the second display unit. The second image processing apparatus that transmits to the first information processing apparatus, and the image displayed on the first display unit is an additional image indicating that the first information processing apparatus has received the user operation. A first operation to be displayed in a superimposed manner, and a second operation to cause the second information processing apparatus to display an additional image representing that the control data has been received over the image displayed on the second display unit. A communication system in which at least one of them is performed, an information processing method thereof, and a program for causing a computer to execute the method. As a result, the first information processing apparatus displays the additional image indicating that the user operation has been received, superimposed on the image displayed on the first display unit, and the second information processing apparatus controls the control data. This brings about the effect that at least one of the second operation for displaying the additional image representing the acceptance of the image superimposed on the image displayed on the second display unit is performed.

  According to the present technology, it is possible to achieve an excellent effect that the user can easily grasp the reception state of the user operation.

It is a figure showing an example of composition of communications system 100 in a 1st embodiment of this art. It is a block diagram showing an example of functional composition of information processor 200 in a 1st embodiment of this art. It is a block diagram showing an example of functional composition of information processor 300 in a 1st embodiment of this art. It is a figure showing an example of a display transition of display part 350 when a user operates information processor 300 in a 1st embodiment of this art. It is a figure showing an example of a display transition of display part 350 when a user operates information processor 300 in a 1st embodiment of this art. It is a figure showing an example of a display transition of display part 350 when a user operates information processor 300 in a 1st embodiment of this art. It is a figure showing an example of a display transition of display part 350 when a user operates information processor 300 in a 1st embodiment of this art. It is a figure showing an example of a display transition of display part 350 when a user operates information processor 300 in a 1st embodiment of this art. It is a figure showing an example of a display transition of display part 350 when a user operates information processor 300 in a 1st embodiment of this art. 3 is a sequence chart illustrating an example of communication processing between devices included in the communication system 100 according to the first embodiment of the present technology. 7 is a flowchart illustrating an example of a processing procedure of data communication control processing by the information processing device 200 according to the first embodiment of the present technology. 7 is a flowchart illustrating an example of a processing procedure of a data communication control process performed by the information processing device 300 according to the first embodiment of the present technology. It is a sequence chart which shows the example of a communication process between each apparatus which comprises the communication system 100 in 2nd Embodiment of this technique. 14 is a flowchart illustrating an example of a processing procedure of a data communication control process performed by the information processing device 200 according to the second embodiment of the present technology. 22 is a flowchart illustrating an example of a processing procedure of a data communication control process performed by the information processing device 300 according to the second embodiment of the present technology. It is a figure which shows the example of a display of each information processing apparatus which comprises the communication system 100 in 3rd Embodiment of this technique. It is a block diagram showing an example of functional composition of information processor 201 in a modification of an embodiment of this art. It is a figure showing an example of composition of communications system 700 in a modification of an embodiment of this art. It is a figure showing an example of composition of communications system 800 in a modification of an embodiment of this art.

Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described. The description will be made in the following order.
1. First embodiment (display control: an example in which an additional image is displayed when a user operation is accepted and the additional image is erased based on a delay time)
2. Second Embodiment (Display Control: Example in which an additional image is displayed when a user operation is accepted, and the additional image is erased based on data from the information processing apparatus on the transmission side)
3. Third embodiment (display control: an example in which an additional image is displayed on the information processing apparatuses on the transmission side and the reception side)
4). Modification 5 Modified example

<1. First Embodiment>
[Configuration example of communication system]
FIG. 1 is a diagram illustrating a configuration example of a communication system 100 according to the first embodiment of the present technology. FIG. 1 shows an example of a system configuration when two information processing apparatuses (information processing apparatuses 200 and 300) are directly wirelessly connected.

  The communication system 100 includes information processing apparatuses 200 and 300. Each of the information processing apparatuses 200 and 300 has a short-range wireless communication function. These are, for example, wireless communication devices compliant with IEEE (Institute of Electrical and Electronics Engineers) 802.11 specifications. The information processing apparatuses 200 and 300 can exchange various types of information using the short-range wireless communication function.

  Here, for example, a wireless local area network (LAN) or Wi-Fi Direct can be used as the short-range wireless communication used in the communication system 100. Moreover, as near field communication used for the communication system 100, TDLS (Tunneled Direct Link Setup) and an ad hoc network can be used, for example. Moreover, as near field wireless AV (Audio Visual) transmission communication used in the communication system 100, for example, Wi-Fi CERTIFIED Miracast can be used. Wi-Fi CERTIFIED Miracast uses Wi-Fi Direct and TDLS technologies to transmit audio and display video to be played on one terminal to another terminal. This is a mirroring technology that outputs video data.

  For example, the communication system 100 is implement | achieved by applying UIBC (user interface back channel) to Wi-Fi CERTIFIED Miracast. UIBC is a protocol used for exchange between a host PC (host computer) and a USB (Universal Serial Bus), for example. In Wi-Fi CERTIFIED Miracast, UIBC is realized on TCP IP (Transmission Control Protocol / Internet Protocol). Note that VNC (Virtual Network Computing) may be applied instead of Wi-Fi CERTIFIED Miracast.

  Note that the information processing device 200 is, for example, a display device (for example, a television or a personal computer) having a wireless communication function or a portable information processing device (for example, a camera, a tablet terminal, or a smartphone). The information processing apparatus 300 is, for example, a portable information processing apparatus (for example, a tablet terminal or a smartphone) having a wireless communication function.

  Further, FIG. 1 shows a state where the communication system 100 is used while the user 10 holds the information processing apparatus 300 by hand. For example, the information processing apparatus 200 can transmit image data using near field communication, and the information processing apparatus 300 can receive and display the image data. Further, the user 10 can perform various operations on the information processing apparatus 300 held by hand.

[Configuration example of information processing device]
FIG. 2 is a block diagram illustrating a functional configuration example of the information processing device 200 according to the first embodiment of the present technology.

  The information processing apparatus 200 includes an antenna 210, a wireless communication unit 220, an encoder 230, an operation reception unit 240, a display unit 250, and a control unit 260.

  The wireless communication unit 220 transmits and receives information (for example, image data) to and from other information processing apparatuses (for example, the information processing apparatus 300) using the short-range wireless communication via the antenna 210. It is. For example, when transmission processing is performed, the image data (video data) output from the control unit 260 is compressed by the encoder 230, and the compressed image data is transmitted from the antenna 210 via the wireless communication unit 220. Is done.

  The encoder 230 compresses (encodes) image data to be transmitted, and outputs the compressed image data to the control unit 260. The encoder 230 may be realized by executing encoding by software, or may be realized by executing encoding by hardware. Note that the image data to be transmitted may be transmitted as raw data without being encoded by the encoder 230.

  The operation reception unit 240 is an operation reception unit that receives an operation input performed by the user, and outputs operation information corresponding to the received operation input to the control unit 260.

  The display unit 250 is a display unit that displays various types of information (images, games, etc.) based on the control of the control unit 260. As the display unit 250, for example, a display panel such as an organic EL (Electro Luminescence) panel or an LCD (Liquid Crystal Display) panel can be used. Note that the operation receiving unit 240 and the display unit 250 can be integrally configured using a touch panel that allows a user to input an operation by touching or approaching the finger with the display surface.

  The control unit 260 controls each unit of the information processing apparatus 200. For example, the control unit 260 performs control to transmit image data for displaying an image to the information processing apparatus 300. For example, when the control unit 260 receives control data corresponding to a user operation received by the information processing device 300 from the information processing device 300, the control unit 260 performs processing based on the received control data.

[Configuration example of information processing device]
FIG. 3 is a block diagram illustrating a functional configuration example of the information processing device 300 according to the first embodiment of the present technology.

  The information processing apparatus 300 includes an antenna 310, a wireless communication unit 320, a decoder 330, an operation reception unit 340, a display unit 350, an attitude detection unit 360, a delay time calculation unit 370, and a control unit 380. .

  The wireless communication unit 320 transmits and receives each piece of information (for example, image data) to and from another information processing apparatus (for example, the information processing apparatus 200) using short-range wireless communication. It is. For example, when reception processing is performed, the image data received by the antenna 310 is decoded by the decoder 330 via the wireless communication unit 320, and the decoded image data is supplied to the control unit 380. Then, the control unit 380 causes the display unit 350 to display an image corresponding to the decoded image data.

  The decoder 330 decodes (decodes) the image data output from the information processing apparatus 200, and outputs the decoded image data to the control unit 380. The decoder 330 may be realized by executing decoding by software, or may be realized by executing decoding by hardware. When the image data transmitted from the information processing apparatus 200 is raw data, the raw data is used without being decoded by the decoder 330.

  The operation accepting unit 340 is an operation accepting unit that accepts an operation input made by the user, and outputs operation information corresponding to the accepted operation input to the control unit 380. For example, when the display screen based on the image data received from the information processing apparatus 200 using short-range wireless communication is displayed on the display unit 350, the operation reception unit 340 receives a user operation regarding the display screen. The operation reception unit 340 is realized by, for example, a touch panel, a keyboard, and a mouse.

  The display unit 350 is a display unit that displays various types of information (images, games, etc.) based on the control of the control unit 380. The display unit 350 displays an image (for example, the display screen shown in FIGS. 4 to 9) based on the image data transmitted from the information processing apparatus 200, for example. As the display unit 350, for example, a display panel such as an organic EL panel or an LCD panel can be used. In the embodiment of the present technology, the operation reception unit 340 and the display unit 350 are integrally configured using a touch panel that allows a user to input an operation by touching or approaching the finger with the display surface. An example is shown.

  The posture detection unit 360 detects a change in posture of the information processing device 300 by detecting acceleration, movement, inclination, and the like of the information processing device 300, and controls the posture information regarding the detected posture change to the control unit 380. Output to. For example, a sensor that can detect rotation angles of three axes (for example, the X axis, the Y axis, and the Z axis) in the information processing apparatus 300 is preferably used as the posture detection unit 360. For example, an acceleration sensor that can detect the direction of acceleration, an inclination sensor that detects the inclination of the device, an angular velocity sensor that can detect rotational motion, and a gyro sensor can be used as the posture detection unit 360.

  The delay time calculation unit 370 calculates a delay time of short-range wireless communication that occurs with the information processing apparatus 200, and outputs the calculated delay time to the control unit 380. The method for calculating the delay time will be described in detail with reference to FIG.

  The control unit 380 controls each unit of the information processing apparatus 300. For example, the control unit 380 performs control for causing the display unit 350 to display a display screen based on the image data received from the information processing apparatus 200. In this case, when a user operation is accepted, the control unit 380 performs control to output notification information for notifying that the user operation has been accepted. As this notification information, for example, an additional image (for example, a ripple image 402 shown in b of FIG. 4) indicating that a user operation has been received is received from the display screen (information processing device 200) displayed on the display unit 350. (Display screen based on image data). In addition, when a user operation is accepted, the control unit 380 performs control to transmit control data corresponding to the user operation to the information processing apparatus 200.

  For example, the control unit 380 performs control to delete the additional image displayed on the display unit 350 at a predetermined timing. For example, the control unit 380 can determine the timing based on the delay time calculated by the delay time calculation unit 370.

[Display example of the information processing device on the receiving side]
4 to 9 are diagrams illustrating display transition examples of the display unit 350 when the user operates the information processing apparatus 300 according to the first embodiment of the present technology.

[Display example when pressing operation]
FIG. 4A shows a display example when the content list screen 400 is displayed on the display unit 350 of the information processing apparatus 300. The content list screen 400 is displayed based on the image data transmitted from the information processing apparatus 200. For example, on the content list screen 400, representative images representing content (for example, moving image content and still image content) are displayed for each content. Then, the user can use the desired content by performing a representative image selection operation (for example, a touch operation by the user) corresponding to the desired content on the content list screen 400.

  Here, when only the display screen based on the image data transmitted from the information processing apparatus 200 is displayed on the display unit 350, it is assumed that the user performs an operation input (user operation) while viewing the display screen. In this case, for example, when a user operation is performed on the content list screen 400, control data corresponding to the user operation is transmitted from the information processing device 300 to the information processing device 200. When the information processing apparatus 200 receives the control data, the information processing apparatus 200 performs a process according to the control data, and then displays an image for displaying the display screen after the process (a display screen reflecting the control data). Data is transmitted to the information processing apparatus 300.

  As described above, when a user operation is performed on the content list screen 400, after the information is exchanged between the information processing apparatus 300 and the information processing apparatus 200, a display screen reflecting the control data is displayed. The For this reason, it may take a long time to display the display screen reflecting the user operation. In such a case, although the user has performed a user operation on the content list screen 400, the user cannot quickly recognize whether the user operation has been correctly received, and the user operation is actually It becomes difficult to determine whether or not it has been accepted. In this case, it may be determined that the user operation has not been correctly received, and an erroneous operation such as inputting the operation again may be performed.

  Therefore, in the first embodiment of the present technology, when a user operation is performed on the display screen displayed on the display unit 350, an additional image (for example, FIG. The ripple image 402) shown in 4b of FIG. 4 is immediately superimposed on the display screen. Thereby, it is possible to quickly notify the user that the user operation has been accepted.

  FIG. 4B shows a display example immediately after a user operation is performed on the content list screen 400. This example shows a display example when a user operation for selecting the representative image 401 on the content list screen 400 is performed.

  As illustrated in FIG. 4B, when a user operation (for example, a pressing operation) for selecting the representative image 401 is performed on the content list screen 400, the control unit 380 displays the selected representative image. A ripple image (additional image) 402 is displayed around 401. Then, the control unit 380 transmits control data (including a pressed position and a pressed time) indicating that the representative image 401 has been selected to the information processing apparatus 200 on the transmission side. In this manner, immediately after the user operation for selecting the representative image 401 is performed, the ripple image 402 indicating that the user operation has been correctly received is displayed, so that the user can perform the operation performed by the user. It is possible to quickly and easily grasp that it has been accepted.

  Here, FIG. 4B shows an example in which a ripple pattern image 402 having a plurality of colors is displayed as an additional image, but a semi-transparent ripple image may be displayed as an additional image. Also, an image having another shape may be displayed as an additional image.

  4B shows an example in which a pressing operation is performed as a user operation. Similarly, when another pressing operation (for example, long pressing or double tapping) is performed, a ripple image is displayed as an additional image. You may do it. In this case, the color and shape of the ripple image may be changed according to the type of user operation. Further, for example, when a long press operation is performed as a user operation, the color and size may be changed according to the length of the press time. For other pressing operations, an image having another shape may be displayed as an additional image.

  Similarly, an additional image can be displayed when each operation other than the pressing operation (for example, a tracing operation and an enlargement / reduction operation) is performed. These display examples are shown in FIGS.

[Display example when tracing operation]
FIG. 5 a shows a display example when the operation screen 410 is displayed on the content list screen 400 shown in FIG. 4. The operation screen 410 is displayed based on image data transmitted from the information processing apparatus 200 in accordance with, for example, a predetermined timing or a user operation.

  On the operation screen 410, an operation area image 411 and an operation indicator 412 are displayed. The operation area image 411 is an operation area image for setting each process related to the content list screen 400 (for example, setting a display magnification) by moving the operation indicator 412.

  The operation indicator 412 can be moved by a user's touch operation or a tracing operation on the operation area image 411. For example, in a state where the display portion of the operation sign 412 in the operation area image 411 is touched with a finger, the user moves the finger to a desired position in the operation area image 411 so that the operation mark 412 is moved (scrolled). It can be carried out. Further, when the user performs an operation (touch operation) in which the finger touches a desired position in the operation area image 411, the operation marker 412 can be moved to the position touched by the finger.

  FIG. 5 b shows a display example immediately after a user operation is performed on the operation screen 410. This example shows a display example when a user operation for moving the operation marker 412 in the operation region image 411 from the left side to the right side is performed.

  As shown in FIG. 5 b, when a user operation (for example, a tracing operation) for moving the operation indicator 412 from the left side to the right side is performed on the operation screen 410, the control unit 380 Ripple images (additional images) 415 and 416 representing the last position are displayed. Further, the control unit 380 displays a trajectory image (additional image) 417 representing the trajectory of the movement. Then, the control unit 380 transmits control data (including the contact position and its trajectory) indicating that the tracing operation for moving the operation marker 412 from the left side to the right side is performed to the information processing apparatus 200 on the transmission side. As described above, immediately after the user operation for moving the operation marker 412 from the left side to the right side is performed, additional images (ripple images 415 and 416, a trajectory image 417) indicating that the user operation has been correctly received are displayed. indicate. Thereby, the user can grasp | ascertain quickly and easily that operation which he performed was received.

  As in FIG. 4b, instead of the additional images shown in FIG. 5b, an image in another display mode may be displayed as the additional image.

[Display example when performing reduction operation]
FIG. 6A shows a display example when the content display screen 420 is displayed. The content display screen 420 is, for example, a display screen (for example, a moving image or still image playback screen) that is displayed after the selection operation of the representative image 401 illustrated in FIG. 4 is performed, and is transmitted from the information processing apparatus 200. Displayed based on image data.

  Here, as indicated by arrows 421 and 422, a user operation for reducing the image displayed on the content display screen 420 (for example, an operation of pinching with two fingers of the hand 20) is performed. Assuming that In this case, as shown in FIG. 6b, arrow images (additional images) 425 and 426 corresponding to the user operation are displayed.

  FIG. 6 b shows a display example immediately after a user operation is performed on the content display screen 420. This example shows a display example when the user operation shown in FIG. 6A is performed. In FIG. 6b, the hand 20 is not shown for ease of explanation.

  As shown in FIG. 6 b, when a user operation for reducing the image displayed on the content display screen 420 is performed, the control unit 380 displays an arrow image (first and last positions). Additional images) 425 and 426 are displayed. Then, the control unit 380 transmits control data (including two contact positions and their trajectories) indicating that a user operation for reducing an image displayed on the content display screen 420 has been performed. To device 200.

  Here, immediately after the user operation is performed on the content display screen 420, the image data reflecting the user operation is not transmitted from the information processing apparatus 200. Therefore, as shown in FIG. 6b, arrow images (additional images) 425 and 426 are displayed on the display screen at the time of the user operation. That is, the arrow images (additional images) 425 and 426 are displayed in a state before the image reduction process corresponding to the user operation is performed. When the image data reflecting the user operation is transmitted from the information processing apparatus 200, a display screen on which a reduction process according to the user operation is performed is displayed.

  As described above, immediately after the user operation for reducing the image displayed on the content display screen 420 is performed, the arrow images (additional images) 425 and 426 indicating that the user operation has been correctly received are displayed. To do. Thereby, the user can grasp | ascertain quickly and easily that operation which he performed was received.

[Display example when zooming in]
FIG. 7 a shows a display example when the content display screen 420 is displayed. FIG. 7B shows a display example immediately after a user operation for enlarging the image displayed on the content display screen 420 is performed.

  The example illustrated in FIG. 7 is an example in which an enlargement operation is performed instead of the reduction operation in the example illustrated in FIG. 6, and thus detailed description thereof is omitted here. That is, in the example shown in FIG. 7, as indicated by arrows 431 and 432, a user operation for enlarging the image displayed on the content display screen 420 (for example, an operation of pinching with two fingers of the hand 20). This is an example of the case where (pinch)) is performed. In this case, as shown in FIG. 7b, arrow images (additional images) 435 and 436 corresponding to the user operation are displayed.

  As described above, when a contact operation (or proximity operation) is received on the display surface (touch panel) of the display unit 350, the control unit 380 receives the contact operation (or proximity operation). The additional image is displayed based on the position at. For example, as illustrated in FIG. 4B, when a pressing operation for pressing the representative image 401 (operation image) is received, the control unit 380 centers on the position on the display surface where the pressing operation is received. The ripple image 402 to be displayed is displayed as an additional image. Further, for example, as illustrated in FIGS. 5 to 7, when an operation for tracing an operation image on the display surface is received, the control unit 380 displays an image representing a locus on the display surface of the tracing operation as an additional image. Let The images representing the trajectories are, for example, ripple images 415 and 416, trajectory images 417 shown in FIG. 5b, arrow images 425 and 426 shown in FIG. 6b, and arrow images 435 and 436 shown in FIG. 7b. .

[Display example when operating by tilting the information processing device]
FIG. 8A shows a display example when the content display screen 440 is displayed. The content display screen 440 is, for example, a display screen that is displayed based on a user operation, and is displayed based on image data transmitted from the information processing apparatus 200. The content display screen 440 is, for example, a display screen for playing a game for operating the car 441 while avoiding obstacles (cow, stump) present on the road. For example, it is assumed that the vehicle 441 can be bent to the left by tilting the information processing device 300 to the left, and the vehicle 441 can be bent to the right by tilting the information processing device 300 to the right.

  Here, it is assumed that an operation of tilting the information processing apparatus 300 to the right (an operation of bending the vehicle 441 to the right) is performed. In this case, as shown in FIG. 8b, an arrow image (additional image) 442 corresponding to the user operation is displayed.

  FIG. 8 b shows a display example immediately after a user operation is performed on the content display screen 440. In this example, a display example in a case where an operation of tilting the information processing apparatus 300 to the right (an operation of bending the vehicle 441 to the right) is performed. In FIG. 8, the illustration of the user's hand is omitted for ease of explanation.

  As illustrated in b of FIG. 8, when an operation of tilting the information processing device 300 to the right (an operation of bending the vehicle 441 to the right) is performed, the control unit 380 outputs the posture information output from the posture detection unit 360. The operation can be detected based on (information on change in posture). As described above, when the operation is detected, the control unit 380 displays the arrow image (additional image) 442 indicating the direction of the inclination. The control unit 380 then transmits control data (including a tilt direction and a tilt angle) indicating that a user operation for tilting the information processing apparatus 300 to the right (a user operation for bending the vehicle 441 to the right) has been performed. To device 200.

  Here, immediately after the user operation is performed on the content display screen 440, the image data reflecting the user operation is not transmitted from the information processing apparatus 200. Therefore, as shown in FIG. 8b, an arrow image (additional image) 442 is displayed on the image at the time of the user operation. That is, the arrow image (additional image) 442 is displayed in a state before the vehicle 441 is moved according to the user operation. When the image data reflecting the user operation is transmitted from the information processing apparatus 200, a display screen in which the car 441 is moved according to the user operation is displayed.

  Thus, immediately after the operation of tilting the information processing apparatus 300 to the right (the operation of bending the car 441 to the right) is performed, an arrow image (additional image) 442 indicating that the user operation has been correctly received is displayed. . Thereby, the user can grasp | ascertain quickly and easily that operation which he performed was received.

[Display example when operation is performed by rotating the information processing device]
FIG. 9A shows a display example when the content display screen 450 is displayed. The content display screen 450 is, for example, a display screen that is displayed based on a user operation, and is displayed based on image data transmitted from the information processing apparatus 200. The content display screen 450 is a display screen for reproducing animation, for example. In addition, it is assumed that the vertically long state and the horizontally long state of the display unit 350 can be changed by rotating the information processing apparatus 300 by 90 degrees with the axis perpendicular to the display surface of the display unit 350 as a rotation axis.

  Here, as indicated by an arrow 451, an operation of rotating the information processing apparatus 300 by 90 degrees in the CW (ClockWise) direction is performed with the axis perpendicular to the display surface of the display unit 350 as the rotation axis. Assume a case. In this case, as shown in FIG. 9b, an arrow image (additional image) 452 corresponding to the user operation is displayed.

  FIG. 9B shows a display example immediately after a user operation is performed on the content display screen 450. This example shows a display example when an operation for rotating the information processing apparatus 300 by 90 degrees in the CW direction is performed. In FIG. 9, illustration of the user's hand is omitted for ease of explanation.

  As illustrated in b of FIG. 9, when an operation of rotating the information processing apparatus 300 by 90 degrees in the CW direction is performed, the control unit 380 displays posture information (posture change) output from the posture detection unit 360. The operation can be detected based on the information). As described above, when the operation is detected, the control unit 380 displays the arrow image (additional image) 452 indicating the direction of the rotation. Then, the control unit 380 transmits control data (including a rotation direction and a rotation angle) indicating that an operation for rotating the information processing apparatus 300 by 90 degrees in the CW direction is performed to the information processing apparatus 200 on the transmission side.

  Here, immediately after the user operation is performed on the content display screen 450, the image data reflecting the user operation is not transmitted from the information processing apparatus 200. Therefore, as shown in FIG. 9b, an arrow image (additional image) 452 is displayed on the image at the time of the user operation. That is, the arrow image (additional image) 452 is displayed in a state before the image is rotated according to the user operation. When the image data reflecting the user operation is transmitted from the information processing apparatus 200, a display screen in which the image is rotated in accordance with the user operation is displayed.

  As described above, immediately after the operation for rotating the information processing apparatus 300 by 90 degrees in the CW direction is performed, the additional image 452 indicating that the user operation is correctly received is displayed. Thereby, the user can grasp | ascertain quickly and easily that operation which he performed was received.

  As described above, when the posture of the information processing device 300 is changed, the control unit 380 has received a user operation based on the change (an operation performed by changing the posture of the information processing device 300). Control to output notification information for notification. In this case, for example, the control unit 380 superimposes additional images (an arrow image 442 shown in FIG. 8 and an arrow image 452 shown in FIG. 9) that visually represent the change on the image displayed on the display unit 350. By displaying the notification information, it is possible to output the notification information.

  As in FIG. 4b and FIG. 5b, instead of the additional images shown in FIG. 6b, FIG. 7b, FIG. 8b, and FIG. It may be displayed as an additional image. For example, the arrow images (additional images) 442 and 452 shown in FIG. 8b and FIG. 9b may be displayed lighter on the entire display screen and blinked.

[Example of communication]
FIG. 10 is a sequence chart illustrating an example of communication processing between devices included in the communication system 100 according to the first embodiment of the present technology. In this communication processing example, the communication processing in the case where an operation input by the user is performed is shown in a simplified manner.

  First, network connection processing is performed between the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side (601, 602). In other words, connection setup is performed between the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side (601, 602).

  Subsequently, in the communication between the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side, capability exchange processing for exchanging capabilities (for example, exchange processing of codec information and other supported data format information). Are performed (603, 604). In this capability exchange processing, for example, exchange of information indicating the compression format of image data, what image size the data is transmitted, and what control data can be transmitted is performed (603, 604). For example, when using Wi-Fi CERTIFIED Miracast, it is possible to exchange data using RTSP (Real Time Streaming Protocol) packets and determine the data format to be transmitted. Note that the capability may be determined in advance between the apparatuses, and the image data may be transmitted with the predetermined capability. In this case, capability exchange processing can be omitted.

  Subsequently, a codec (Video Codec) to be used is determined in each of the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side (605, 606). Subsequently, the transmission-side information processing device 200 generates image data corresponding to a display screen (UI screen) to be displayed on the reception-side information processing device 300 (607). Then, the information processing apparatus 200 on the transmission side encodes the generated image data and transmits it to the information processing apparatus 300 on the reception side (608, 609). For example, when using Wi-Fi CERTIFIED Miracast, image data is transmitted using RTP (Real-time Transport Protocol) data.

  When the image data transmitted from the information processing apparatus 200 on the transmission side is received (609), the information processing apparatus 300 on the reception side decodes the received image data and responds to the decoded image data. A display screen is displayed on the display unit 350 (610). For example, the content list screen 400 shown in FIG.

  Subsequently, the delay time calculation unit 370 of the information processing apparatus 300 on the reception side calculates an approximate delay time between the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side (611). For example, the delay time is calculated based on the RTT (Round Trip Time) value of the packet data exchanged between the devices and the buffer size of the information processing device 300 on the receiving side (611). Here, the RTT value of packet data is a value that means the time from when a signal is transmitted until the response is returned. Further, as the RTT value of the packet data, for example, the RTT of the packet data exchanged between the information processing apparatuses 200 and 300 in the communication of the capability exchange process (603, 604) or the image data communication process (608, 609). A value can be used.

Here, an example of a calculation method for calculating the delay time (estimated delay time) will be described. The delay time can be calculated using the following Equation 1.
Delay time = network delay + (packet arrival time−reproduction time of the packet) Equation 1

  Here, the network delay can be obtained using the RTT value of the packet data. Further, (packet arrival time−reproduction time of the packet) can be obtained using a buffer size provided in the information processing apparatus 300.

  Then, the information processing apparatus 300 on the receiving side holds the calculated delay time. Note that the value of the delay time may be updated every time a packet is transmitted / received, or the value calculated first may be used as a fixed value.

  Here, it is assumed that a user performs an operation input (user input) in a state where a display screen corresponding to the image data transmitted from the transmission-side information processing device 200 is displayed on the display unit 350. When the operation input is performed in this way (612), the information processing apparatus 300 on the reception side receives the operation input and displays the operation input on the display screen displayed on the display unit 350. An additional image (effect image) indicating acceptance is superimposed and displayed (613). That is, an additional image indicating that the operation input has been accepted is displayed on the display screen corresponding to the image data transmitted from the information processing apparatus 200 on the transmission side (overlay display). For example, as shown in FIG. 4b, when a pressing operation is accepted, a rippled image (ripple image 402) is displayed in an overlay manner. Further, as shown in FIG. 5b, when a scroll operation is accepted, additional images (ripple images 415, 416, trajectory image 417) representing the operation content are displayed in an overlay manner.

  Subsequently, the information processing apparatus 300 on the receiving side sets a timer corresponding to the held delay time (614). This timer is a timer that measures the timing of erasing the additional image. That is, the additional image continues to be displayed until the set timer expires, and the additional image is erased when the timer expires.

  Subsequently, the information processing apparatus 300 on the reception side transmits control data (user input data) corresponding to the operation input to the information processing apparatus 200 on the transmission side (615, 616). For example, the control data corresponding to the operation input is transmitted to the information processing apparatus 200 on the transmission side using the UIBC protocol (615, 616).

  In this example, the control data is transmitted (615, 616) after displaying the additional image (613). However, the additional image may be displayed after transmitting the control data.

  Further, for example, when the delay time held in the information processing apparatus 300 on the receiving side is shorter than the reference value, the additional image may not be displayed. In other words, only when a delay occurs to some extent, the additional image may be displayed to notify the user that the user operation has been correctly accepted.

  Further, the information processing apparatus 300 on the receiving side deletes the additional image at the timing when the set timer (timer corresponding to the delay time) ends (617). That is, the additional image is deleted after the calculated delay time has elapsed (617).

  Here, the additional image may be subjected to image processing such as fading out according to the drawing time. For example, it is possible to perform image processing so that the additional image becomes thinner as time elapses from the display timing of the additional image, and to erase the additional image when a predetermined time is reached.

  When control data transmitted from the information processing apparatus 300 on the reception side is received (616), the information processing apparatus 200 on the transmission side performs processing based on the received control data (618). For example, as shown in b of FIG. 4, when a pressing operation of the representative image 401 is accepted, a content display screen 420 (shown in FIGS. 6 and 7) corresponding to the representative image 401 is displayed on the information processing apparatus 300. The process for displaying is performed as a process based on the control data.

  Subsequently, the transmission-side information processing device 200 generates image data in which processing based on the control data is reflected (619). Subsequently, the information processing apparatus 200 on the transmission side encodes the generated image data and transmits the encoded image data to the information processing apparatus 300 on the reception side (620, 621). Further, the information processing apparatus 300 on the receiving side decodes the image data and causes the display unit 350 to display a display screen corresponding to the decoded image data (622).

  Here, it is assumed that immediately after an operation input by the user is performed, an additional image indicating that the operation input has been accepted is not displayed in an overlapping manner. In this case, the result of the processing based on the operation input by the user is reflected when the receiving information processing apparatus 300 receives and displays the image data reflecting the processing. However, a period (period from 612 to 622) from when an operation input by the user is performed until a result of processing based on the operation input is reflected depends on a network delay, a processing delay, and the like. There are many cases. And the longer this period, there is a possibility that the usability (ease of use) may decrease.

  In contrast, in the first embodiment of the present technology, immediately after an operation input by the user is performed, an additional image indicating that the operation input has been received is displayed in an overlapping manner (612 and 613). For this reason, it is possible to promptly notify the user that the user operation (command) is correctly accepted at the timing when the operation input by the user is performed.

[Operation example of information processing device]
FIG. 11 is a flowchart illustrating an example of a processing procedure of a data communication control process performed by the information processing device 200 according to the first embodiment of the present technology. In this example, short-range wireless communication is performed between the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side.

  First, the control unit 260 executes network connection processing with the information processing apparatus 300 on the reception side (step S901). In other words, connection setup is performed between the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side (step S901).

  Subsequently, the control unit 260 executes capability exchange processing for exchanging capabilities in communication with the information processing apparatus 300 on the reception side (step S902). Subsequently, the control unit 260 determines a codec (Video Codec) used for short-range wireless communication performed with the information processing apparatus 300 on the reception side (step S903).

  Subsequently, the control unit 260 generates image data corresponding to a display screen (UI screen) to be displayed on the information processing apparatus 300 on the reception side (step S904). Subsequently, the encoder 230 encodes the generated image data, and the control unit 260 transmits the encoded image data to the information processing apparatus 300 on the reception side (step S905).

  Subsequently, the control unit 260 determines whether or not control data corresponding to the operation input has been received from the information processing apparatus 200 on the transmission side (step S906). If control data corresponding to the operation input has not been received (step S906), the process returns to step S904. On the other hand, when control data corresponding to the operation input is received from the information processing apparatus 200 on the transmission side (step S906), the control unit 260 performs processing based on the received control data (step S907).

  Subsequently, the control unit 260 determines whether or not a network connection end instruction is accepted (step S908), and when the end instruction is accepted, ends the operation of the data communication control process. On the other hand, if the end instruction has not been accepted (step S908), the process returns to step S904. Then, the control unit 260 generates image data (image data reflecting processing based on the received control data) corresponding to a display screen (UI screen) to be displayed on the information processing apparatus 300 on the receiving side ( Step S904).

[Operation example of information processing device]
FIG. 12 is a flowchart illustrating an example of a processing procedure of a data communication control process by the information processing device 300 according to the first embodiment of the present technology. In this example, short-range wireless communication is performed between the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side.

  First, the control unit 380 performs network connection processing with the information processing apparatus 200 on the transmission side (step S911). That is, connection setup is performed between the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side (step S911).

  Subsequently, the control unit 380 performs capability exchange processing for exchanging capabilities in communication with the information processing apparatus 200 on the transmission side (step S912). Subsequently, the control unit 380 determines a codec (Video Codec) used for short-range wireless communication performed with the information processing apparatus 200 on the transmission side (step S913).

  Subsequently, the control unit 380 determines whether or not the image data generated by the information processing apparatus 200 on the transmission side has been received (step S914). If the image data has not been received, the control unit 380 proceeds to step S917. move on. On the other hand, when the image data is received (step S914), the decoder 330 decodes the received image data, and the control unit 380 displays a display screen corresponding to the decoded image data on the display unit 350. (Step S915). Note that step S915 is an example of a first procedure described in the claims.

  Subsequently, the control unit 380 calculates an approximate delay time between the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side (step S916). Then, the control unit 380 holds the calculated delay time.

  Subsequently, the control unit 380 determines whether or not an operation input by the user has been performed (step S917). If no operation input is performed by the user (step S917), the process proceeds to step S921. On the other hand, when an operation input is performed by the user (step S917), the control unit 380 accepts the operation input on the display screen displayed on the display unit 350 when the operation input is accepted. An additional image indicating this is superimposed and displayed (step S918). Step S918 is an example of the second procedure described in the claims.

  Subsequently, the control unit 380 sets a timer corresponding to the held delay time (step S919). Subsequently, the control unit 380 transmits control data corresponding to the operation input to the information processing apparatus 200 on the transmission side (step S920). Note that step S920 is an example of a third procedure described in the claims.

  Subsequently, the control unit 380 determines whether or not the set timer (timer corresponding to the delay time) has expired (step S921). If the set timer has not expired (step S921), the process proceeds to step S923. On the other hand, when the set timer has expired (step S921), the control unit 380 erases the additional image whose timer has expired (step S922). When a plurality of user operations are performed, an additional image is displayed for each of the plurality of user operations. In this case, a timer is set for each additional image, and the additional images for which the timer has expired are sequentially deleted.

  Subsequently, the control unit 380 determines whether or not a network connection end instruction has been received (step S923). If the end instruction is received, the operation of the data communication control process ends. On the other hand, if the end instruction has not been accepted (step S923), the process returns to step S914.

  In this example, the control data is transmitted (step S920) after the additional image is displayed (step S918). However, the additional image may be displayed after the control data is transmitted.

  As described above, in the embodiment of the present technology, when a user operation is performed on the display screen displayed on the display unit 350, an additional image indicating that the user operation is correctly received (in the user operation) The corresponding additional image) is immediately superimposed on the display screen. Thereby, it is possible to promptly notify the user that the user operation has been accepted.

  Accordingly, it is possible to visually feedback that the user operation has been accepted, and to notify the user of the acceptance of the user operation clearly and promptly, thus providing a comfortable operation feeling to the user. be able to. In addition, since it is possible to notify the user of acceptance of user operations clearly and promptly, erroneous operations can be prevented.

  Further, by dynamically changing the period during which the additional image is displayed according to the delay time, it is possible to prevent the display of the additional image from overlapping the image that has been processed based on the user operation. Thereby, it is possible to prevent the image that has been processed based on the user operation from becoming difficult to see. As described above, in the embodiment of the present technology, the user can easily grasp the reception state of the user operation, and the user feedback considering the delay of the remote operation can be realized.

<2. Second Embodiment>
In the first embodiment of the present technology, the example in which the information processing apparatus on the reception side calculates the communication delay time between the two information processing apparatuses and the additional image is deleted based on the delay time has been described. However, the information processing apparatus on the reception side may delete the additional image based on information from the information processing apparatus on the transmission side.

  Therefore, in the second embodiment of the present technology, an example in which the information processing apparatus on the reception side erases the additional image based on information from the information processing apparatus on the transmission side is shown. Note that the configuration of the communication system (information processing apparatus configuring the same) in the second embodiment of the present technology is substantially the same as the communication system 100 and the information processing apparatuses 200 and 300 illustrated in FIG. For this reason, portions common to the communication system 100 and the information processing devices 200 and 300 are denoted by the same reference numerals, and a part of these descriptions is omitted.

[Example of communication]
FIG. 13 is a sequence chart illustrating an example of communication processing between the devices included in the communication system 100 according to the second embodiment of the present technology. Since FIG. 13 is a modification of FIG. 10, parts common to FIG. 10 are denoted by the same reference numerals, and a part of the description is omitted.

  In this example, the delay time calculation (611) and the timer setting (614) shown in FIG. 10 are not performed.

  In addition, after performing processing based on the received control data (618), the transmission-side information processing device 200 transmits an Ack (ACKnowledgement) for the control data to the reception-side information processing device 300 (631, 632). . That is, the fact that the operation input corresponding to the received control data has been received is transmitted to the information processing apparatus 300 on the receiving side (631, 632).

  Further, the information processing apparatus 300 on the receiving side may delete the additional image at the timing (632) at which the Ack for the control data is received (633). However, since a delay occurs between the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side, the additional image is deleted at the timing when the image data reflecting the processing based on the control data is displayed. Is preferred.

  Therefore, for example, the information processing apparatus 200 on the transmission side describes the sequence number corresponding to the image data at the time when the control data is received in the Ack packet for the control data. Then, when the information processing apparatus 300 on the receiving side receives the image data corresponding to the sequence number (or when displaying the image data corresponding to the sequence number), the additional image may be deleted. Good (633). Here, for the Ack packet storing the sequence number, for example, a unique packet can be defined and used.

  As described above, when the control unit 260 of the information processing device 200 receives the control data corresponding to the user operation accepted by the information processing device 300 from the information processing device 300, the data indicating that the control data has been received. Is transmitted to the information processing apparatus 300. The data indicating that the control data has been received is, for example, Ack indicated by 631 and 632 in FIG.

  In addition, the control unit 380 of the information processing device 300 can set, for example, the timing at which the data indicating that the control data has been received from the information processing device 200 as the timing to delete the additional image. Further, the control unit 380 erases the additional image at the timing when the image data corresponding to the image data identification information included in the data indicating that the information processing apparatus 200 has received the control data is received from the information processing apparatus 200, for example. The timing can be. The control unit 380 also displays, for example, a timing for displaying an image based on the image data corresponding to the image data identification information included in the data indicating that the information processing apparatus 200 has received the control data, and a timing for deleting the additional image. It can be. The image data identification information is, for example, a sequence number for identifying image data.

[Operation example of information processing device]
FIG. 14 is a flowchart illustrating an example of a processing procedure of a data communication control process performed by the information processing device 200 according to the second embodiment of the present technology. Since FIG. 14 is a modification of FIG. 11, portions common to FIG. 11 are denoted by the same reference numerals, and a part of the description is omitted.

  After performing the process based on the received control data (step S907), the control unit 260 transmits Ack for the control data to the information processing apparatus 300 on the receiving side (step S931).

[Operation example of information processing device]
FIG. 15 is a flowchart illustrating an example of a processing procedure of a data communication control process by the information processing device 300 according to the second embodiment of the present technology. Since FIG. 15 is a modification of FIG. 12, portions common to FIG. 12 are denoted by the same reference numerals and description thereof is partially omitted.

  After transmitting the control data corresponding to the operation input (step S920), control unit 380 determines whether or not an Ack for the control data has been received (step S941). If an Ack for the control data has been received (step S941), the control unit 380 determines whether image data corresponding to the sequence number included in the Ack packet for the control data has been received (step S941). Step S942). When the image data corresponding to the sequence number is received (step S942), the control unit 380 deletes the additional image related to the Ack (step S943).

  If no Ack has been received for the control data (step S941), the process proceeds to step S923. If the Ack for the control data is received, but the image data corresponding to the sequence number included in the packet of the Ack is not received (step S942), the process proceeds to step S923.

  As described above, in the second embodiment of the present technology, it is possible to appropriately delete the additional image without calculating the communication delay time between the two information processing apparatuses.

<3. Third Embodiment>
In the first and second embodiments of the present technology, an example in which an additional image is displayed on the information processing apparatus on the reception side has been described. However, the additional image may be displayed on the information processing apparatus on the transmission side. Further, the additional image may be displayed on both the transmitting side and receiving side information processing apparatuses. Further, the additional image may be displayed on any one of the two information processing apparatuses on the transmission side and the reception side.

  Therefore, in the third embodiment of the present technology, an example in which an additional image is displayed on the two information processing devices on the transmission side and the reception side is shown. Note that the configuration of the communication system (information processing apparatus configuring the same) in the third embodiment of the present technology is substantially the same as the communication system 100 and the information processing apparatuses 200 and 300 illustrated in FIG. For this reason, portions common to the communication system 100 and the information processing devices 200 and 300 are denoted by the same reference numerals, and a part of these descriptions is omitted.

[Example of additional image display]
FIG. 16 is a diagram illustrating a display example of each information processing device configuring the communication system 100 according to the third embodiment of the present technology. The display example in the information processing apparatus 300 is the same as b in FIG. In addition, the same display screen as the display screen displayed on the display unit 350 of the information processing device 300 is displayed on the display unit 250 of the information processing device 200.

  Here, the control unit 260 of the information processing device 200 on the transmission side displays the same image (content list screen 405) as the image to be transmitted to the information processing device 300 on the reception side (content list screen 400). To display. Further, as described above, it is assumed that a user operation (for example, a pressing operation) for selecting the representative image 401 is performed on the content list screen 400. In this case, the control unit 380 of the information processing apparatus 300 on the reception side displays a ripple image (additional image) 402 around the selected representative image 401. The control unit 380 transmits control data indicating that the representative image 401 has been selected to the information processing apparatus 200 on the transmission side. When the control data is received, the control unit 260 of the transmission-side information processing device 200 adds a ripple image (added) to the content list screen 405 based on the control data (for example, the position and type of the user operation). Image) 406 is displayed. That is, on the content list screens 400 and 405, the same ripple images (additional images) 402 and 406 are displayed in an overlapping manner.

  Thus, the ripple image 402 is displayed immediately after the user operation for selecting the representative image 401 is performed, and the ripple image 406 is also displayed on the information processing apparatus 200 on the transmission side that has received the control data related to the user operation. Can be displayed. As a result, the user can clearly and quickly easily understand that the operation performed by the user has been accepted in both the information processing apparatus 200 on the transmission side and the information processing apparatus 300 on the reception side. it can.

  It should be noted that it is possible to set by user operation whether an additional image is displayed on both of the information processing devices on the transmission side and the reception side or whether the additional image is displayed on one of the two information processing devices. May be.

  Thus, the communication system 100 can be set to perform at least one of the first operation and the second operation. Here, the first operation is an operation in which the information processing apparatus 300 displays an additional image indicating that the user operation has been received, overlaid on the image displayed on the display unit 350. The second operation is an operation in which the information processing apparatus 200 displays an additional image indicating that the control data related to the user operation has been received so as to overlap the image displayed on the display unit 250.

<4. Modification>
FIG. 2 illustrates an example in which the information processing apparatus on the transmission side includes an operation reception unit and a display unit. However, the embodiment of the present technology can also be applied to a transmission-side information processing apparatus that does not include an operation reception unit and a display unit.

  Therefore, in this modification, an example of an information processing apparatus on the transmission side that does not include an operation reception unit and a display unit is shown.

[Configuration example of information processing device]
FIG. 17 is a block diagram illustrating a functional configuration example of the information processing apparatus 201 according to the modification of the embodiment of the present technology.

  The information processing apparatus 201 includes an antenna 210, a wireless communication unit 220, an encoder 230, and a control unit 260. Note that the information processing apparatus 201 is a modification of the information processing apparatus 200 illustrated in FIG. 2, and therefore, portions common to the information processing apparatus 200 are denoted by the same reference numerals and description thereof is omitted.

  As described above, the embodiment of the present technology can also be applied to a case where the information processing apparatus 201 that does not include the operation reception unit and the display unit is used as the information processing apparatus on the transmission side.

<5. Modification>
In the first to third embodiments of the present technology, an example in which two information processing apparatuses (the information processing apparatuses 200 and 300) are directly wirelessly connected has been described. However, other network configurations may be used.

  Therefore, in this modification, an example of another network configuration is shown. The configuration of the information processing apparatus in this modification is substantially the same as the example shown in FIGS. For this reason, a part of the description of the parts common to the first to third embodiments of the present technology is omitted.

[Configuration example of communication system]
FIG. 18 is a diagram illustrating a configuration example of a communication system 700 according to a modification example of the embodiment of the present technology. FIG. 18 illustrates an example of a system configuration when two information processing apparatuses (information processing apparatuses 710 and 720) are wirelessly connected via the router 730.

  The communication system 700 includes an information processing device 710, an information processing device 720, and a router 730. The information processing device 710 corresponds to the information processing device 200 illustrated in FIG. 1, and the information processing device 720 corresponds to the information processing device 300 illustrated in FIG. 1.

  18 illustrates an example in which two information processing apparatuses (information processing apparatuses 710 and 720) are wirelessly connected using the router 730, but the two information processing apparatuses are wirelessly connected using another relay device. It may be. For example, two information processing apparatuses (information processing apparatuses 710 and 720) may be wirelessly connected via an access point (wireless LAN access point).

[Configuration example of communication system]
FIG. 19 is a diagram illustrating a configuration example of a communication system 800 according to a modification of the embodiment of the present technology. FIG. 19 shows an example of a system configuration when two information processing apparatuses (information processing apparatuses 810 and 820) are wirelessly connected via the Internet 850.

  The communication system 800 includes an information processing device 810, an information processing device 820, routers 830 and 840, and the Internet 850. Note that the information processing apparatus 810 corresponds to the information processing apparatus 200 illustrated in FIG. 1, and the information processing apparatus 820 corresponds to the information processing apparatus 300 illustrated in FIG.

  FIG. 19 shows an example in which two information processing devices (information processing devices 810 and 820) are wirelessly connected via the Internet 850. However, two information processing devices (for example, a public line network) are connected via two networks. The information processing apparatus may be wirelessly connected.

  As shown in FIG. 19, it can be configured to control a video of a remote place via the Internet 850. As described above, the embodiment of the present technology can be realized without depending on the network path between the control terminal and the image output terminal.

  In the embodiment of the present technology, an example (an example in which an additional image is displayed) is output visually as notification information for notifying that a user operation has been received, but other notification information is output. You may do it. For example, the notification information may be audibly output by outputting a sound for notifying that a user operation has been accepted. Further, for example, in order to notify that a user operation has been accepted, the notification information may be output by vibrating the information processing apparatus (for example, vibration).

  Further, in the embodiments of the present technology, the display device having a wireless communication function and the portable information processing device have been described as examples. However, the embodiments of the present technology also apply to other information processing devices having a wireless communication function. Can be applied. For example, the present invention can be applied to information processing devices such as home game devices, portable video processing devices, portable game devices, digital still cameras, digital video cameras (for example, camera-integrated recorders), and digital photo frames. For example, the present invention can also be applied to an information processing apparatus (for example, a personal computer that does not have a wireless communication function) capable of performing wireless communication by mounting a wireless communication device having a wireless communication function.

  The above-described embodiment shows an example for embodying the present technology, and the matters in the embodiment and the invention-specific matters in the claims have a corresponding relationship. Similarly, the invention specific matter in the claims and the matter in the embodiment of the present technology having the same name as this have a corresponding relationship. However, the present technology is not limited to the embodiment, and can be embodied by making various modifications to the embodiment without departing from the gist thereof.

  Further, the processing procedure described in the above embodiment may be regarded as a method having a series of these procedures, and a program for causing a computer to execute these series of procedures or a recording medium storing the program. You may catch it. As this recording medium, for example, a CD (Compact Disc), an MD (MiniDisc), a DVD (Digital Versatile Disk), a memory card, a Blu-ray (registered trademark) Disc, or the like can be used.

In addition, this technique can also take the following structures.
(1)
An operation accepting unit that accepts a user operation regarding the image when an image based on image data received from another information processing apparatus using short-range wireless communication is displayed on the display unit;
A control unit that performs control to output notification information for notifying that the user operation has been received when the user operation is received and to transmit control data corresponding to the user operation to the other information processing apparatus An information processing apparatus comprising:
(2)
The information processing apparatus according to (1), wherein the control unit outputs the notification information by displaying an additional image indicating that the user operation has been received on an image displayed on the display unit.
(3)
The operation receiving unit and the display unit are configured by a touch panel,
The user operation is a contact operation or a proximity operation on the touch panel,
When the touch operation or the proximity operation is received on the touch panel, the control unit displays the additional image on the basis of the position on the touch panel at which the contact operation or the proximity operation is received. Information processing device.
(4)
When a pressing operation for pressing an operation image is received on the touch panel, the control unit displays a ripple image centered on a position on the touch panel where the pressing operation is received as the additional image. The information processing apparatus according to (3), wherein when an operation for tracing an operation image is received, an image representing a locus on the touch panel of the tracing operation is displayed as the additional image.
(5)
The information processing apparatus according to any one of (2) to (4), wherein the control unit deletes the additional image at a predetermined timing.
(6)
The information processing apparatus according to (5), wherein the control unit determines the predetermined timing based on a delay time of the short-range wireless communication that occurs with the other information processing apparatus.
(7)
The information processing apparatus according to (5), wherein the control unit uses the predetermined timing as a timing at which data indicating that the control data has been received is received from the other information processing apparatus.
(8)
The control unit is based on timing at which image data corresponding to image data identification information included in data indicating that the other information processing apparatus has received the control data is received from the other information processing apparatus or on the image data The information processing apparatus according to (5), wherein a timing for displaying an image is the predetermined timing.
(9)
Further comprising an attitude detection unit for detecting the attitude of the information processing apparatus;
The user operation is an operation performed by changing the posture of the information processing apparatus,
The control unit according to (1) or (2), wherein when the attitude of the information processing apparatus is changed, the notification information for notifying that the user operation based on the change is received is output. Information processing device.
(10)
When the attitude of the information processing apparatus is changed, the control unit outputs the notification information by displaying an additional image visually representing the change on the image displayed on the display unit. The information processing apparatus according to (9).
(11)
An image based on the image data received from the second information processing apparatus using short-range wireless communication is displayed on the first display unit, and control data corresponding to the user operation when a user operation related to the image is accepted A first information processing device that transmits the information to the second information processing device;
The first information processing apparatus transmits image data for displaying the image on the first information processing apparatus using the short-range wireless communication and displays the same image as the image on a second display unit. A second information processing device that transmits data indicating that the control data has been received to the first information processing device when the control data is received from
A first operation for displaying an additional image representing that the first information processing device has accepted the user operation superimposed on the image displayed on the first display unit; and the second information processing device, A communication system in which at least one of a second operation for displaying an additional image indicating that the control data has been received overlaid on the image displayed on the second display unit is performed.
(12)
A first procedure for displaying an image based on image data received from another information processing apparatus using short-range wireless communication on the display unit;
A second procedure for outputting notification information indicating that the user operation has been received when a user operation regarding the image displayed on the display unit is received;
And a third procedure for transmitting control data corresponding to the user operation to the other information processing apparatus.

10 users 20 hands 100, 700, 800 communication system 200, 201, 300, 710, 720, 810, 820 information processing device 210, 310 antenna 220, 320 wireless communication unit 230 encoder 240, 340 operation reception unit 250, 350 display unit 260, 380 Control unit 330 Decoder 360 Attitude detection unit 370 Delay time calculation unit 730, 830, 840 Router 850 Internet

Claims (22)

A touch panel for displaying an image based on image data received from another information processing apparatus and accepting a contact operation or a proximity operation as a user operation related to the displayed image;
When the user operation is accepted, an additional image representing the acceptance of the user operation is displayed on the image displayed on the touch panel based on the position on the touch panel where the user operation is accepted, An information processing apparatus comprising: a control unit that performs control to transmit control data corresponding to the user operation to the other information processing apparatus. The information processing apparatus according to claim 1 , wherein the touch panel receives at least one of a pressing operation for pressing an operation image, an operation for tracing the operation image, an enlargement operation, and a reduction operation . The information processing apparatus according to claim 2 , wherein when the pressing operation is received on the touch panel, the control unit displays a ripple image centered on a position on the touch panel where the pressing operation is received as the additional image. . The information processing apparatus according to claim 2, wherein when the tracing operation is received on the touch panel, the control unit displays an image representing a locus on the touch panel of the tracing operation as the additional image . The information processing apparatus according to claim 2, wherein when the tracing operation is received on the touch panel, the control unit causes the additional image to be displayed at an initial position and a final position of the tracing operation . The information processing apparatus according to claim 1, wherein the touch panel further displays an image based on image data processed by the other information processing apparatus based on the transmitted control data . The touch panel displays an image based on the image data received using Wi-Fi (Wireless Fidelity),
The control unit transmits the control data using Wi-Fi.
The information processing apparatus according to claim 1 . The touch panel displays an image based on the image data received using Wi-Fi Direct,
The control unit transmits the control data using Wi-Fi Direct.
The information processing apparatus according to claim 7 . The information processing apparatus according to claim 8, wherein the control unit performs capability exchange processing with the other information processing apparatus using an RTSP packet . The information processing apparatus according to claim 8, wherein the touch panel displays the image data transmitted from the other information processing apparatus using RTP data . An operation receiving unit that receives a user operation regarding the image when an image based on the image data received from another information processing apparatus is displayed on the display unit;
When the user operation is accepted, an additional image representing acceptance of the user operation is displayed on the image displayed on the display unit, and the displayed additional image is erased at a predetermined timing. A control unit that performs control to transmit control data corresponding to the user operation to the other information processing apparatus;
An information processing apparatus comprising: The information processing apparatus according to claim 11, wherein the control unit determines the predetermined timing based on a delay time of communication occurring with the other information processing apparatus.   The information processing apparatus according to claim 11, wherein the control unit uses the predetermined timing as a timing at which data indicating that the control data has been received is received from the other information processing apparatus.   The control unit receives the timing at which the image data corresponding to the image data identification information included in the data indicating that the other information processing apparatus has received the control data from the other information processing apparatus as the predetermined timing. The information processing apparatus according to claim 11.   The control unit receives image data corresponding to image data identification information included in data indicating that the other information processing apparatus has received the control data from the other information processing apparatus, and converts the received image data into the received image data. The information processing apparatus according to claim 11, wherein a timing for displaying a base image is the predetermined timing.   An attitude detection unit for detecting the attitude;
  An operation accepting unit that accepts, as a user operation related to the image, an operation performed by changing the posture when an image based on image data received from another information processing apparatus is displayed on the display unit;
  A control unit that performs control to output notification information for notifying that the user operation has been received when the user operation is received and to transmit control data corresponding to the user operation to the other information processing apparatus When
An information processing apparatus comprising:   When the attitude of the information processing apparatus is changed, the control unit outputs the notification information by displaying an additional image visually representing the change on the image displayed on the display unit. The information processing apparatus according to claim 16.   Image data for displaying an image is transmitted to another information processing apparatus, an image based on the transmitted image data is displayed on a touch panel of the other information processing apparatus, and a user operation relating to the displayed image is accepted. An additional image indicating that the user operation has been accepted is displayed over the image displayed on the touch panel based on the position at which the user operation is accepted on the touch panel, and then the other image is displayed. An information processing apparatus comprising a control unit that performs processing based on control data corresponding to the user operation transmitted from the information processing apparatus.   A first procedure for displaying an image based on image data received from another information processing apparatus on the touch panel;
  When a contact operation or a proximity operation is received as a user operation related to an image displayed on the touch panel, an additional image indicating that the user operation is received is based on the position on the touch panel where the user operation is received. A second procedure for displaying the image superimposed on the image displayed on the touch panel;
  A third procedure for transmitting control data corresponding to the user operation to the other information processing apparatus;
An information processing method comprising:   A transmission procedure for transmitting image data for displaying an image to another information processing apparatus;
  When an image based on the transmitted image data is displayed on the touch panel of the other information processing apparatus and a user operation related to the displayed image is received, an additional image indicating that the user operation is received is Processing based on control data corresponding to the user operation transmitted from the other information processing apparatus after being displayed superimposed on the image displayed on the touch panel based on the position where the user operation is received on the touch panel Control procedure and
An information processing method comprising:   A first procedure for displaying an image based on image data received from another information processing apparatus on the touch panel;
  When a contact operation or a proximity operation is received as a user operation related to an image displayed on the touch panel, an additional image indicating that the user operation is received is based on the position on the touch panel where the user operation is received. A second procedure for displaying the image superimposed on the image displayed on the touch panel;
  A third procedure for transmitting control data corresponding to the user operation to the other information processing apparatus;
A program that causes a computer to execute.   A transmission procedure for transmitting image data for displaying an image to another information processing apparatus;
  When an image based on the transmitted image data is displayed on the touch panel of the other information processing apparatus and a user operation related to the displayed image is received, an additional image indicating that the user operation is received is Processing based on control data corresponding to the user operation transmitted from the other information processing apparatus after being displayed superimposed on the image displayed on the touch panel based on the position where the user operation is received on the touch panel Control procedure and
A program that causes a computer to execute.

Images