JP2015060380A - Information terminal and information processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable, when a content is shared between terminals in an information processing system, real-time provision of a reaction of a user on the terminal having been supplied with the content to the terminal having supplied the content.SOLUTION: Three different states of an information terminal on the content transmission side are shown as information terminals 100A, 100B and 100C. Two different states of an information terminal on the content reception side are shown as information terminals 200A and 200B. When a user sees the information terminal 200B and reacts to it, reaction information indicating the details of the reaction is created and transmitted to the information terminal on the transmission side. On the information terminal 200C on the transmission side, the reaction information is output as an image 301.

Description

  The present invention relates to an information terminal that shares content with other terminals and an information processing system including the information terminal.

  2. Description of the Related Art Conventionally, in communication between two terminals, a technique for transmitting a user's reaction and emotion acquired by one terminal to the other terminal has been proposed. For example, Japanese Patent Application Laid-Open No. 2007-280074 (Patent Document 1) discloses an information terminal that transmits a mail receiver's reaction to a mail sender and a control method for controlling the information terminal. Japanese Patent Laid-Open No. 2003-271501 (Patent Document 2) discloses that a transmission destination terminal that receives transmission data transmitted from a transmission side terminal reproduces the transmission data, acquires reaction information, and the acquired reaction A technique for transmitting information to a transmitting terminal is disclosed. Japanese Patent Laying-Open No. 2003-110703 (Patent Document 3) discloses an information communication system that can acquire non-language information such as emotions of other users via communication terminals at various timings.

JP 2007-280074 A JP 2003-271501 A JP 2003-110703 A

  The technology described in Patent Document 1 acquires a recipient's face image from when the recipient displays an email until the end, creates user-related information using the face image, A reply mail containing information is automatically created and sent to the original sender. Therefore, in Patent Document 1, the original sender knows which mail the user-related information is a reaction to, but does not know which part of the mail is a reaction.

  In the technique described in Patent Document 2, transmission of information representing a reaction from a transmission destination terminal is performed after reproduction of transmission data at the terminal. Therefore, the information provided to the transmission source terminal is not a real-time reaction at the transmission destination terminal.

  In the technique described in Patent Document 3, non-linguistic information representing emotion is transmitted via a non-linguistic information management server. Therefore, a non-language information management server is required. In Patent Document 3, as shown in FIG. 10A and FIG. 10B, non-language information is not displayed together with contents transmitted and received between apparatuses. For this reason, it is difficult for the provided emotion to be provided in association with specific content such as content shared between devices.

  The present disclosure has been conceived in view of such a situation, and the purpose of the present disclosure is in real-time when a content is shared between terminals in an information processing system so that a user's reaction at the terminal to which the content is supplied is performed in real time. It is to be given to the terminal on the supply side.

  According to an aspect, a communication unit for receiving content data from another terminal, a playback unit for playing back content based on data received from another terminal, and the content are played back in the playback unit An information terminal is provided that includes a detection unit for detecting a user's reaction sometimes and a generation unit for generating reaction information that is information indicating the content of the reaction based on the reaction detected by the detection unit. The The communication unit transmits the reaction information to another terminal.

  Preferably, the information terminal further includes an input unit for receiving information input. The generation unit generates reaction information in response to information input to the input unit.

  According to another aspect, an information terminal including a reproducing unit for reproducing content and a communication unit for transmitting the content reproduced by the reproducing unit to another terminal is provided. The communication unit receives reaction information that is information indicating details of a reaction at another terminal with respect to the content. The playback unit plays back the reaction information together with the content.

  Preferably, the information terminal further includes an input unit for inputting timing information that is information for designating a timing for requesting a reaction to the content. The communication unit receives reaction information acquired by another terminal at a timing specified by the timing information.

  According to yet another aspect, an information processing system is provided that includes a first information terminal including a first reproduction unit that reproduces content, and a second information terminal that can communicate with the first information terminal. The first information terminal further includes a first communication unit for transmitting content reproduced in the first reproduction unit to the second information terminal. The second information terminal includes a second reproduction unit for reproducing the content received from the first information terminal, and a user in the second information terminal when the content is being reproduced in the second reproduction unit. A detecting unit for detecting the reaction, a generating unit for generating reaction information based on the reaction detected by the detecting unit, and a second communication unit for transmitting the reaction information to the first information terminal Including. The first communication unit receives reaction information from the second information terminal. The first playback unit plays back the reaction information together with the content.

  According to the present disclosure, when content is shared in an information processing system, a reaction detected during content reproduction in a supply destination information terminal is sent to the supply source information terminal. Thereby, in the information processing system, the reaction of the user at the terminal supplied with the content is given in real time to the terminal supplied with the content.

1 is a diagram schematically illustrating a configuration of an information processing system. It is a figure for demonstrating the operation | movement outline | summary of the information terminal in an information processing system. It is a figure which shows the specific example of the hardware constitutions of an information terminal. It is a figure which shows the specific example of the hardware constitutions of an information terminal. It is a figure for demonstrating specification of a facial expression. It is a figure which shows the example of a storage of the information in a reaction information database. It is a figure which shows the sequence of the process performed in an information processing system. It is a flowchart of the process performed when a content is shared in an information processing system. It is a figure which shows the modification of the display mode of reaction information.

  Hereinafter, embodiments of an information terminal and an information processing system according to the present disclosure will be described with reference to the drawings. In the following description, parts having the same function and action are denoted by the same reference numerals throughout the drawings, and redundant description will not be repeated.

[First Embodiment]
<System configuration>
FIG. 1 is a diagram schematically showing the configuration of an information processing system. The information processing system includes an information terminal 100 and an information terminal 200. The information terminal 100 and the information terminal 200 share content. In this specification, the information terminal 100 will be mainly described as a content supply side, and the information terminal 200 will be described as a content supply side. The information terminal 100 and the information terminal 200 communicate via the network 500. In FIG. 1, functional configurations of the information terminals 100 and 200 are described, but each function will be described later.

<Overview of operation>
FIG. 2 is a diagram for explaining an outline of operations of the information terminals 100 and 200 in the information processing system. The information terminal 100 includes a display 15. The information terminal 200 includes a display 25. In FIG. 2, three different states of the information terminal 100 are shown as information terminals 100A, 100B, and 100C. Also, two different states of the information terminal 200 are shown as information terminals 200A and 200B.

  In the information processing system, as shown as the information terminal 100A, the content is played back on the terminal on the transmission side. The information terminal 100 </ b> A transmits the content to the information terminal 200 in order to share the content with the information terminal 200. As a result, as shown as the information terminal 200A, the content is also reproduced at the receiving terminal. Since the content is shared between the information terminal 100 and the information terminal 200, when the reproduction content in the information terminal 100 is changed from the information terminal 100A to the information terminal 100B, the reproduction content in the information terminal 200 is also changed to the information terminal 200A. To the information terminal 200B.

  It is assumed that the user of the information terminal 200 takes a surprising reaction by changing the facial expression or making a voice while watching the display as shown on the information terminal 200B. The information terminal 200 detects such a reaction based on, for example, an image acquired by a camera or a sound acquired by a microphone. And the information terminal 200 produces | generates the reaction information which is the information which shows the content of a user's reaction based on the said detection result, and transmits it to the information terminal 100. FIG.

  When the reaction information is received from the information terminal 200, the information terminal 100 outputs the reaction information. In FIG. 2, the information terminal 100 </ b> C displays an image 301 corresponding to the detected reaction as reaction information on the display 15 together with the content displayed up to the information terminal 100 </ b> B.

<Hardware configuration>
(Information terminal 100)
FIG. 3 is a diagram illustrating a specific example of the hardware configuration of the information terminal 100. As shown in FIG. 3, the information terminal 100 includes a CPU (Central Processing Unit) 10 that is an arithmetic device, a ROM (Read Only Memory) 11 for storing a program (application) executed by the CPU 10, and the CPU 10. A RAM (Random Access Memory) 12 that functions as a work area when the program is executed, a memory 13 for storing contents and the like, and an operation I / F (interface) 14 for receiving an operation input to the information terminal 100 A display 15 as an image output unit, a speaker 16 as an audio output unit, a communication I / F 17 for performing wireless communication with an external device such as the information terminal 200, a camera 18, a microphone (hereinafter referred to as "microphone"). 19).

(Information terminal 200)
FIG. 4 is a diagram illustrating a specific example of the hardware configuration of the information terminal 200. As shown in FIG. 4, the information terminal 200 is similar to the information terminal 100, the CPU 20 that is an arithmetic device, the ROM 21 for storing a program (application) executed by the CPU 20, and the CPU 20 executes the program. RAM 22 that functions as a work area when performing the operation, a memory 23 for storing contents, an operation I / F 24 for receiving an operation input to the information terminal 200, a display 25 as an image output unit, and an audio output unit A communication I / F 27 for performing wireless communication with an external device such as the information terminal 100, a camera 28, and a microphone 29. For example, the camera 28 is provided at a position where the user's face looking at the display 25 can be photographed.

<Functional configuration>
(Information terminal 100)
The functional configuration of the information terminal 100 will be described with reference to FIG. In the information terminal 100, the operation input unit 102 accepts input of information from the outside. The display unit 104 displays a video (image). The content DB (database) 110 stores content files. The content playback unit 106 plays back the content stored in the content DB 110. The communication unit 120 includes an antenna 122 and transmits a content file (data) to the information terminal 200 via the network 500. The operation input unit 102 is realized by the operation I / F 14. The display unit 104 is realized by the display 15, the communication unit 120 is realized by the communication I / F 17, the content reproduction unit 106 is realized by the CPU 10, and the content DB 110 is realized by the memory 13.

  As will be described later, when the information terminal 200 receives content from the information terminal 100, the information terminal 200 generates reaction information while reproducing the content, and transmits the reaction information to the information terminal 100. Upon receiving the reaction information, the content reproduction unit 106 outputs the reaction information while reproducing the content.

(Information terminal 200)
The functional configuration of the information terminal 200 will be described with reference to FIG. In the information terminal 200, the operation input unit 202 receives input of information from the outside. The communication unit 220 includes an antenna 222 and receives a content file transmitted from the information terminal 100. The display unit 204 displays a playback image of the content. An A / D (analog / digital) converter 206 converts analog video data input to the camera 28 into digital data, and converts audio analog data input to the microphone 29 into digital data. Then, the A / D converter 206 outputs the converted digital data to the reaction detection unit 210.

  The reaction detection unit 210 detects a reaction by the user of the information terminal 200 based on the digital data input from the A / D converter 206. Then, when detecting the reaction, the reaction detection unit 210 outputs the input digital data to the facial expression recognition unit 212.

  As an example of “detecting a reaction”, it is detected that there is a change in the user's facial expression based on two or more images captured by the camera 28, and the user detected in the image captured by the camera 28 For example, the fact that the voice acquired by the microphone 29 includes the voice of the user is detected. The reaction detection unit 210 can detect only the image captured by the camera 28, only the sound input to the microphone 29, or both the image and the sound. The presence / absence of the reaction can be detected not only based on the image and sound but also based on the change in temperature and the presence / absence of vibration. For example, the presence / absence of a reaction based on a change in temperature can be determined based on detection of a hand temperature by a temperature sensor, detection of a face temperature by camera photography, or the like. The presence / absence of vibration can be determined based on, for example, whether the face position is identified based on a technique such as pattern recognition from an image captured by a camera and whether the face position is displaced in a specific pattern. . Detection of “change in facial expression” can be realized based on, for example, known techniques (chase focus, smile shutter, etc.).

  FIG. 5 is a diagram for explaining identification of facial expressions. Referring to FIG. 5, the reaction detection unit 210 detects a face using, for example, chase focus technology. Thereby, for example, the region 401 is extracted as a region representing a face from the image 400 of FIG. Then, the reaction detection unit 210 identifies the position of the part (eyes, nose, mouth) in the face from among the detected faces, identifies the degree of opening of the eyes, the degree of lowering of the eyes, and the like. Used for discrimination. In addition, the reaction detection unit 210 may use the degree of opening of the mouth, the degree of rising of the mouth corner, or the like for determination. In addition, the reaction detection unit 210 comprehensively specifies the positional relationship between two or more parts in the face, and comprehensively specifies facial expressions such as laughter and surprise.

  In the detection of reaction based on voice, the reaction detection unit 210 refers to a database storing acoustic models and language models corresponding to, for example, laughter, surprise, sadness, praise, and the like. The voice data input to the microphone 29 or the data from which the voice uttered by the user is separated) is matched with the model data. And the reaction detection part 210 outputs the reaction corresponding to model data with a high matching score as a result. The reaction detection unit 210 may identify a language (word) that is uttered by analyzing a voice uttered by a user in detecting a reaction, and may use the identified language itself or analyze the voice. By doing so, the tone (inflection etc.) of the user's voice may be specified and the tone of the voice may be used.

  In the information terminal 200, the camera 28 continuously captures the user's face, and the captured data can be continuously input to the reaction detection unit 210. Alternatively, in the information terminal 200, the microphone 29 can continuously generate voice data, and the generated voice data can be continuously input to the reaction detection unit 210.

  The facial expression recognition unit 212 outputs, to the emotion estimation unit 214, information specifying a pattern to which the input digital data corresponds from two or more patterns of reactions registered in advance. The facial expression recognition unit 212 determines, for example, whether the image captured by the camera 28 corresponds to a reaction pattern (for example, a smile, an angry face, a crying face, a troubled face, etc.) registered in advance, The result is output to emotion estimation unit 214. Further, the facial expression recognition unit 212 corresponds to any of the reaction patterns (for example, smile, angry face, crying face, troubled face, etc.) registered in advance in the voice input to the microphone 29. Whether or not to do so, and outputs the result to emotion estimation section 214. The facial expression recognition unit 212 may output either one of the reaction pattern for the image and the reaction pattern for the sound, or may output both.

  The emotion estimation unit 214 identifies the type of reaction of the user of the information terminal 200 based on the reaction pattern output from the facial expression recognition unit 212. The type of reaction specified is, for example, “laughter”, “surprise”, “sadness”, or “praise”. The information terminal 200 stores, for example, a table that associates the reaction pattern output from the facial expression recognition unit 212 with the specified reaction type. The emotion estimation unit 214 identifies the type of user reaction corresponding to the pattern output from the facial expression recognition unit 212 by referring to the table. In the table, a combination of an image pattern and an audio pattern output from the facial expression recognition unit 212 may be associated with a reaction type.

  The reaction information DB 216 stores the type of reaction in association with the reaction information. FIG. 6 is a diagram schematically illustrating a storage mode of information in the reaction information DB 216. As shown in FIG. 6, in the reaction information DB 216, an image, a character, a pictograph, and a sound are associated with each type of reaction. For example, the type “laughter” is associated with a sun image, characters “Nico Nico” and “hahaha”, a pictograph representing a smile, and audio information representing a laughter.

  The emotion estimation unit 214 outputs image, character, pictogram, and / or sound information associated with the identified reaction type in the reaction information DB 216 to the reaction information generation unit 218.

  The reaction information generation unit 218 uses the information output from the emotion estimation unit 214 to generate reaction information to be transmitted to the information terminal 100. More specifically, reaction information is generated so that at least one of the information output from the emotion estimation unit 214 is output at the information terminal 100. Then, the reaction information generation unit 218 transmits the reaction information to the information terminal 100 via the communication unit 220.

  As described above, when receiving the content from the information terminal 100, the information terminal 200 detects the user's reaction using the camera 28 and / or the microphone 29 while reproducing the content. When the reaction is detected, the information terminal 200 specifies the type of the reaction, generates reaction information for outputting information associated with the type specified in the information terminal 100, and the reaction Information is transmitted to the information terminal 100.

  In the information terminal 200, the A / D converter 206, the reaction detection unit 210, the facial expression recognition unit 212, the emotion estimation unit 214, and the reaction information generation unit 218 are realized by the CPU 20. In the information terminal 200, the operation input unit 202 is realized by the input I / F 24, the display unit 204 is realized by the display 25, and the communication unit 220 is realized by the communication I / F 27.

<Processing sequence>
Next, a sequence of processing executed in the information processing system will be described. FIG. 7 is a diagram illustrating a sequence of processes executed by the information terminal 100 and the information terminal 200.

  When the user of the information terminal 100 inputs an instruction to reproduce the content while sharing the content with the information terminal 200 to the operation I / F 14, the information terminal 100 inquires the information terminal 200 whether the content can be shared. (S1: Content sharing inquiry).

  When the information terminal 200 receives the inquiry of S1, the information terminal 200 displays on the display 25 a message that prompts the user to input whether the content can be shared. In response to this, the user of the information terminal 200 inputs information on whether or not to accept content sharing to the operation I / F 24. Then, the information terminal 200 transmits the input information to the information terminal 100 (S2: reply). Here, a case where information for accepting content sharing is input will be described.

  Upon receiving a response from the information terminal 200, the information terminal 100 transmits the content data to the information terminal 200 while reproducing the content on the information terminal 100 (S3: content sharing start, S4, S5: content data). . In response to this, the information terminal 200 reproduces the content that has received the data.

  In the information terminal 200, when a reaction is detected during content reproduction, reaction information is generated and transmitted to the information terminal 100 (S6: reaction detection / reaction information generation, S7: reaction information transmission). In response to this, the information terminal 100 reproduces the content and outputs reaction information (S8: reaction output).

  The information terminal 100 transmits content data to the information terminal 200 until an instruction to end content sharing is input or until the content instructed to share ends (S9: content data). . In the information terminal 200, the reproduction of the content is continued. During the reproduction of the content, when the user's reaction is detected in the information terminal 200, reaction information is generated and the reaction information is transmitted to the information terminal 100 (S10: reaction detection / reaction information generation, S11: reaction information). Send). And the information terminal 100 will output the said reaction information, if reaction information is received (S12: reaction output).

  When content sharing ends, the information terminal 100 notifies the information terminal 200 to that effect (S13: content data end notification). In response to this, the information terminal 200 notifies the information terminal 100 that the notification has been received (S14: reply).

<Flowchart>
FIG. 8 is a flowchart of processing executed in the information terminal 100 and the information terminal 200 when content is shared in the information processing system. In FIG. 8, the flowchart of the process in the information terminal 100 is shown on the left side, and the flowchart of the process in the information terminal 200 is shown on the right side. Hereinafter, with reference to FIG. 8, the content of the process performed in the information terminal 100 and the information terminal 200 is demonstrated.

  In the information terminal 100, when a content sharing instruction is input to the input I / F 14, in step S102, the CPU 10 instructs the information terminal 200 to share the content as described in S1 of FIG. Send information to inquire about availability.

  In the information terminal 200, in step S202, the CPU 20 receives the inquiry. Then, control proceeds to step S204.

  In step S <b> 204, the CPU 20 prompts the user to input an instruction as to whether or not sharing of content with the information terminal 100 is permitted by displaying a message on the display 25. And CPU20 will advance control to step S208, if the instruction | indication which permits sharing is input (it is YES at step S204). On the other hand, when an instruction not to permit sharing is input (NO in step S204), CPU 20 advances the control to step S206.

  In step S206, CPU20 transmits the information which refuses sharing with respect to the information terminal 100, and complete | finishes a process.

  In step S <b> 208, the CPU 20 transmits information permitting sharing to the information terminal 100. And control is advanced to step S210.

  In step S <b> 210, the CPU 20 starts shooting with the camera 28 and sound collection with the microphone 29.

  On the other hand, in the information terminal 100, in step S104, the CPU 10 receives the information transmitted from the information terminal 200 in step S206 or step S208. Then, the control proceeds to step S106.

  In step S106, the CPU 10 determines whether or not the information received from the information terminal 200 in step S104 permits content sharing. If the CPU 10 determines that the information permits content sharing, the CPU 10 advances the control to step S110 (YES in step S106). On the other hand, when CPU 10 determines that the information does not permit sharing of the content, control proceeds to step S108 (NO in step S106).

  In step S108, the CPU 10 performs control for canceling the content sharing instruction input from the user at the start of the process shown in FIG. 8 without transmitting the content data to the information terminal 200. Then, the process is terminated.

  In step S <b> 110, the CPU 10 transmits content data to the information terminal 200. This corresponds to S4, S5 and S9 in FIG.

  In the information terminal 200, as described above, after information for permitting content sharing is transmitted to the information terminal 100 in step S208, shooting by the camera 28 is started in step S210, and sound collection by the microphone 29 is performed. Has been started. In step S110, content data is transmitted from the information terminal 100 to the information terminal 200. In step S212, the CPU 20 receives content data transmitted from the information terminal 100. Then, the CPU 20 reproduces the content according to the data. Then, the control proceeds to step S214.

  In step S214, the CPU 20 determines whether or not a reaction has been detected. The detection of the reaction is realized, for example, according to the contents already described as the function of the reaction detection unit 210. When CPU 20 determines that a reaction has been detected, CPU 20 advances the control to step S216 (YES in step S214). On the other hand, if CPU 20 determines that no reaction has been detected, it returns control to step S212 (NO in step S214).

  In step S216, the CPU 20 generates reaction information. The generation of the reaction information is realized, for example, according to the contents already described as the function of the reaction information generation unit 218. Step S216 corresponds to S6 and S10 in FIG. Then, control proceeds to step S218.

  In step S218, the CPU 20 transmits the reaction information generated in step S216 to the information terminal 100. Step S218 corresponds to S7 and S11 in FIG. Then, the control proceeds to step S220.

  In the information terminal 100, in step S112, the CPU 10 determines whether or not reaction information has been received. When CPU 10 determines that the reaction information has been received, CPU 10 advances the control to step S114 (YES in step S112). On the other hand, when determining that the reaction information has not been received, the CPU 10 returns the control to step S110 (NO in step S112).

  In step S114, the CPU 10 outputs the received reaction information. The output mode of the reaction information is shown as an information terminal 100C in FIG. 2, for example. In the information terminal 100 </ b> C, an image 301 corresponding to the reaction information is displayed along with the content in the display 15.

  When the condition for ending content sharing is established in the information terminal 100, the CPU 10 executes control for ending content sharing in step S116. Thereby, the process ends. The condition is satisfied, for example, when an instruction to end the supply of content is input at the information terminal 100 or the information terminal 200, or when the content being reproduced is ended. The control for terminating the content supply includes, for example, notification to the information terminal 200 as described as S13 in FIG.

  In the information terminal 200, after the reaction information is transmitted in step S218, in step S220, the CPU 20 determines whether or not the reproduction of the content that has received data from the information terminal 100 is finished. This determination is realized, for example, by determining whether or not the information terminal 100 has received a notification to end content sharing as described in S13 of FIG. When CPU 20 determines that the reproduction of the content has ended, it proceeds to step S222 (YES in step S220). On the other hand, if the CPU 20 determines that the reproduction of the content has not ended yet, the CPU 20 returns the control to step S212 (NO in step S220).

  In step S222, the CPU 20 ends the shooting by the camera 28 and the sound collection by the microphone 29, which are started in step S210. Thereby, the process ends.

<Effect of Embodiment>
According to the present embodiment described above, when content is shared between information terminals, the reaction on the side that received the content is sent in real time to the side that supplied the content. As a result, communication between the users of the information terminals sharing the content can be performed more smoothly. In the information terminal that receives the supply of the content, the reaction of the user is detected using a camera and / or a microphone. Therefore, the information terminal that has received the content can return a reaction to the information terminal that has supplied the content without the user actively inputting information during the reproduction of the content. In this way, since the information terminal on the supply side does not require an active operation to transmit the reaction, the reaction can be returned to the information terminal on the supply side even when the content is displayed on the full screen. .

  In the information processing system of the present embodiment, content is directly shared between information terminals, and reaction information is transmitted and received. Thereby, it is not necessary to provide a special server such as a non-language information management server.

<Modification>
In the information processing system of this embodiment, the reaction information DB 216 can be stored on a server with which the information terminal 200 or the information terminal 100 can communicate. The information terminal 200 can generate reaction information by acquiring an image or the like corresponding to the detected reaction type from the server. Alternatively, the reaction information includes information specifying an image associated with the type of reaction detected in the information terminal 100, and the information terminal 100 that has received the reaction information obtains an image associated with the type. And output.

  In addition, the user's reaction to the content reproduced on the information terminal 200 may be detected by a device other than the information terminal 200. The device detects the presence / absence of a reaction and its type. More specifically, when the device detects that a reaction has occurred, the device identifies the type of the reaction and outputs it to the information terminal 200. The information terminal 200 detects that there has been a user's reaction and the type of the user's reaction based on information input from the device.

  The content shared in the present embodiment is, for example, a moving image file, audio file, text file, character, still image, video captured in real time by the camera 18, and / or audio acquired in real time by the microphone 19. It is comprised by.

  Whether or not there is a reaction is determined based on the state at the start of content sharing (facial expression photographed by the camera 28 and the tone of the sound acquired by the microphone 29), the state at the time of reaction detection, or when the previous reaction is detected. It may be made on the basis of the difference from the state.

  As the reaction information output in the present embodiment, in addition to the image (image 301) shown as the information terminal 100C in FIG. 2, as described with reference to FIG. It may be a pictograph, a voice, or a combination thereof. Which of the plurality of reaction information is output may be selected in a predetermined order, for example. Specifically, for example, the type of content information to be output is selected in the order of image, character, pictogram, sound, etc., according to the number of times the same type of reaction is detected during playback of the same content. May be. More specifically, when a reaction of the type “laughter” is detected for a certain content for the first time, an image corresponding to “laughter” is output as reaction information. Next, when a reaction of the type “laughter” is detected, characters corresponding to “laughter” are output as reaction information. Next, when a reaction of the type “laughter” is detected, a pictograph corresponding to “laughter” is output as reaction information. Next, when a reaction of the type “laughter” is detected, a sound corresponding to “laughter” is output as reaction information.

  Since the reaction information is output not by directly outputting the user's figure or voice, but by displaying the image etc., the reaction of the user of the information terminal receiving the content is indirectly Thus, the information is transmitted to the information terminal that has supplied the content. As a result, the user of the information terminal on the side receiving the content can transmit the reaction to the user of the partner information terminal with relatively little resistance.

  In the information terminal 200, the reaction was continuously detected by continuously performing the shooting with the camera 28 and the sound collection with the microphone 29 during the reproduction of the content. Each time a reaction is detected, reaction information is generated and transmitted to the information terminal 100.

  The reaction may be detected at the information terminal 200 only at a specific timing. For example, when the user on the information terminal 200 operates a specific operation button, the user of the information terminal 200 designates the reaction. It may be performed only at the timing.

  Alternatively, the reaction is detected on the information terminal 200 only at a timing designated by the user on the information terminal 100 side, such as when the user operates a specific operation button on the information terminal 100 on the content transmission side. Also good. Moreover, in the information terminal 100, the timing which should detect reaction in the information terminal 200 can also be set previously. For example, when content sharing is started, the user of the information terminal 100 can set a condition “acquire user reaction at the information terminal 200 one minute after the start of content sharing”.

  The reaction information is output in such a manner that, for example, pre-registered audio information is output, or a pre-registered image is output for a certain period of time. The image is displayed on the display 15 for a given time, for example. Further, the display position of the image may be displayed at a fixed position on the display 15, or may be scrolled on the display 15 as shown as information terminals 100D, 100E, and 100F in FIG. . In the information terminals 100D, 100E, and 100F, the reaction information is shown as a combination of the image 302 and the character 303. In the information terminals 100D, 100E, and 100F in FIG. 9, the image 302 and the characters 303 are scroll-displayed from the left side to the right side.

  In the present embodiment, the reaction information is generated in the information terminal 200 and transmitted to the information terminal 100. In the information processing system, the reaction information may be generated also at the terminal on the content transmission side. That is, the reaction information can be further generated at the information terminal 100 and transmitted to the information terminal 200.

[Second Embodiment]
As described with reference to FIG. 6, the reaction information may be not related to the type of reaction other than the information changed according to the type of reaction. That is, the same reaction information may be output even if the reaction types are different.

[Third Embodiment]
As the output mode of the reaction information, in addition to the image shown in FIG. 6, the display of the background on the screen may be changed, or an animation display that vibrates the screen may be used. It may be a blinking display on the screen.

[Fourth Embodiment]
As the output mode of the reaction information, the display size of the images, characters and pictograms shown in FIG. 6 is changed according to the volume of the reaction voice and the change in the expression, and the volume of the output voice is changed. You may do it.

  It should be thought that embodiment disclosed this time and its modification are illustrations in all the points, and are not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  10,20 CPU, 15,25 Display, 100,200 Information terminal.

Claims (5)

A communication unit for receiving content data from other terminals;
A playback unit for playing back content based on the data received from the other terminal;
A detection unit for detecting a user reaction when the content is being reproduced in the reproduction unit;
Based on the reaction detected by the detection unit, a generation unit for generating reaction information that is information indicating the content of the reaction,
The communication unit is an information terminal that transmits the reaction information to the other terminal. It further includes an input unit for receiving information input,
The information terminal according to claim 1, wherein the generation unit generates the reaction information in response to information input to the input unit. A playback unit for playing back content,
A communication unit for transmitting content played back by the playback unit to another terminal;
The communication unit receives reaction information that is information indicating details of a reaction at the other terminal with respect to the content,
The playback unit plays back the reaction information together with the content. An input unit for inputting timing information which is information for designating a timing for requesting a reaction to the content;
The information terminal according to claim 3, wherein the communication unit receives reaction information acquired by the other terminal at a timing specified by the timing information. An information processing system comprising a first information terminal including a first reproduction unit for reproducing content, and a second information terminal capable of communicating with the first information terminal,
The first information terminal is
A first communication unit for transmitting the content reproduced in the first reproduction unit to the second information terminal;
The second information terminal is
A second playback unit for playing back the content received from the first information terminal;
A detection unit for detecting a user reaction in the second information terminal when the content is being reproduced in the second reproduction unit;
Based on the reaction detected by the detection unit, a generation unit for generating reaction information;
A second communication unit for transmitting the reaction information to the first information terminal,
The first communication unit receives the reaction information from the second information terminal,
The information processing system, wherein the first reproduction unit reproduces the reaction information together with the content.

Images