JP6023955B2 - COMMUNICATION SYSTEM, COMMUNICATION DEVICE, PROGRAM, AND COMMUNICATION CONTROL METHOD - Google Patents

Description

  The present invention relates to a communication system, a communication device, a program, and a communication control method, and more particularly to a communication system, a communication device, a program, and a communication control method using, for example, a doll or robot having a figure imitating a human being.

  An example of background art of the present invention is disclosed in Patent Document 1. According to the communication device of Patent Document 1, the voice transmission / reception unit transmits / receives a voice signal such as a conversation, and the shared robot that is a speaker and a listener responds to the voice signal by moving its head and opening / closing the mouth. It behaves like movement, blinking eyes, or body gesture. A robot as a speaker performs blinking operation and mouth opening / closing operation. On the other hand, a robot as a listener performs a whispering motion. However, the timing of each operation is determined based on the ON / OFF of the electric signal, taking the audio signal as ON / OFF of the time-series electric signal.

JP2000-349920A

  However, in the technique of Patent Document 1, even if the pull-in phenomenon is expressed by sharing the rhythm of conversation, the shared robot responds to the audio signal in response to a whispering action of the head, opening / closing action of the mouth, It only performs a fixed action such as a blinking action or a gesture action of the body, and it is not possible to sufficiently obtain the feeling of talking with the opponent or the presence of the opponent.

  Therefore, a main object of the present invention is to provide a novel communication system, communication apparatus, program, and communication control method.

  Another object of the present invention is to provide a communication system, a communication device, a program, and a communication control method capable of sufficiently obtaining a feeling of talking to a remote partner and the presence of the partner.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate correspondence relationships with embodiments described later to help understanding of the present invention, and do not limit the present invention in any way.

A first invention is a communication system comprising a telephone device and a doll having a housing portion for accommodating the telephone device, the doll having a figure imitating a human being and being held by a user during use. Detecting means for detecting the voice of the conversation partner output from, sound information calculating means for calculating the loudness and height of the voice detected by the detecting means, and loudness and height of the voice calculated by the sound information calculating means Vibration generating means for generating vibration based on the height, and the vibration generating means is a communication system arranged at a position corresponding to the chest of the doll .

In the first invention, the communication system includes a telephone device (14) such as a mobile phone and a doll (12) having an accommodating portion (12a) for accommodating the telephone device. The doll has a figure imitating a human and is embraced by the user when in use. The detecting means (16) detects the voice of the conversation partner output from the telephone device. The sound information calculation means (32, S1, S7) calculates the loudness and height of the voice detected by the detection means. The vibration generating means (20, 32, 36, S11, S13) generates vibration based on the loudness and height of the voice calculated by the sound information calculating means. This vibration generating means is arranged at a position corresponding to the chest of the doll. Therefore, the body of Yoo over THE that had Idaku dolls, vibration corresponding to the size and voice pitch of the voice of the dialogue partner is transmitted through the doll. That is, a change in voice volume and voice pitch is transmitted to the user.

According to a first aspect of the present invention, vibration corresponding to the voice of dialogue partner, since through dolls Yu over THE is embraced transmitted to the fuselage of the user, knowing the feelings of the dialogue partner by vibration, hug dialogue partner present to the senses and remote you are talking Te dialogue partner and a real feeling that the story can be sufficiently obtained. In addition, the presence of the conversation partner can be increased.

  The second invention is dependent on the first invention, and the loudness and height of the voice of the current conversation partner calculated by the sound information calculation means and the dialog in the normal state calculated by the sound information calculation means Comparing means for comparing the loudness and height of the opponent's voice is further provided, and the vibration generating means generates vibration according to the comparison result of the comparing means.

  In the second invention, the communication system further includes comparison means (32, S9). The comparison means (32, S9) is configured such that the current conversation partner's voice loudness and height calculated by the sound information calculation means and the conversation partner's voice in the normal state calculated by the sound information calculation means. Compare size and height respectively. That is, the emotion of the current user is estimated by comparison with the normal state. Therefore, the vibration generating means generates vibration according to the comparison result of the comparing means, that is, the current user's emotion.

  According to the second invention, since the current emotion of the conversation partner can be known not only from the voice but also from the vibration, it is possible to obtain a greater feeling of talking with the conversation partner and the presence of the conversation partner. .

  A third invention is dependent on the second invention, and the vibration generating means changes at least one of the intensity and period of the vibration according to the comparison result of the comparison means.

  In the third invention, the vibration generating means changes at least one of the intensity and period of the vibration according to the comparison result of the comparison means. For example, the intensity and / or cycle of vibration is changed according to the emotion of the conversation partner. That is, the emotion of the conversation partner is expressed by the type of vibration.

  According to the third invention, it is possible to convey the feeling of the conversation partner by the intensity and period of the vibration.

4th invention is a communication apparatus provided with the doll which has the accommodating part which can accommodate a telephone apparatus, Comprising: A doll has a figure imitating a person, and it is embraced by the user at the time of use, and is output from a telephone apparatus Based on the sound detection means for detecting the voice of the conversation partner, the sound information calculation means for calculating the loudness and height of the voice detected by the detection means, and the loudness and height of the voice calculated by the sound information calculation means The vibration generating means is a communication device arranged at a position corresponding to the chest of the doll .

According to a fifth aspect of the present invention, the telephone device has a shape imitating a human being, and has a housing portion for housing the telephone device and a vibration generating unit disposed at a position corresponding to the chest, and is hugged by a user when used. A program executed in a communication system including a doll, wherein the communication system processor detects (a) a voice of a conversation partner output from the call device, and (b) is detected in step (a). A program for executing a step of calculating a loudness and a height of a voice, and (c) generating a vibration by a vibration generating unit based on the loudness and the height of the voice calculated in step (b). .

A sixth aspect of the present invention has a telephone device and a figure imitating a human being, and also includes a housing unit for housing the telephone device and a vibration generating unit disposed at a position corresponding to the chest, and is hugged by a user during use. a communication control method for a communication system comprising a doll that is, communication system, (a) to detect the voice of dialogue partner outputted from the communication device, (b) the magnitude of the detected voice in step (a) And (c) a communication control method in which vibration is generated by the vibration generator based on the loudness and height of the voice calculated in step (b).

  In the fourth to sixth inventions, as in the first invention, it is possible to sufficiently obtain a feeling of talking with a remote conversation partner and the presence of the conversation partner.

  According to the present invention, the vibration according to the voice of the conversation partner is transmitted through the doll touched by the user's hand or torso. For example, by talking about the emotion of the conversation partner by the vibration, it is possible to talk with the remote conversation partner. You can get a full sense of the presence and presence of the conversation partner.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

FIG. 1 is an illustrative view showing one example of a communication system of the present invention. FIG. 2 is a block diagram showing an electrical configuration of the communication system shown in FIG. FIG. 3 is an illustrative view for explaining a situation in which the communication system shown in FIG. 1 is used. FIG. 4 is a table showing an example of a vibration pattern of the vibration motor shown in FIGS. FIG. 5 is an illustrative view for explaining the drive voltage of the vibration motor shown in FIGS. 1 and 2. FIG. 6 is an illustrative view showing one example of a memory map of a RAM built in the processor shown in FIG. FIG. 7 is a flowchart showing an example of conversion processing of the processor shown in FIG.

  Referring to FIG. 1, a communication system (hereinafter simply referred to as “system”) 10 of this embodiment includes a doll 12. The doll 12 also functions as a communication device and has a figure (appearance etc.) imitating a human. For example, the surface (skin) of the doll 12 is formed of silicon or vinyl chloride, and padding such as cotton is filled therein.

  In this embodiment, the doll 12 functioning as a communication device is used. However, a stuffed toy having a form similar to that of a human may be used. In this embodiment, it is called a “doll”, but it can also be called a “robot” having a sound sensor such as a microphone 16 and vibration generating means including the vibration motor 20 described later.

The doll 12 has an accommodating portion 12a such as a pocket in a part corresponding to the head, and a cellular phone 14 such as a future phone or a smartphone is accommodated in the accommodating portion 12a. However, it is not necessary to be limited to the mobile phone 14, and the speakerphone of the main body and the communication device (speakerphone) that are communicably connected using a communication technology such as Bluetooth (registered trademark) is stored in the housing portion 12 a. You may make it accommodate.

  The doll 12 includes a microphone 16, a switch 18, a vibration motor 20, and a control box 30. The microphone 16, the switch 18 and the vibration motor 20 are connected to the electric circuit 12b in the control box 30 (see FIG. 2).

  The microphone 16 is provided in the vicinity of the accommodating portion 12a. In this embodiment, when the cellular phone 14 is accommodated in the accommodating portion 12a, it is arranged at a position where sound or sound output from the speaker of the cellular phone 14 can be detected. Therefore, for example, a microphone 16 having directivity is used. This is to detect only the voice or sound output from the speaker of the mobile phone 14 and not to detect the voice of the user who uses the system 10 (mobile phone 14).

  The switch 18 is a push button switch, a contact sensor, a slide switch, or the like, and is arranged so that its operation part (pressing part, contact part, slide part, etc.) is exposed on the surface of the doll 12 body. The vibration motor 20 is disposed inside the doll 12 and at a position corresponding to the chest of the doll 12. In this embodiment, the control box 30 is disposed inside the doll 12 and at a position corresponding to the neck of the doll 12.

  FIG. 2 is an example of a block diagram showing the electrical configuration of the system 10. As shown in FIG. 2, the system 10 includes a mobile phone 14. As described above, the microphone 16, the vibration motor 20, and the control box 30 are provided inside the doll 12. Further, as described above, the switch 18 is provided on the doll 12 so that the operation unit is exposed.

  The electric circuit 12 b provided inside the control box 30 drives the vibration motor 20 based on the voice of the conversation partner (call partner) detected through the microphone 16. That is, the electric circuit 12b has a function of converting voice information (voice information) of the conversation partner into vibration information (vibration information). In this embodiment, the voice information includes a voice volume (sound pressure level (voice power)) and a voice pitch (pitch (frequency)), and a voice signal corresponding to the voice of the conversation partner detected through the microphone 16. Detected (calculated).

  The electric circuit 12 b includes a processor 32, and an A / D converter 34, a motor drive circuit 36, and a power supply control circuit 38 are connected to the processor 32. The power control circuit 38 is connected to the motor drive circuit 36. Further, a battery 40 is provided inside the control box 30, and the battery 40 is connected to the power supply control circuit 38.

  The microphone 16 is connected to the A / D converter 34, the switch 18 is connected to the processor 32, and the vibration motor 20 is connected to the motor drive circuit 36.

  When the user has a conversation (call) using the system 10, the doll 12 is used as a substitute for the conversation partner (call partner). As shown in FIG. 3, the doll 12 in which the mobile phone 14 is accommodated in the accommodating portion 12a is used. Embraced by the user. At this time, one hand (arm) of the user contacts a portion corresponding to the back of the doll 12, and the other hand (arm) of the user contacts a portion corresponding to the back or the back of the doll 12. . For example, the user talks using the mobile phone 14 in a so-called hands-free manner, and at this time, the vibration motor 20 is driven according to the voice of the conversation partner. Therefore, vibration generated by driving the vibration motor 20 is transmitted to the user's hand (arm) and chest (torso) through the doll 12.

  In this embodiment, the user is hugging the doll. However, the present invention is not limited to this. As another embodiment, the user may face the doll 12 and hold the doll 12 with both hands. .

  Specifically, vibration according to the emotion of the conversation partner estimated from the voice information is presented. In this embodiment, the vibration pattern is determined as shown in FIG. Although a detailed description is omitted, this vibration pattern is created based on research for estimating or recognizing human emotion from voice information.

  As shown in FIG. 4, in the vibration pattern table, estimated emotions and vibration patterns are described according to the magnitude relationship between the loudness of the voice (Po) and the height of the voice (Pi). In this embodiment, for the sake of easy understanding, emotions estimated from the loudness and the loudness of the voice are described. However, it is not necessary to actually estimate the emotions, and the table shows the loudness of the voice. Only the vibration pattern needs to be described according to the magnitude relationship between (Po) and the voice pitch (Pi).

  In the example shown in FIG. 4, the loudness (Po) is larger than the normal (normal state) loudness (APo), and the loudness (Pi) is higher than the normal loudness (APi). Is too large, the emotion of “joy” is estimated. Accordingly, as the vibration pattern, the intensity P is set to “large” and the period T is set to “medium (normal)”.

  Here, a drive signal or drive voltage of the vibration motor 20 will be described with reference to FIG. As shown in FIG. 5, the drive signal or drive voltage is represented by a pulsed rectangular wave. For example, a rectangular wave has a peak value P and represents the intensity of vibration. However, in this embodiment, the strength P corresponds to the drive voltage of the vibration motor 20. Therefore, for example, the strength P is changed according to the magnitude of the drive voltage input (applied) to the vibration motor 20 and is set in three stages of “large”, “medium (normal)”, and “small” ( Switched). Further, in this embodiment, the pulse width Tb and the period T of the rectangular wave are set (determined) through experiments or the like so as to express a human heartbeat by vibration generated by the vibration motor 20. )

  In this embodiment, the vibration intensity P is changed by the magnitude of the drive voltage, but the present invention is not limited to this. A plurality of vibration motors (eccentric motors) provided with weights having different weights may be provided, and the driving vibration motors may be switched.

  In this embodiment, the period T is set (switched) in three stages of “long”, “medium (normal)”, and “short”. The period T in the case of medium (normal) is set to an average value (about 1 second) of the period of the human heartbeat.

  However, in this embodiment, the normal voice volume (APo) and the normal voice pitch (APi) are determined based on the conversation partner detected in the initial fixed time (for example, 5 to 10 seconds). The average value of the voice volume and the average value of the voice pitch. This is because, at the beginning of a call, the emotion of the other party (the conversation partner) is considered to be a normal (normal) state. In addition, the average value for a certain period of time is calculated because it is difficult to estimate the emotion of the conversation partner with the instantaneous voice volume and voice pitch. In addition, the current voice volume (Po) and voice pitch (Pi) are the average value and voice of the conversation partner for the past certain period of time (for example, 5 to 10 seconds) including the current voice. It is the average value of height. Hereinafter, this is the same in this embodiment.

  Returning to FIG. 4, when the loudness (Po) is larger than the normal loudness (APo) and the loudness (Pi) is smaller than the normal loudness (APi). In this case, the feeling of “anger” is estimated. Accordingly, as the vibration pattern, the intensity P is set to “large” and the period T is set to “short”.

  Furthermore, when the loudness (Po) is smaller than the normal loudness (APo) and the loudness (Pi) is smaller than the normal loudness (APi), “sadness”. Is estimated. Accordingly, as the vibration pattern, the intensity P is set to “small” and the period T is set to “long”.

  Furthermore, in cases other than the above, that is, when the voice volume (Po) is the same as the normal voice volume (APo), or the voice pitch (Pi) is the normal voice pitch ( If it is the same as APi) or both, “normal” emotions are estimated. Accordingly, as the vibration pattern, the intensity P is set to “medium (normal)” and the period T is set to “medium (normal)”.

  By doing so, you can feel the feelings of the other party through vibration, so you can get a feeling of hugging and speaking with the other party, you can get the feeling of talking with the other party, Can be improved.

  FIG. 6 shows an example of the memory map 50 of the RAM 32a provided in the processor 32 shown in FIG. As shown in FIG. 6, the RAM 32 a includes a program storage area 52 and a data storage area 54. The program storage area 52 stores a voice detection program 52a, a conversion processing program 52b, and the like.

  The voice detection program 52a is a program for detecting the voice (voice) of the conversation partner during a call. For example, when a main power supply (not shown) is turned on, the microphone 16 is turned on, and sound (ringing tone, ringing melody) or voice (voice of the conversation partner) of a certain level (dB) or more based on an input from the microphone 16. Is detected, voice detection processing is executed. In this embodiment, the loudness and pitch of the conversation partner are detected.

  The conversion processing program 52b is a program for driving and controlling the vibration motor 20 based on the detected loudness and pitch of the conversation partner. However, the vibration pattern is determined based on the detected loudness and pitch of the conversation partner according to the table shown in FIG. 4 (table data 54d described later).

  Although illustration is omitted, other programs necessary for converting voice information into vibration information are also stored.

  The data storage area 54 is provided with an audio data buffer 54a. The data storage area 54 stores normal audio data 54b, current audio data 54c, table data 54d, and motor control data 54e.

  The voice data buffer 54a temporarily stores voice data corresponding to the voice of the conversation partner detected through the microphone 16 in time series. The audio data stored in the audio data buffer 54a is deleted after being used for the processing of the processor 32.

  The normal voice data 54b is data regarding the loudness (APo) and the loudness (APi) of the voice when the emotion of the conversation partner is normal (normal) or predicted to be normal. As described above, the normal (state) conversation partner's voice volume (APo) and voice pitch (APi) are the average of the conversation partner's voice volume for a certain time from the start of the call (dialog). The average value and the pitch of the voice.

  The current voice data 54c is data on the loudness (Po) and voice pitch (Pi) of the current conversation partner. As described above, the voice level (Po) and the voice level (Pi) of the current conversation partner are the average value of the voice magnitudes of the conversation partner's voice for a certain past period including the present time, and This is the average voice pitch. Table data 54d is data on the vibration pattern table shown in FIG. As described above, since it is not necessary to estimate the emotion, the estimated emotion may be deleted from the table.

  The motor control data 54e is data on the value of the strength (voltage) P and the value of the period T for defining the drive signal (drive pulse) or drive voltage corresponding to the vibration pattern determined according to the table data 54d.

  Although not shown, the data storage area 54 stores other data necessary for executing a program such as the voice detection program 52a and the conversion processing program 52b, and is provided with a counter (timer) and a flag. Or

  FIG. 7 is a flowchart of the conversion process of the processor 32 shown in FIG. For example, when the processor 32 detects a sound (ringing tone, ringing melody) or voice (voice of the conversation partner) of a predetermined level or higher, the processor 32 starts this conversion process. In another embodiment, the conversion process may be started in response to a user instruction (for example, operation of the switch 18).

  As shown in FIG. 7, when the conversion process is started, the processor 32 calculates the average value of the voice volume and the average value of the voice pitch in a certain time in step S1. However, at least a certain amount of audio data is stored in the audio data buffer 54a. Although not shown, the process of detecting the voice of the conversation partner and storing the voice data in the voice data buffer 54a is executed as a task different from the conversion process.

  In the next step S3, the average value of the conversation partner's voice and the average value of the voice pitch calculated in step S1 are stored as normal voice magnitude (APo) and voice pitch (APi). To do. That is, the normal audio data 54 b is stored in the data storage area 54.

  In a succeeding step S5, it is determined whether or not the process is finished. Here, the processor 32 determines whether or not there is an end instruction from the user by determining whether or not the operation data from the switch 18 has been input. In addition, when the voice call is finished, it is not necessary to execute the conversion process. Therefore, the conversion process is also finished when the voice of the conversation partner is not detected for a certain time (for example, 30 seconds) or longer. Also good.

  If “YES” in the step S5, that is, if it is finished, the conversion process is finished as it is. On the other hand, if “NO” in the step S5, that is, if not completed, the current voice volume (Po) and voice pitch (Pi) are detected (calculated) in a step S7. That is, the processor 32 calculates the average value of the conversation partner's voice volume and the average voice pitch for a certain period of time including the present time, and calculates the calculated average value of the conversation partner's voice and The average value of the voice pitch is stored as the current voice volume (Po) and voice pitch (Pi). That is, the current audio data 54 c is stored in the data storage area 54. However, in the second and subsequent times, the current audio data 54c is updated.

  In the next step S9, the current voice volume (Po) and voice pitch (Pi) are compared with the normal voice volume (APo) and voice pitch (APi), respectively. Subsequently, in step S11, the intensity P and the period T are determined. Here, the processor 32 refers to the table data 54d, determines the intensity P and the period T according to the comparison result in step S9, and stores the data (motor control data 54e) corresponding thereto in the data storage area 54. Remember. In step S13, the vibration motor 20 is driven with the determined intensity P and cycle T, and the process returns to step S5. In step S <b> 13, the processor 32 generates a drive signal (drive pulse) based on the intensity P and the period T, and outputs the drive signal to the motor drive circuit 36. The motor drive circuit 36 generates a drive voltage according to the drive pulse and applies it to the vibration motor 20. Then, the vibration motor 20 is driven, and vibration according to the emotion estimated from the voice is generated. This vibration is transmitted to the user's hand (arm) and chest (torso) through the doll 12.

  According to this embodiment, the vibration according to the emotion estimated from the voice information of the conversation partner is transmitted to the user through the doll that the user is hugging. You can get a real feeling. In addition, the presence of the conversation partner can be increased.

  In this embodiment, the vibration motor is driven so as to generate a vibration like a beating using a table of vibration patterns corresponding to emotions estimated from the loudness and pitch of the conversation partner. However, it need not be limited to this. Instead of the vibration pattern expressing the beat according to the estimated emotion, a table of vibration patterns expressing the loudness and pitch of the voice may be prepared. Further, the driving voltage (driving pulse) applied to the vibration motor need not be limited to a rectangular wave, and may be a sawtooth wave or a triangular wave. That is, the vibration motor may be driven so as to generate vibration with an arbitrary pattern (intensity and period) according to the voice state of the conversation partner.

  In this embodiment, the intensity and period of vibration, that is, the magnitude of the driving voltage of the vibration motor and the period of vibration (driving) are set in three stages, respectively. However, the present invention is not limited to this. There is no. If the magnitude of the drive voltage and the period of vibration can be set in at least two stages, they may be set in four or more stages. In addition, by using a predetermined arithmetic expression from the loudness and the pitch of the voice without determining the strength of the vibration (the magnitude of the driving voltage) and the period of the vibration (the driving) based on the vibration pattern table. Alternatively, the intensity of vibration and the period of vibration may be obtained by calculation to generate a drive pulse (drive voltage).

  In addition, the specific numerical values shown in this embodiment are merely examples, and should not be limited, and can be appropriately changed according to the product to be implemented.

10 ... Communication system 12 ... Doll (communication device)
DESCRIPTION OF SYMBOLS 14 ... Mobile phone 16 ... Microphone 18 ... Switch 20 ... Vibration motor 32 ... Processor 38 ... Power supply control circuit

Claims (6)

A communication system comprising a telephone device and a doll having an accommodating portion for accommodating the telephone device,
The doll has a figure imitating a human being and is embraced by a user when used,
Detecting means for detecting the voice of the conversation partner output from the telephone device;
Sound information calculating means for calculating the loudness and height of the voice detected by the detecting means; and vibration generating means for generating vibration based on the loudness and height of the voice calculated by the sound information calculating means. Prepared ,
The communication system , wherein the vibration generating means is arranged at a position corresponding to the chest of the doll . The current loudness and height of the conversation partner's voice calculated by the sound information calculation means are compared with the loudness and height of the conversation partner's voice in the normal state calculated by the sound information calculation means, respectively. A comparison means for
The communication system according to claim 1, wherein the vibration generating unit generates a vibration according to a comparison result of the comparison unit.   The communication system according to claim 2, wherein the vibration generating unit changes at least one of a vibration intensity and a period according to a comparison result of the comparison unit. A communication device including a doll having a storage unit capable of storing a telephone device,
The doll has a figure imitating a human being and is embraced by a user when used,
Sound detection means for detecting the voice of the conversation partner output from the telephone device;
Sound information calculating means for calculating the loudness and height of the voice detected by the detecting means; and vibration generating means for generating vibration based on the loudness and height of the voice calculated by the sound information calculating means. Prepared ,
The communication device , wherein the vibration generating means is disposed at a position corresponding to the chest of the doll . A communication system comprising a telephone device and a doll that has a figure imitating a human, a housing unit that houses the telephone device, and a vibration generating unit that is disposed at a position corresponding to the chest, and is held by a user when used A program executed in
In the processor of the communication system,
(A) detecting a voice of a conversation partner output from the telephone device;
(B) calculating the loudness and height of the voice detected in step (a); and (c) generating the vibration based on the loudness and height of the voice calculated in step (b). A program for executing a step of generating vibration by a part . A communication system comprising a telephone device and a doll that has a figure imitating a human, a housing unit that houses the telephone device, and a vibration generating unit that is disposed at a position corresponding to the chest, and is held by a user when used Communication control method,
The communication system includes:
(A) detecting the voice of the conversation partner output from the telephone device;
(B) calculating the loudness and height of the voice detected in the step (a), and (c) the vibration generator based on the loudness and height of the voice calculated in the step (b). A communication control method that generates vibrations.

Images