JP2005251133A - Method and system for physical pull-in appealing to tactile sense - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a physical pull-in method for providing nonverbal information which does not require switching of a sight line for a speaker or a listener and produces a physical pull-in phenomenon for each, in a conversation when a concerned party is present right before, or in a scene when speakers or listeners are unspecified or many. <P>SOLUTION: For producing, for a speaker, a physical pull-in phenomenon with respect to a listener, the speaker is made to carry an operation unit 14 driven by a driving part 13 at operation timing calculated by the operation part of a computer 31, the operation part calculates, as the operation timing, the time point when an operation predictive value estimated from a signal of the voice of the speaker exceeds a predetermined operation threshold, and at the operation timing, the driving part 13 drives the operation unit 14 to teach the speaker about the operation timing as tactile sensory information, thus producing, for the speaker, the physical pull-in phenomenon with respect to the listener. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a physical pull-in method that provides non-verbal information that causes a physical pull-in phenomenon to a speaker or listener as tactile sensory information, and a system that provides the tactile sensory information.

  Talkers and listeners who communicate through conversations listen to their own or the other's voice (verbal information) and acquire sensory information (non-verbal information) including their own or other's body movements to share their physical rhythms. , Create a state that is easier to talk or hear, and get drawn into the conversation. This phenomenon of being drawn into conversation by sharing physical rhythm is called “physical attraction”. From now on, if there is a device or system that causes a physical retraction phenomenon to the speaker instead of the listener, or a device or system that causes a physical retraction phenomenon to the listener instead of the speaker, the physical retraction phenomenon will be brought to the speaker or listener, respectively. be able to.

  As an apparatus using such a physical pull-in phenomenon, for example, there is Patent Document 1. This patent document 1 causes a physical pull-in phenomenon to a speaker who talks to the robot or a listener who hears a voice flowing from the robot, based on visual sensory information as non-verbal information, that is, a body motion of the robot. Here, the body motion includes a selective combination of mouth opening / closing operation, eye blinking operation, or body gesture operation, and the operation of each part is determined based on the operation timing of the head movement operation. This whispering operation timing is a specific time point on the time series where it is estimated that the listener's reaction appears most prominently. In this patent document 1, the whirling prediction value estimated from speech exceeds a predetermined whispering threshold. It is said. Here, instead of the robot, a character based on a computer image may be used.

Japanese Patent Laid-Open No. 2001-009169 (pages 2 to 6, FIG. 1)

  The apparatus of the above-mentioned patent document 1 assumes a conversation where there is no other party at hand, and gives the robot's body motion, which is captured as visual sensory information, as non-verbal information, causing a physical pull-in phenomenon to the speaker or listener. However, it is difficult to give visual sensory information as non-verbal information because the other party is usually at hand in conversation. For example, if a robot or computer image character is prepared for a speaker separately from the listener, the gaze of the speaker must be switched between the listener and the robot or character. However, such gaze switching makes it difficult to cause a physical pull-in phenomenon due to visual sensory information as non-verbal information obtained from a listener, a robot, or a character.

  The problem due to the switching of the line of sight can also occur in scenes where the number of speakers or listeners is unspecified or many, specifically, television broadcasting, lectures, street speeches or presentations. In particular, it is difficult to give nonverbal information as visual sensory information in presentations. In a normal presentation, the screen is darkened as a whole, and a speaker or listener's line of sight is directed to the display screen of the presentation contents. For this reason, it is difficult to give visual sense information as non-verbal information from other than the display screen.

  Here, the apparatus of Patent Document 1 has a function of causing a physical pull-in phenomenon in that a body motion of a robot centering on a whispering motion is executed as visual sense information at a specific motion timing. From this, it can be considered that if non-verbal information other than visual sensory information is given at a specific timing, a physical pull-in phenomenon can be brought to the speaker or listener. Therefore, in conversations where the other party is in front of you, or in situations where the number of speakers or listeners is unspecified or large, non-verbal information that causes physical pull-in to each other without requiring the speakers or listeners to switch gazes. In order to develop a physical pull-in method to give and a system to give the non-verbal information, it was studied.

  What has been developed as a result of the study is a method and system that provides tactile sensory information as non-verbal information to a speaker or listener to bring about a physical pull-in phenomenon. First, in a conversation where the other party is in front of you, or in a scene where the number of speakers or listeners is unspecified or many, the unit of motion that is driven by the drive unit at the timing calculated by the calculation unit when bringing the physical pull-in phenomenon to the speaker to the speaker The calculation unit calculates the operation timing when the predicted motion value estimated from the speaker's voice signal exceeds a predetermined operation threshold, and the drive unit drives the operation unit at the operation timing. A physical pull-in method that appeals to the tactile sensation that brings the speaker to the physical pull-in phenomenon is used by teaching the timing of movement to the speaker as tactile sensory information. Here, when a speaker “has” a motion unit, it means that the motion unit is in direct or indirect contact with the body of the speaker. In addition to holding the motion unit in the hand, the motion unit is put in a pocket. Including cases (hereinafter the same).

  The above operation timing is a specific time point on the time series that drives the operation unit, but in relation to the physical pull-in phenomenon, the operation timing is a specific time point on the time series where it is estimated that the listener's reaction will appear prominently. Means. In other words, the physical pull-in method that appeals to the tactile sensation of the speaker gives the tactile sensory information directly to the speaker in response to a motion unit that is driven at a specific point in time on the time series where it is estimated that the listener's response will appear prominently. This brings about a physical pull-in phenomenon suitable for a speaker possessing this movement unit.

  The motion prediction value estimated from the speaker's voice signal may be calculated by integrating the degree of influence of the speaker's voice signal acquired in the past within a certain time range from the present. The degree of influence can be derived by relating the present and the past by a linear combination, a non-linear combination, a neural network, or the like. For example, when the degree of influence of a speaker's voice signal acquired in the past in a certain time range from the present on the present is derived by linear combination, the motion prediction value is estimated by a moving average of the speaker's voice signal. For example, the moving average is used as the moving average. In Equation 1, y (i) represents a motion prediction value, a (j) represents a prediction coefficient, and x (i−j) represents a speaker's voice signal.

  As can be seen from the following equation (2), a noise w (i) based on a random number may be added to the equation (1), and a different value may be set each time the motion prediction value y (i) is calculated. The noise w (i) has a function of adding natural fluctuation to the motion prediction value y (i) and excluding unnatural regularity from the determination of motion timing.

  In the present invention, the operation unit is not limited as long as the operation unit is driven at the operation timing and tactile sensory information can be given to the speaker. From now on, the motion unit may be one of or a combination of a configuration in which a pulling motion is given to a possessed speaker, a configuration in which a pressing motion is imparted to a possessed speaker, a configuration in which a vibration is imparted to a possessed speaker, or a configuration in which a wind is given to a possessed speaker. Can be used.

  Next, in a conversation where the other party is in front of you, or in a scene where the number of speakers or listeners is unspecified or many, an operation driven by the drive unit at the operation timing calculated by the calculation unit when bringing the physical pull-in phenomenon to the speaker to the listener The operation unit calculates the point in time when the predicted motion value estimated from the speaker's voice signal exceeds a predetermined motion threshold, and the drive unit drives the motion unit at the motion timing. Then, a physical pull-in method that appeals to the tactile sensation that brings the physical pull-in phenomenon to the speaker to the listener by teaching the operation timing as tactile sensory information to the listener is used.

  The above operation timing is a specific time point on the time series that drives the operation unit, but in relation to the physical pull-in phenomenon, the operation timing is a specific time point on the time series where it is estimated that the listener's reaction will appear prominently. Means. In other words, the physical pull-in method that appeals to the listener's tactile sensation directly synchronizes the listener with tactile sensory information that is driven at a specific point in time on the time series where it is estimated that the listener's response will appear prominently. This brings about a physical pull-in phenomenon appropriate for the listener who possesses this movement unit.

  The motion prediction value estimated from the speaker's voice signal may be calculated by integrating the degree of influence of the speaker's voice signal acquired in the past within a certain time range from the present. The degree of influence can be derived by relating the present and the past by a linear combination, a non-linear combination, a neural network, or the like. For example, when the degree of influence of the speaker's voice signal acquired in the past in a certain time range from the present on the current is derived by linear combination, the motion prediction value is estimated by the moving average of the speaker's voice signal. As the moving average, for example, the above equation 1 can be used.

  Further, the value 2 may be used to add a noise w (i) based on a random number, and a different value each time the motion prediction value y (i) is calculated. The noise w (i) has a function of adding natural fluctuation to the motion prediction value y (i) and excluding unnatural regularity from the determination of motion timing.

  Here, the predicted motion values of the speaker and the listener can be set to the same value by using the same prediction coefficient because the speech signal of the basic speaker is the same. In this case, if the operation thresholds of the speaker and the listener are the same, the operation timings of the operation units of the speaker and the listener also coincide. Nonverbal information in normal conversation is dominated by the listener's response to the speaker. For this reason, it is considered that the speaker and the listener can equally feel the specific time point on the time series estimated to have a noticeable response of the listener to the speaker, share the physical rhythm, and bring about the physical pull-in phenomenon respectively. It is done. From this, it is considered realistic that the operation timings of the respective operation units of the speaker and the listener are made coincident and tactile sensory information is given to the speaker and the listener at the same timing. Strictly speaking, the motion timing, which is a specific point in time that is suitable for bringing a physical pull-in phenomenon to the speaker, may be different from the motion timing of the listener's motion unit. In addition to using different coefficients, it is better to use different calculation formulas.

  The operation unit of the listener is not limited to the configuration as long as it can be driven at the operation timing and can provide tactile sensory information to the listener, like the operation unit of the speaker described above. From now on, the motion unit is one of or a combination of a configuration in which a pulling motion is given to a listener, a configuration in which a pressing motion is given to a listener, a configuration in which a vibration is given to a listener, or a configuration in which a wind is given to a listener Can be used.

  The system using the physical pull-in method appealing to the sense of touch of the present invention can be configured as follows. In other words, in a conversation where the other party is in front of you or a scene where the number of speakers or listeners is unspecified or many, this system provides the speaker with a physical pull-in phenomenon to the speaker, and gives tactile sensory information to the speaker Unit, a drive unit that drives the operation unit, a calculation unit that calculates the operation timing of the drive unit, and a signal input unit that captures the voice signal of the speaker into the calculation unit. When the predicted motion value estimated from the speech signal of the speaker captured from the speaker exceeds the predetermined motion threshold, the drive unit calculates the motion timing, and the drive unit drives the motion unit at the motion timing to sense a tactile sensation. A physical retraction system that tells the speaker the timing of movement as information and appeals to the sense of touch that brings the physical retraction phenomenon to the speaker. It is.

  The operation unit of the physical pull-in system is an object that can be sensed by tactile sense, the drive unit is a motor or cylinder that drives the object, the calculation unit is a computer, and the signal input unit is an input such as a microphone connected to the computer. Consists of interfaces. Here, the drive unit means a part that drives an operation unit. For example, in a small vibration motor, a vibration weight may be considered as an operation unit and the motor may be considered as a drive unit, or a motor equipped with a vibration weight is integrated. A portion that sends a drive signal to the motor as an operation unit may be considered as a drive unit. Even in such a physical entrainment system, the motion prediction value may be estimated by a moving average of a speaker's voice signal. In this case, for example, by using the above Equation 1 or Equation 2, the moving average can reduce the time interval for calculating the motion prediction value while reducing the burden on the calculation unit, and can provide detailed motion timing. Calculation is possible.

  The specific motion unit is configured to give the speaker a pulling motion, for example, a configuration in which a speaker's finger is inserted into a ring that repeatedly moves by a motor or a cylinder and pulled repeatedly by the ring, or a pressing motion is given to the speaker Configuration, for example, a configuration in which a speaker's finger is inserted into a ring that repeatedly expands and contracts by a motor or a cylinder and the ring is repeatedly tightened by a ring, a configuration that gives vibration to the speaker in possession, for example, an eccentric weight within a predetermined case by a motor A configuration in which the speaker's hand that rotates and grasps the predetermined case feels vibration, or a configuration that gives wind to the speaker in possession, for example, a fan that changes the rotation speed intermittently or periodically by a motor. The structure which makes wind feel can be illustrated.

  In addition, a system that provides the listener with a physical pull-in phenomenon to the listener in conversations where the other party is in front of you, or in situations where the number of speakers or listeners is unspecified or large, and provides tactile sensory information to the listener Unit, a drive unit that drives the operation unit, a calculation unit that calculates the operation timing of the drive unit, and a signal input unit that captures the voice signal of the speaker into the calculation unit. When the predicted motion estimated from the speech signal of the speaker captured from the speaker exceeds the predetermined listener motion threshold, it is calculated as the motion timing, and the drive unit drives the motion unit at the motion timing to sense the listener Physical pull-in system that teaches the tactile sensation that tells the listener the physical pull-in phenomenon to the speaker by teaching the timing of movement as realistic sensory information Arm also.

  The operation unit of the physical pull-in system is an object that can be sensed by tactile sense, the drive unit is a motor or cylinder that drives the object, the calculation unit is a computer, and the signal input unit is an input such as a microphone connected to the computer. Consists of interfaces. Here, the drive unit means a part that drives an operation unit. For example, in a small vibration motor, a vibration weight may be considered as an operation unit and the motor may be considered as a drive unit, or a motor equipped with a vibration weight is integrated. A portion that sends a drive signal to the motor as an operation unit may be considered as a drive unit. Even in such a physical entrainment system, the motion prediction value may be estimated by a moving average of a speaker's voice signal. In this case, for example, by using the above Equation 1 or Equation 2, the moving average can reduce the time interval for calculating the motion prediction value while reducing the burden on the calculation unit, and can provide detailed motion timing. Calculation is possible.

  The specific motion unit is configured to give the listener a pulling motion, for example, a configuration in which the finger of the listener is inserted into a ring that is repeatedly moved by a motor or a cylinder and pulled repeatedly by the ring, or a pressing motion is given to the listener Configuration, for example, a configuration in which a listener's finger is inserted into a ring that repeatedly expands and contracts by a motor or a cylinder and the ring is repeatedly tightened by the ring, or a configuration that gives vibration to the listener in possession, for example, an eccentric weight within a predetermined case by a motor. A configuration in which the listener's hand that rotates and grasps the predetermined case is perceived to vibrate, or a configuration that imparts wind to the listener in possession, for example, a fan that changes the rotation speed intermittently or periodically by a motor. The structure which makes wind feel can be illustrated.

  According to the present invention, even in a conversation in which the other party is in front of you, or in a scene where the number of speakers or listeners is unspecified or many, non-verbal that causes a physical pull-in phenomenon for each speaker or listener without requiring a change of line of sight Tactile sensory information can be given as information, and an appropriate physical pull-in phenomenon can be brought about for each speaker or listener. As a result, the speaker and the listener do not have to take their eyes away from the other party, so that the speaker is easy to talk and the listener is easy to hear the speaker.

  In particular, since tactile sensory information is given, a physical pull-in phenomenon can be caused to a speaker or a listener even when the surroundings are dark or the scene has poor visibility. As a result, in television broadcasting, lectures, street speeches, presentations, etc., the speaker is easy to talk, and the listener can listen to the broadcast contents, lecture contents, speech contents, or presentation contents more effectively.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of a physical pull-in system in which a speaker and a listener have vibrators 10 and 20 respectively, and FIG. 2 is a block diagram of the physical pull-in system. This example assumes a case where there is a one-to-one speaker and listener. On the other hand, when there are a plurality of speakers or listeners, the number of vibrators 10 and 20 to be held is increased, and therefore, here, only a case where there is a one-to-one speaker and listener will be described.

  As shown in FIGS. 1 and 2, the physical pull-in system of this example is such that only the vibrators 10 and 20 that are held by the speaker and the listener are separate, and the others are combined. Specifically, the voice of the speaker is picked up by the pin microphone 11 and wirelessly transmitted to the transmission / reception device 30 and captured by the computer 31. The computer 31 calculates the operation timing from the voice, and the drive signal is matched to the operation timing. The data is transmitted from the transmitting / receiving device 30 to each of the vibrating bodies 10 and 20 of the speaker and the listener to give vibration to the speaker or the listener. By connecting the pin microphone 11 and the computer 31 and the computer 31 and the vibrators 10 and 20 wirelessly, the complexity of the wired system configuration can be avoided, and the number of speakers and listeners can be easily increased and decreased. A system configuration suitable for lectures where people can enter and exit freely is possible.

  The operation timing is calculated from the voice of the speaker captured from the pin microphone 11 that is a signal input unit. Although the operation timings of the vibrators 10 and 20 with respect to the speaker or the listener can be made different, it is easier to share the body rhythm of the speaker and the listener when the operation timings of the vibrators 10 and 20 are the same. Therefore, in this example, without distinguishing between the speaker and the listener, the motion timing calculated by comparing the motion prediction value estimated by a single algorithm with a single motion threshold is determined for each vibrator 10 of the speaker or the listener. 20 operation timings.

  The calculation unit is configured by the computer 31. As described above, in this example, since the estimation of the motion prediction value for the speaker and the listener uses a single algorithm and is compared with a single motion threshold, it relates to the calculation of the motion timing for each of the speaker and the listener. There is no time difference. The computer 31 executes a program constituting the calculation unit and calculates an operation timing. As described above, since the calculation unit is configured by executing a program on the computer, for example, when using the same calculation formula for calculating the operation timing for the speaker and the listener, different prediction coefficients and operation thresholds are used, or when the speaker and the listener are used. Even when different calculation formulas are used to calculate the operation timing for the speaker, the calculation units for the speaker and the listener can be easily configured.

  The computer 31 transmits a drive signal to the vibrators 10 and 20 possessed by the speaker or the listener via the transmission / reception device 30. The vibrating bodies 10 and 20 of the present example are provided in the substantially cylindrical cases 12 and 22, respectively, by driving units 13 and 23 that are driven by receiving the driving signal, and the driving units 13 and 23 are driven by the driving units 13 and 23. Built-in motion units 14 and 24 that vibrate the entire 20. As described above, in this example, the operation timings for the speaker and the listener are matched, and the vibrators 10 and 20 possessed by the speaker and the listener have the same configuration. Therefore, actually, the vibrator 10 and the listener possessed by the speaker are actually the same. Therefore, it is not necessary to distinguish and handle the vibrating body 20 possessed by the. From the above, in the physical pulling system of the present invention, if a vibrating body or the like having the same configuration is actually possessed without distinguishing between the speaker and the listener, a physical pulling phenomenon can be brought about for each of the speaker and the listener.

  The physical retraction system of this example works as follows. When the speaker starts speaking, the listener obtains the gesture of the speaker (non-verbal information) as visual sensory information while listening to the speaker's voice (verbal information), thereby causing a physical pull-in phenomenon. However, if the listener turns away from the speaker, the visual sensory information is drastically reduced, and there is a possibility that the physical pull-in phenomenon will not be brought about smoothly. However, if the physical pull-in system of this example is used, regardless of the line of sight of the listener, the vibrating body 20 teaches the listener by vibration at a specific time point on the time series where the listener's reaction is estimated to appear prominently. Non-verbal information that the listener lacks is compensated for, and the physical pull-in phenomenon is brought about smoothly. When the listener does not look away from the speaker, tactile sensory information is given in addition to visual sensory information, so that a physical pull-in phenomenon is more positively brought to the listener.

  The speaker obtains the listener's reaction (non-verbal information), thereby causing a physical pull-in phenomenon. Therefore, if the listener's reaction is not sufficient, the physical pull-in phenomenon will not be brought smoothly to the speaker. Even in this case, if the physical pull-in system of this example is used, the vibrating body 10 teaches the speaker by vibration at a specific point in time series where it is estimated that the listener's response will appear prominently regardless of the listener's response. Can compensate for non-verbal information that the speaker lacks, and the physical pull-in phenomenon is smoothly brought to the speaker. In addition, because the physical pull-in system of this example brings the physical pull-in phenomenon to the listener, the listener's response is good and the speaker receives this good listener's response as visual sensory information. Thus, the physical pull-in phenomenon is better brought to the speaker.

  In addition, in this example, since the operation timings for the speaker and the listener are matched and tactile sensory information is given by the vibrators 10 and 20 having the same configuration, there is an advantage that sharing of the body rhythm is facilitated. As is clear from the above, visual sensory information and tactile sensory information are not mutually exclusive, but rather synergistic. For example, the speaker and the listener are physically attracted. When a conversation is made on a videophone on which a character causing a phenomenon is displayed, the same operation unit is held by the speaker and the listener in addition to the character displayed on each videophone, thereby making the conversation easier. As described above, the present invention has an effect of efficiently bringing a physical pull-in phenomenon to a speaker or a listener.

  Next, in order to confirm how much the physical retraction system that appeals to the tactile sensation of the present invention brings about a physical retraction phenomenon to the speaker, we actually created motion units that give several types of tactile sensory information, The sensory evaluation test to confirm was implemented. In this sensory evaluation test, visual sensory information and tactile sensory information will be used in combination with a display-displayed character that causes physical pull-in phenomenon by the previously proposed visual sensory information. The synergistic effect was also confirmed.

  In this sensory evaluation test, a physical pull-in system consisting only of a speaker side device is used for the sake of simplicity of the test. In practice, a similar device is also placed on the listener side to use as a symmetrical physical retraction system as seen in FIG. The symmetrical physical retraction system provides visual sensory information to a speaker or listener by a character on the display that moves in place of the other party, thereby causing a physical retraction phenomenon. The character has a structure imitating a human being, and in addition to a whispering action, a blinking action, and a mouth opening / closing action, it moves the neck, waist, shoulder, elbow or wrist joint to perform an overall gesture action.

  In this sensory evaluation test, a motion unit that gives tactile sensory information is added to the physical pull-in system consisting only of the device on the speaker side, and the visual sensory information by the character on the display for the speaker, The tactile sensory information by the motion unit is given to the speaker. The combined use of such visual sensory information and tactile sensory information is expected to have the following effects.

  First, it is considered that non-verbal information that can be given to a speaker can be increased by using a character that gives visual sensory information in combination with a motion unit that gives tactile sensory information. This is an effect (synergistic effect) of increasing the total amount of non-verbal information by combining visual sensory information and tactile sensory information.

  Second, tactile sensory information is thought to make it easier for a speaker to imagine the existence of another virtual listener and to cause a physical pull-in phenomenon. This is an effect (complementary effect) obtained by complementing visual sensory information with tactile sensory information. That is, by providing tactile sensory information to the speaker, it is considered that the speaker can recognize a virtual other listener other than the character, and this speaker can be easily brought into a physical pull-in phenomenon. As for the listener, it is likely that the listener will be able to recognize the existence of another virtual listener other than his / her own, and this will be likely to cause a physical pull-in phenomenon.

  Thirdly, even if visual sensory information is impaired, if tactile sensory information exists, it is considered that a physical pull-in phenomenon is brought to the speaker. This is a direct effect (inherent effect) by providing tactile sensory information that allows the speaker and listener to recognize that they share a physical rhythm in conversation. From this, it can be considered that, in some cases, if only tactile sensory information can be handled instead of visual sensory information, communication between a remote speaker and a listener can be achieved.

  FIG. 4 shows a specific configuration of the apparatus on the speaker side, and FIG. 5 shows a usage state of the apparatus on the speaker side. In FIG. 4, the operation unit is shown as a vibrating structure = vibration type. However, as will be described later, in this sensory evaluation test, there is also a purpose of comparing various tactile sensory information. 4 types of motion units of type, vibration type and wind type are used, and each motion unit is estimated to have a noticeable reaction of the listener by pulling, squeezing, vibrating or blowing on the hand or finger of the speaker. Teach a speaker a specific point in time as tactile sensory information.

  The speaker's voice is captured into the computer by a microphone. In the computer, a motion prediction value is estimated from the captured speaker's voice signal, and a program for calculating motion timing is executed by comparing with a predetermined motion threshold value, thereby constituting a calculation unit. Then, the computer outputs a drive signal at the calculated operation timing, sends the drive signal to the drive unit (actuator) via the USB interface (USB I / O), and this drive unit drives various operation units. .

  The computer has a AMD K6 266MHz CPU and 128MB memory, and is equipped with a sound card for capturing the speaker's voice from the microphone and a USB port for outputting drive signals. The program (executable file) constituting the calculation unit is about 300 KB, and the program is executed on Microsoft Windows (registered trademark) 2000. The USB interface uses Cypress Semiconductor's CY7C63001A-PC (USB 1.1 standard, Low-Speed type 1.5 Mbps) as the USB controller microcomputer, and is powered by + 5V from the computer's USB port. ing. The drive unit is controlled using a general-purpose USB I / O manufactured by Morphee Planning.

  The predicted motion value estimated from the speech signal of the speaker is expressed by Equation 1 described above as a linear combination of deriving the degree of influence of the speech signal of the speaker acquired in the past for a certain time range from the present to the present. It is estimated by the moving average of the speaker's voice signal using the calculated formula. Here, the speaker's voice signal x (ij) uses a value in units of 1/30 sec, assuming that the signal in the past 2 sec from the present time is divided into 60, that is, J = 1-60. In this sensory evaluation test, a predetermined constant is used for the prediction coefficient a (j) at a time interval of 1/30 sec. In an actual physical pull-in system, the motion prediction value may be estimated based on the number 2 to which the noise w (i) is added.

  The physical pull-in system used in this sensory evaluation test gives the speaker both visual sensory information and tactile sensory information, so that both of them are physically operated or driven at the same operation timing. Therefore, the motion prediction value for calculating the motion timing of the body motion of the character that gives visual sense information to the speaker, and the motion prediction value for calculating the motion timing of the motion unit that gives tactile sensory information to the speaker are: The same value is used.

  Specifically, the body motion of a character is roughly divided into a whispering motion, a blinking motion, and a gesture motion, and the whispering motion is executed at the motion timing calculated by comparing the motion prediction value with the motion threshold value. The gesture timing is executed with an exponential distribution starting from the motion timing, and the gesture motion is performed at the motion timing calculated by comparing the predicted motion value with a threshold value lower than the motion threshold value of the motion motion. In contrast, since the motion unit has one motion mode, the motion prediction value is only driven at the motion timing calculated by comparing the motion predicted value with the motion threshold value.

  There are four types of motion units that provide tactile sensory information prepared in this sensory evaluation test as shown in FIGS. There are a variety of specific means for giving tactile sensory information to the speaker, so pulling motion (Fig. 6), pressing as tactile sensory information that is easy to feel when held in the hand A motion unit giving tactile sensation information of motion (FIG. 7), vibration (FIG. 8) and air blowing (FIG. 9) was created. Each motion unit uses the drive part of a robot (Masudaya Corporation's “Nozuki-kun”) that performs physical motion that causes physical pull-in.

  As shown in FIG. 6, the pressing-type motion unit is formed by hanging a rubber string around a motion unit that swings and hooking the rubber cord around the speaker's finger. It is the structure which gives a pulling motion. The pulling movement is considered to be a representative movement that attracts human attention, and is promising as tactile sensory information that brings about a physical pull-in phenomenon.

  As shown in FIG. 7, the pressing-type operation unit contains the operation unit for rotating the cam in the plastic bottle, and the rubber string connected to the cam is taken out from the plastic bottle and hung around the finger of the speaker. In this configuration, the rubber cord is squeezed into the finger of the speaker who hangs the rubber cord to give a pressing motion by pulling or loosening the rubber cord by rotating the cam. The drawing with the rubber string is similar to the above-described pulling motion, but pulling the rubber string gives a feeling of pressing to the speaker's finger by shortening the length of the rubber string.

  As shown in FIG. 8, the vibration-type operation unit includes a small vibration motor (CM05J manufactured by TPC Co.) as an operation unit in a plastic bottle, and the whole plastic bottle is vibrated by driving the operation unit. In this configuration, the speaker's hand holding the plastic bottle is vibrated. Such tactile sensory information by vibration is widely used as a substitute for a ring tone of, for example, a mobile phone, and since it is familiar, it is promising as tactile sensory information that causes a physical pull-in phenomenon.

  As shown in FIG. 9, the wind-type motion unit sends a wind from the fan to the speaker's hand holding the housing by housing the motor in a stick-like housing as the motion unit and rotating the fan by the motor. It is a configuration. Since the tactile sensory information by blowing is accompanied by cold, it is considered that more natural tactile sensory information can be given compared to physical movement such as pulling, pressing or vibration. In this sensory evaluation test, since the power supply to the motor is relied on USB in consideration of other operation units, a quick response of fan rotation and stop cannot be expected. Therefore, the fan is always rotated at a low speed and switched to a high speed rotation according to the operation timing.

  Table 1 shows the specifications of each motion unit together with the driving time of the body motion of the character. Here, the activation time is a delay time from the reference time when the reference time (0 msec) is a time point at which the character actually starts the whispering action. As can be seen from Table 1, since each operation unit has a different driving means and mode, the activation time is different, and the influence of the length of the activation time is also evaluated. The character operates at 33 msec (30 fps) per frame.

  First, it was confirmed that the physical pull-in phenomenon was brought about to the extent that the speaker felt in any movement unit. However, in detail, it was found that there is a difference in physical pull-in phenomenon caused by the difference in activation time of each motion unit. Specifically, it seems that the longer the activation unit, the more the sense of discomfort is felt between the movement of the body, including the character's whirling movement, and the driving of each movement unit. This means that when different sensory information is given to the speaker, if the sensory information is shifted, the sensory information is not synergized but offset. From now on, it is effective to use a motion unit that gives tactile sensory information alone, but when used in combination with a character that gives visual sensory information, it is necessary to prevent deviations in both sensory information. I understood it.

  Therefore, based on the results of the sensory evaluation test, a configuration of a physical pull-in system as shown in FIG. 10 was considered. This physical pull-in system has a configuration in which a delay circuit is added between the computer and the display in order to cause the character to physically move by delaying by a delay time for starting the motion unit. One of the reasons why the motion unit is driven with a delay compared to the physical motion of the character is the presence of a drive unit that provides mechanical motion. The delay circuit can be regarded as an additional circuit that brings a delay caused by the drive unit to the body motion of the character.

  As a result of performing the same sensory evaluation test using the physical pull-in system including this delay circuit, the shorter the activation time of each motion unit, that is, the smaller the difference between the character's physical motion and the drive of each motion unit. In addition to the physical pull-in phenomenon that was not uncomfortable, the answer was that it was easier to pull in compared to the case of the character's physical movement alone. As a result, it has been found that physical pull-in phenomenon is brought about by tactile sensory information, and that physical pull-in phenomenon is brought about better by using visual sensory information and tactile sensory information together.

  The present invention provides a robot with respect to a physical retraction system in which visual sensation information is given to a speaker or a listener by a physical movement of a character displayed on a robot or a display, and a physical retraction phenomenon is given to the speaker or the listener. We propose the effectiveness of motion units that provide tactile sensory information that can be used in place of characters and characters, and the effectiveness of motion units that provide tactile sensory information that can be used with the robot or character.

  Specifically, when the talking speakers and listeners converse with each other, if both have a unit of action and have a conversation, the driving of each unit of action becomes a so-called “hand of hand”, making it easier for the speaker to speak. Give the listener an easier listening experience so that they can have a closer conversation. In addition, the physical pull-in system found in the sensory evaluation test can be directly applied to a videophone for a speaker and a listener who are remote persons.

  In addition, in situations where the number of speakers or listeners is unspecified or large, specifically in TV broadcasting, lectures, street speeches or presentations, each speaker and listener possesses a unit of motion, so that all speakers and listeners are physically You can share rhythms. Here, the difference from the physical pull-in phenomenon by visual sensory information is that the line of sight of the speaker or listener is not constrained. For example, in a presentation, if you imagine a scene where a speaker and a listener are looking at a display screen that displays the contents of the presentation, you can understand the superiority of the physical pull-in phenomenon brought about by tactile sensory information. Thus, the present invention has a feature that expands the range of use of the physical pull-in phenomenon.

It is a block diagram of the physical drawing-in system comprised by having a speaker and a listener each possess a vibrating body. 1 is a block diagram of the same physical retraction system. It is a block diagram of the physical retraction system in which the apparatus on the speaker side and the apparatus on the listener side are arranged symmetrically. It is a block diagram of the apparatus by the side of the concrete speaker in a physical retraction system. It is a photograph showing the usage state of the apparatus on the speaker side. It is a photograph of the pull-type motion unit actually produced. It is a photograph of the actual pressing type motion unit. It is a photograph of the vibration type motion unit actually produced. It is a photograph of the wind-type motion unit actually produced. It is a block diagram of the apparatus by the side of the speaker in the physical retraction system which added the delay circuit.

Explanation of symbols

10 Speaker's vibrator
11 pin microphone
12 Substantially cylindrical case
13 Drive unit
14 operation units
20 Listener vibration body
22 Substantially cylindrical case
23 Drive unit
24 operation units
30 Transceiver
31 computers

Claims (24)

In a conversation with the other party in front of you, or in a scene where the number of speakers or listeners is unspecified or many, the speaker is the unit of motion driven by the drive unit at the operation timing calculated by the calculation unit when bringing the physical pull-in phenomenon to the speaker to the speaker. The calculation unit calculates a time point when the predicted motion value estimated from the speaker's voice signal exceeds a predetermined motion threshold as a motion timing, and the drive unit drives the motion unit at the motion timing and senses the touch. A physical pull-in method that appeals to the tactile sensation that brings the speaker with a physical pull-in phenomenon to the listener by teaching the timing of movement to the speaker as realistic sensory information. The tactile physical pull-in system according to claim 1, wherein the motion prediction value is estimated by a moving average of a speaker's voice signal. The physical pull-in method for tactile sensation according to claim 1, wherein the motion unit applies a pulling motion to the possessed speaker. The physical pull-in method for tactile sensation according to claim 1, wherein the motion unit applies a pressing motion to the possessed speaker. The physical pull-in method for tactile sensation according to claim 1, wherein the motion unit applies vibration to a speaker possessed. The physical pull-in method for tactile sensation according to claim 1, wherein the unit of motion gives wind to a possessed speaker. In a conversation where the other party is in front of you, or in a scene where the number of speakers or listeners is unspecified or many, the listener will hear the unit of motion driven by the drive unit at the timing calculated by the calculation unit when bringing the physical pull-in phenomenon to the speaker. The calculation unit calculates a time point when the predicted motion value estimated from the speaker's voice signal exceeds a predetermined motion threshold as a motion timing, and the drive unit drives the motion unit at the motion timing and senses the touch. A physical pull-in method that appeals to the tactile sensation that brings the listener to the physical pull-in phenomenon of the speaker by teaching the listener the movement timing as realistic sensory information. The physical pull-in system that appeals to the sense of touch according to claim 7, wherein the motion prediction value is estimated by a moving average of a listener's voice signal. The physical pull-in method for tactile sensation according to claim 7, wherein the motion unit gives a pulling motion to a possessed listener. The physical pull-in method for tactile sensation according to claim 7, wherein the motion unit gives a pressing motion to the listener possessed. The physical pull-in method for tactile sensation according to claim 7, wherein the motion unit applies vibration to a listener possessed. The physical pull-in method for tactile sensation according to claim 7, wherein the motion unit gives wind to a listener possessed. A system that provides the speaker with a physical pull-in phenomenon to the speaker in a conversation where the other party is in front of the user, or where the number of speakers or listeners is unspecified or many, and a unit of action that gives tactile sensory information to the speaker A driving unit that drives the operation unit, a calculation unit that calculates an operation timing of the driving unit, and a signal input unit that captures a speaker's voice signal into the calculation unit, and the calculation unit is captured from the signal input unit. When the predicted motion value estimated from the voice signal of the talker exceeds a predetermined motion threshold, the motion timing is calculated as motion timing, and the drive unit drives the motion unit at the motion timing as tactile sensory information. A physical retraction system that teaches the timing of movement to the speaker and appeals to the sense of touch that brings the physical retraction phenomenon to the speaker. 14. The tactile physical pull-in system according to claim 13, wherein the motion prediction value is estimated by a moving average of a speaker's voice signal. 14. The physical retraction system for tactile sensation according to claim 13, wherein the motion unit gives a pulling motion to the possessed speaker. 14. The physical retraction system for tactile sensation according to claim 13, wherein the motion unit applies a pressing motion to the possessed speaker. 14. The physical retraction system for tactile sensation according to claim 13, wherein the motion unit applies vibration to a speaker possessed by the motion unit. 14. The physical pull-in system for tactile sensation according to claim 13, wherein the motion unit gives wind to a possessed speaker. A system that provides the listener with a physical pull-in phenomenon to the listener in conversations where the other party is in front of the audience, or where the number of speakers or listeners is unspecified or numerous, and a unit of motion that provides tactile sensory information to the listener A driving unit that drives the operation unit, a calculation unit that calculates an operation timing of the driving unit, and a signal input unit that captures a speaker's voice signal into the calculation unit, and the calculation unit is captured from the signal input unit. The motion prediction value estimated from the speaker's voice signal is calculated as a motion timing when the predicted motion value exceeds a predetermined listener motion threshold, and the drive unit drives the motion unit at the motion timing to provide a tactile feel to the listener. A physical retraction system that teaches movement timing as sensory information and appeals to the sense of touch that brings the physical retraction phenomenon to the listener. 20. The tactile physical pull-in system according to claim 19, wherein the motion prediction value is estimated by a moving average of a listener's voice signal. 20. The physical retraction system for tactile sensation according to claim 19, wherein the motion unit gives a pulling motion to the listener possessed. 20. The physical retraction system for tactile sensation according to claim 19, wherein the motion unit gives a pressing motion to the listener possessed. 20. The physical retraction system for tactile sensation according to claim 19, wherein the motion unit applies vibration to a listener possessed. 20. The physical retraction system for tactile sensation according to claim 19, wherein the motion unit gives wind to a listener possessed.