JP3566646B2 - Music communication device - Google Patents

Description

[0001]
[Industrial applications]
The present invention relates to a music communication device, and more particularly to a music communication device applied to play equipment for, for example, infants or elderly people.
[0002]
[Prior art]
Heretofore, there has been no such music communication device. As a similar prior art, when a specific part of a stuffed animal replaced with a character such as a doll or animal imitating a human figure is touched or moved, sound or music is output, or the specific part moves in a certain manner. There was a toy like that.
[0003]
[Problems to be solved by the invention]
However, in such a conventional technique, when a specific part of a doll or a stuffed animal is touched or moved, a fixed voice (a message such as a greeting) or a fixed music recorded in advance is output, or the specific part is fixed. There was a problem that the game was lacking in play because the movement was repeated. For this reason, the infant was quickly bored.
[0004]
Therefore, a main object of the present invention is to provide a novel music communication device having excellent playability.
[0005]
[Means for Solving the Problems]
The present invention relates to a plurality of sensors for detecting a user's expression.A music communication device that controls a plurality of musical elements by input from a plurality of sensors,Depending on the detection results of multiple sensorsRole setting means for presetting the role of each sensor with respect to a plurality of music elements for each of a plurality of determined interaction state levels, wherein the role of at least one sensor is different for each interaction state level; State setting means for setting any one of a plurality of interaction state levels according to the detection result of the sensor; andAccording to the input from the sensorControlled musical elementsA music communication device including an output unit that outputs a sound.
[0006]
[Action]
In the music communication device according to the present invention, a plurality of sensors for detecting a user's expression, that is, an action, a behavior, or a voice.( The corresponding parts in the examples are 14-54 It is. The same applies hereinafter. )Is provided.Role setting means (66, 66b) However, for example, an interaction state level determined according to the detection result of each sensor and the roles of a plurality of sensors in each state level are set in a memory such as a ROM in advance, and the state setting means is set. (62, S1, S11, S25, S39, S51) Sets any one of the interaction state levels according to the detection results of the plurality of sensors. OutPower means(62, 66, 66a, S9, S23, S37, S49, S61, 24)ButAt that set interaction state levelOutputs sound according to the input from the sensor. In other words, it is possible to respond and communicate with the user's expression by outputting a voice, a sound such as a chord, or music.
[0008]
Further, the interaction state (interaction state) divided into predetermined stages is set to a predetermined stage (level), and the level of the interaction state is maintained or transited according to the input from the sensor, that is, the detection result, and the interaction state is set. The role may be changed when the level is changed.
[0009]
Further, the sound changing means may be configured to control a musical element (for example, scale, chord, rhythm, melody, etc.) in accordance with an input from the sensor, that is, a detection result.)Since the change is made, not only a simple voice but also a sound, a chord, or music along a certain scale can be output, and the volume can be adjusted. That is, depending on the expression of the user, that is, the manner of contacting the user, it is possible to achieve a pleasant communication in which music is played.
[0010]
Specifically, the scale changing means changes the scale according to the input from the sensor, and the code changing means changes the code according to the input from the sensor. The rhythm changing means changes the rhythm according to the input from the sensor, and the melody changing means changes the melody according to the input from the sensor. Such musical elements may be changed individually or in a combined manner.
[0011]
For example, such a music communication device has a stuffed body in which a plurality of sensors are arranged at predetermined portions as a main body, and each of the plurality of sensors detects a user's expression through the stuffed toy. That is, the user can feel as if communicating with the stuffed character.
[0012]
【The invention's effect】
According to the present invention, a sound corresponding to an input from a plurality of sensors is output, and the role of the sensor is changed according to the expression of the user, so that the user does not get bored immediately. That is, the playability is high.
[0013]
The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.
[0014]
【Example】
Referring to FIG. 1A, a music communication device 10 of this embodiment includes a stuffed toy 12, and the stuffed toy 12 is formed in a shape imitating an animal or a character. However, in this embodiment, the stuffed toy 12 has a shape resembling a bear, for example, and has a head 12a, an ear 12b, a nose 12c, a mouth 12d, a body 12e, a tail 12f, a left arm 12g, and a right arm. 12h, a left leg 12i, a right leg 12j, and the like. A plurality of sensors to be described later are provided (built-in) in each of such parts (12a to 12j).
[0015]
1A and 1B, each sensor is shown on the surface (outside) of the stuffed toy 12 for simplicity, and some sensors are omitted.
[0016]
More specifically, the head 12a of the stuffed toy 12 is provided with a head touch sensor 14 arranged to be in contact with the inner surface of the skin of the stuffed toy 12. As the head touch sensor 14, a piezo-type pressure sensor that can detect at least whether or not a contact has been made and a continuous change in force (pressing force) is applied. A microphone 16 as an audio sensor is provided on an ear 12b provided continuously to the head 12a. Further, the nose portion 12c is provided with a CCD camera (hereinafter simply referred to as "camera") 18 as an image sensor and a head proximity sensor 20 such as an infrared sensor.
[0017]
However, the output of the infrared sensor is normally on, and the output is stopped when a person or an object is detected.
[0018]
The mouth 12d disposed below the nose 12c is provided with the mouth touch sensor 22 including the same pressure sensor as the head touch sensor 14 described above. Similarly, the mouth 12d is filtered along one note (do, re, mi, fa, ...) or one note not along the note (sound with ♯ or ♭) or a different note. A speaker 24 is provided for outputting sounds such as voices, chords, or music. This speaker 24 is also shown on the surface of the stuffed animal 12 for easy explanation in FIG. 1A, but is actually provided inside the stuffed animal 12.
[0019]
However, the camera 18 is arranged such that a lens (not shown) is exposed from the stuffed animal 12. Similarly, although not shown, the head proximity sensor 20 is also arranged such that the detection unit (light-emitting unit / light-receiving unit) of the infrared sensor is exposed from the stuffed animal 12.
[0020]
The body 12e is provided with an abdomen touch sensor 26, a gyro (accelerometer) 28, and an internal temperature sensor 30 having the same configuration as the head touch sensor 14 described above. As can be seen from FIG. 1B, the torso portion 12e is also provided with a back proximity sensor 34 composed of the same back touch sensor 32 as the head touch sensor 14 described above and an infrared sensor same as the head proximity sensor 20 described above. Is provided. In addition, a butt portion proximity sensor 36 that is the same as the head proximity sensor 16 is provided at the butt portion included in the body portion 12e.
[0021]
However, the back proximity sensor 34 and the buttocks proximity sensor 36 are arranged such that the detection unit of the infrared sensor is exposed from the stuffed animal 12.
[0022]
Further, an external temperature sensor 38 is provided on a tail portion 12f continuously provided on the body portion 12e. The external temperature sensor 38 is arranged so that at least a temperature detecting unit (not shown) is exposed from the tail part 12f. Also, as shown in FIGS. 1A and 1B, the left arm 12g, the right arm 12h, the left leg 12i, and the right leg 12j determine whether or not each part (12g to 12j) is bent. Each of a left arm bend 40, a right arm bend 42, a left leg bend 44, and a right leg bend 46 using a piezo-type bend capable of detecting a bending angle is provided.
[0023]
However, as can be clearly understood from FIGS. 1A and 1B, each of the bends (40 to 46) is provided so that one end thereof extends to the body portion 12e.
[0024]
As shown in FIG. 1A, the left arm 12g, the right arm 12h, the left leg 12i, and the right leg 12j have the left arm touch sensor 48, the right arm touch sensor 50, and the left leg Each of a touch sensor 52 and a right leg touch sensor 54 is provided.
[0025]
The electrical configuration of the music communication device 10 is shown as in FIG. That is, the music communication device 10 includes the computer 60, and the above-described sensors (14 to 22, 26 to 54) and the speaker 22 are connected to the computer 60.
[0026]
The computer 60 includes a CPU 62, and the CPU 62 is connected to each of the ROM 66, the RAM 68, the counter 70, and the timer 72 via the internal bus 64.
[0027]
Although not shown in FIG. 2, each sensor (excluding the microphone 16 and the camera 18) is connected to the internal bus 64 via an A / D converter and an interface. Although not shown, the microphone 16 is connected to the internal bus 64 via an amplifier, an A / D converter, and an interface. Further, although not shown, the camera 18 is connected to the internal bus 64 via an interface. Further, although not shown, the speaker 24 is connected to the internal bus 64 via an amplifier, a D / A converter, and an interface.
[0028]
The ROM 66 includes memory areas 66a to 66c. In the memory area 66a, the level of the interaction state (interaction state) of the stuffed toy 12 (music communication device 10) is maintained or transited. A program for generating a sound according to an input from the sensor is stored.
[0029]
As a program (application software) for generating a sound, “MAX / MSP” (product name) manufactured by Optocode Corporation that runs on a Mac OS can be used.
[0030]
In the memory area 66b, mapping (arrangement) of the role of each sensor according to the level of the interaction state is stored. This mapping will be described later in detail.
[0031]
Further, in the memory area 66c, data corresponding to a previously recorded voice signal corresponding to a human voice (voice) is stored.
[0032]
However, in this embodiment, since the audio signal is filtered along different pitches (pitch N1 and pitch N2), audio signals of different pitches are simultaneously output from the speaker 24. Therefore, by controlling each of the pitch N1 and the pitch N2, a high voice or a low voice can be produced in a range from a high pitch to a low pitch.
[0033]
For example, in this embodiment, the interaction state is set (transitioned) between levels 0 to 4, and the role of each sensor is changed according to each of levels 0 to 4. That is, the above-described memory area 66b stores the arrangement of roles for each sensor as shown in FIGS.
[0034]
Specifically, when the interaction state is level 0, the role of each sensor is determined as shown in the left column of FIG. However, it means that there is no control in the column where the horizontal bar is drawn. Hereinafter, the same applies to this embodiment.
[0035]
At level 0, the stuffed animal 12 (music communication device 10) outputs a simple, unprocessed breathing sound and stops breathing when attention (interest) is drawn. Such timing control of the breathing sound, that is, output / stop of the breathing sound, is executed in accordance with the input from the microphone 16, the camera 18, and the head proximity sensor 20.
[0036]
For example, the breathing sound is always output from the stuffed toy 12 (speaker 24), and when the stuffed toy 12 (music communication 12) is interested, in this embodiment, when the person (user) approaches, the breathing sound is output. The sound is stopped. That is, the microphone 16 is always activated, and the CPU 62 detects a change in the signal level (volume) or pitch (the pitch of a musical note in music) of an audio signal input through the microphone 16, When the signal level or the frequency changes, it is determined that a person has approached. That is, when there is a human voice or when there is a noise such as opening and closing the door, it can be determined that the person has approached.
[0037]
The frequency of the audio signal may be analyzed to determine the voice of a person, and only when the voice is a human voice, it may be determined that the person has approached.
[0038]
Further, the camera 18 is always activated, and the CPU 62 detects a change in the luminance level from a video signal input via the camera 18 and determines that a person has approached when the luminance level has changed. As described above, specific examples that can determine that a person has approached include a case where the user approaches or touches the stuffed animal 12 with his or her face or another part, or a case where the lighting device in the room where the stuffed animal 12 is placed blinks. Can be considered.
[0039]
Further, the head proximity sensor 20 outputs a detection signal when the user approaches or touches the head 12a of the stuffed animal 12, particularly the nose 12c, of the stuffed toy 12 with its own face or another part. Accordingly, the CPU 62 determines that a person has approached in response to this.
[0040]
Further, the entire tempo, that is, the tempo of the breathing sound at level 0 is changed according to the input from the internal temperature sensor 30 and the external temperature sensor 38. For example, the CPU 62 detects a difference between the internal temperature of the stuffed animal 12 and the external temperature detected by the internal temperature sensor 30 and the external temperature sensor 38, and if the difference is small, slows down the tempo for outputting the breathing sound. Conversely, when the difference is large, the tempo at which the breathing sound is output can be increased. For example, if the tempo (speed) is determined within a predetermined range, the tempo can be changed by a continuous speed change according to the temperature difference.
[0041]
When the overall tempo changes, the tempo for the progress of breathing sounds, rhythm sounds, melodies, chords (chords) and keys (types of scales) changes. Further, as described above, the tempo at which the breathing sound is output is changed, but other than the breathing sound will be described in detail as appropriate, and the detailed description will be omitted here.
[0042]
Next, when the interaction state is level 1, the stuffed toy 12 (voice communication device 10) outputs a voice signal stored in the memory area 66c of the ROM 66 in advance from the speaker 22, and touches the stuffed toy 12, The user replies (replies) to the user's expression such as talking to the stuffed toy 12 or the like. Then, as the user touches the stuffed animal 12 for a longer time, the reply interval is shortened, and as the number of inputs from the sensor of the part touched by the human increases, the synchronization is gradually performed. That is, as the number of times the user touches the stuffed toy 12 increases, the time until a reply is gradually shortened.
[0043]
However, at level 1, when there is an input from a sensor to which no role is assigned, the number of inputs is only counted by the counter 70, and is not used for control (change) of the output sound. The same applies to level 2 and level 3, which will be described later.
[0044]
Specifically, the role of each sensor is indicated as shown in the right column of FIG. 3, and in response to an input from each sensor, the conversation (answer) speed, the response speed (time until reply), the volume, etc. Controlling. For example, the number of inputs from each sensor is counted by the counter 70, and the CPU 62 increases the volume of the audio signal output from the speaker 24 as the count value of the counter 70 increases. That is, as the number of times the user touches the stuffed animal 12 increases, the voice of the reply from the stuffed animal 12 increases.
[0045]
When there is an input from the left arm bend 40, the CPU 62 changes the pitch N1 of the audio signal. Specifically, the pitch N1 increases as the bending angle of the left arm 12g increases, and conversely, the pitch N1 decreases as the bending angle of the left arm 12g decreases. However, the pitch N1 can be increased or decreased by raising or lowering the left arm 12g.
[0046]
Further, when there is an input from the right arm bend 42, the CPU 62 changes the pitch N2 of the audio signal. That is, when the user bends the right arm portion 12j of the stuffed toy 12, the magnitude of the pitch N2 changes according to the bending angle. The change in the magnitude of the pitch N2 is the same as the case where the user bends the left arm 12g to the stuffed toy 12, and thus the repeated description is omitted.
[0047]
That is, the voice to which the stuffed animal 12 answers can be continuously changed from a low voice to a high voice. The low voice and the high voice are determined by the pitch N1 and the pitch N2 when the voice signal is filtered.
[0048]
Then, when there is an input from the mouth touch sensor 22, the CPU 62 increases the conversation speed. That is, when the user touches (strokes) the mouth portion 12d with the stuffed toy 12, the stuffed toy 12 starts to speak quickly.
[0049]
When there is an input from at least one sensor, the CPU 62 controls the output timing of the audio signal, that is, on / off of the output. Specifically, while the user is showing an expression to the stuffed toy 12, that is, while touching the stuffed toy 12 or speaking to the stuffed toy 12, an audio signal is output. On the other hand, when the user stops expressing the stuffed animal 12, the output of the audio signal is stopped.
[0050]
Further, the timer 72 counts (measures) a time (period) from when the level 1 is set to the present time, and the CPU 62 determines a delay time (response speed), that is, a time until a reply is made according to the timer value of the timer 72. Change the time before stopping a reply. For example, as the timer value increases, the response speed can be increased, and the response speed can be gradually synchronized (synchronized) with the user's input. That is, the response time for turning on / off the output of the above-described audio signal is defined by the delay time.
[0051]
In this way, the reaction of the stuffed toy 12 (music communication device 10), that is, the sound to be output is generated by both the interaction state and the input of the sensor.
[0052]
Subsequently, when the interaction state is at level 2, the stuffed toy 12 (voice communication 10) filters the respiration at level 0 with a pitch along a chord and outputs the result. Also, the sound at level 1 is output at a pitch along the scale. Then, the response is gradually synchronized, and a sound reminiscent of music is output, and the tempo and other musical elements of the sound are changed.
[0053]
Specifically, as shown in the left column of FIG. 4, the role of each sensor is arranged, and the CPU 62 changes each element of the overall tempo, voice, and breathing sound according to the input from each sensor. .
[0054]
If there is an input from the left arm bend 40, the right arm bend 42, the left leg bend 44, or the right leg bend 46 at intervals of one second, the entire tempo is changed. That is, the tempo increases as such input continues. That is, when the user grasps one or two of the left arm 12g, the right arm 12h, the left foot 12i, or the right foot 12j of the stuffed toy 12 and moves it in a one-second cycle, the output interval of the sound and breathing sound gradually increases. Is shortened.
[0055]
Whether or not there is an input every second can be determined by detecting the outputs of the left arm bend 40, the right arm bend 42, the left leg bend 44, and the right arm bend 46, and detecting the timer value of the timer 72. .
[0056]
As described above, the sound is almost the same as in the case of the level 1, so that the duplicated description will be omitted. That is, the point that the delay time of the sound is determined by the tempo is different from the level 1. More specifically, if the tempo is fast, the user responds quickly to the expression of the user, and if the tempo is slow, the user responds slightly later to the expression of the user.
[0057]
As for the breathing sound, the CPU 62 calculates the average value of the inputs from the left leg bend 44 and the right leg bend 46, and adjusts the volume of the breathing sound according to the magnitude of the average value. That is, when the user grips the left leg 12i and the right leg 12j of the stuffed animal 12 and moves them so that they intersect, the volume is changed according to the angle formed by the left leg 12i and the right leg 12j. You. That is, if the angle is large, the volume is increased, and if the angle is small, the volume is decreased. Further, if the cycle of moving the left leg 12i and the right leg 12j is 1 second, the output interval of the breathing sound is shortened as described above.
[0058]
Further, the number of input sensors is counted by the counter 70, and the CPU 62 adjusts the sound according to the count value of the counter 70. For example, a count value serving as a threshold value is set in advance, and while the count value of the counter 70 is lower than the set threshold value, a monaural breathing sound is output. A breathing sound can be output.
[0059]
Furthermore, if there is an input from both the left arm touch sensor 48 and the right arm touch sensor 50, that is, if the user grasps both arms of the stuffed toy 12, the CPU 62 outputs a breathing sound in which a harmony is constructed from the speaker 24. Also, a different harmony can be constructed each time there is an input from both the left arm touch sensor 48 and the right arm touch sensor 50, that is, each time the user grabs or releases both arms of the stuffed toy 12. However, if there is an input from the head proximity sensor 20, that is, if the user grasps both arms of the stuffed toy 12 and brings the user's face or torso close to the nose 12c of the stuffed toy 12, the constructed harmony is retained. (Fixed).
[0060]
When the interaction state is level 3, the stuffed toy 12 (voice communication device 10) controls a musical operation. Specifically, the role arrangement of each sensor is shown in the right column of FIG. 4, and mainly musical elements (scale, chord, melody, rhythm, etc.) are changed according to the input from each sensor. .
[0061]
However, in this embodiment, there are six types of chords (chords). There are twelve types of scales: C major, D major, E major,..., Variation E major, and Variation G major. Furthermore, there are four types of rhythms.
[0062]
As a whole, the CPU 62 changes the tempo when detecting a rhythmical (regular) input from any (arbitrary) sensor. That is, if the user grasps the stuffed toy 12 and rhythmically moves the stuffed toy 12 itself, or rhythmically moves or touches a part of the stuffed toy 12, the entire tempo increases. However, if there is an irregular input after the tempo increases, the tempo gradually decreases. When there is no input, the tempo is fixed (held) at a constant speed.
[0063]
However, the CPU 62 can determine that the stuffed animal 12 has been moved rhythmically by detecting an input from the gyro 28.
[0064]
If there is an input from the left leg bend 44, the CPU 62 increases the volume of the sound output from the speaker 24. On the other hand, if there is an input from the right leg bend 46, the CPU 62 reduces the volume of the sound output from the speaker 24. That is, when the user bends the left leg portion 12i and the right leg portion 12j of the stuffed toy 12, the volume is increased or decreased.
[0065]
Further, if there is an input from the right arm bend 42, the CPU 62 changes the code of the sound output from the speaker. As described above, in this embodiment, six types of codes are prepared, and each time there is an input from the right arm bend 42, that is, the user bends and extends the right arm 12h of the stuffed toy 12 or raises and lowers it. Each time, the code type is updated.
[0066]
At this time, the update speed of the code is changed according to the tempo described above. Specifically, when there is an input from the right arm bend 42, the code (I) is updated to the code (II) at a speed specified by the tempo. In other words, if the tempo is fast, the type of the chord is updated immediately when there is an input from the right arm bend 42, but if the tempo is slow, the type of the chord is updated slightly after the input.
[0067]
When the tempo is fast, if there is an input from the right arm bend 42, the code (II) between codes (I) to (III) can be skipped and updated.
[0068]
Further, the type of the cord can be updated according to the size of the angle at which the right arm 12h is bent or the size of the angle at which the right arm 12h is raised.
[0069]
If the input from the head touch sensor 14 continues for, for example, one second or more, the CPU 62 changes the type of the key (scale) of the sound output from the speaker 24. As described above, in this embodiment, twelve types of keys are used, and the keys are raised each time there is an input. That is, if the user keeps touching or stroking the head 12a of the stuffed animal 12 for one second or longer, the key is changed from C major to D major. When the user stops touching the head 12a, the key is fixed in D major. Furthermore, when the head 12a is touched for 1 second or longer, the key is changed from D major to E major.
[0070]
However, the speed at which the key is changed is defined by the above-described tempo. If there is an input of one second or more from the head touch sensor 14, if the tempo is fast, the key is changed immediately and the tempo is slow. In this case, the key is changed a little after the input for one second or more.
[0071]
When the tempo is fast, the key can be changed by skipping the key between C major and E major.
[0072]
When a predetermined key is determined, a voice or a breathing sound is output along the key. That is, the voice or breathing sound is filtered so as to output the sound of the key.
[0073]
Also, the sound is almost the same as the case shown at level 1, and the duplicated description will be omitted. That is, the change of the pitch N1 is different. Specifically, the left arm touch sensor 48 is enabled until 30 seconds elapse after the level 3 is set, and when there is an input from the left arm touch sensor 48, the CPU 62 changes the pitch N1 of the voice. I do. In other words, the volume is increased as the time during which the user is touching the left arm 12g is increased. Further, if 30 seconds have elapsed since the level 3 was set, the left arm bend 40 is enabled, and when there is an input from the left arm bend 40, the CPU 62 changes the pitch of the melody sound as described later.
[0074]
With respect to the breathing sound, when there is an input from the left arm touch sensor 48, that is, when the user touches or grasps the left arm 12g of the stuffed toy 12, the CPU 62 changes the volume of the breathing sound. For example, the volume can be increased when there is continuous input from the left arm touch sensor 48, or the volume can be increased as the magnitude of the pressure input from the left arm touch sensor 48 increases. Conversely, when the magnitude of the pressure input from the left arm touch sensor 48 decreases, the volume can be reduced.
[0075]
When there is an input from the abdomen touch sensor 26 or an input from the left leg bend 44 and the right leg bend 46, the CPU 62 changes the sound of the breathing sound. For example, when the user touches or strokes the abdomen of the stuffed toy 12, there is an input from the abdomen touch sensor 26, and every time the input is made, the breathing sound can be changed from monaural to stereo or from stereo to monaural. Also, when the user bends or extends the left leg 12i and the right leg 12j of the stuffed toy 12 or raises and lowers it, there are inputs from the left leg bend 44 and the right leg bend 46, and the average of those inputs is calculated. It is also possible to switch (change) between monaural and stereo depending on whether the average value exceeds a preset threshold value.
[0076]
As described above, it is possible to change the sound in accordance with the type of the sensor. However, the sound can be changed regardless of the type of the sensor, that is, the left leg bend 44 and the right leg bend 46. For example, after changing from monaural to stereo according to the input from the abdomen touch sensor 26, it is also possible to change from stereo to monaural according to the input from the left leg bend 44 and the right leg bend 46. Conversely, after changing from monaural to stereo according to the input from the left leg bend 44 and right leg bend 46, it is also possible to change from stereo to monaural according to the input from the abdomen touch sensor 46.
[0077]
Further, when there is an input from the back touch sensor 32, the CPU 62 constructs a harmony. For example, when the user touches or rubs the back of the stuffed toy 12, a harmony is established in the sound output from the speaker 24.
[0078]
When there is an input from the left arm bend 40 for the melody, the CPU 62 changes the pitch of the melody. For example, when the user bends the left arm 12g of the stuffed animal 12, the melody sound is increased, and when the left arm 12g is extended, the melody sound is decreased. However, the pitch is continuously changed according to the bending angle.
[0079]
It should be noted that even if the user raises or lowers the left arm 12g, the pitch can be continuously changed at the angle of the raising or lowering.
[0080]
Further, when there is an input from the head proximity sensor 20, that is, when the user brings the user's face or torso close to the stuffed toy 12, the CPU 62 fixes (holds) the pitch of the melody described above.
[0081]
Further, when there is an input from the right arm touch sensor 50, the CPU 62 changes the volume of the melody sound. For example, the volume of the melody sound increases each time the user touches the right arm 12h of the stuffed toy 12, or the volume of the melody sound increases as the time of touching the right arm 12h increases.
[0082]
However, the volume can be raised or lowered according to the force of gripping the right arm 12h. That is, the CPU 62 detects the magnitude of the pressure input from the right arm touch sensor 50, and increases the volume when the pressure increases, and decreases the volume when the pressure decreases.
[0083]
Further, as for the rhythm, when there is a rhythmic input from any of the sensors, the CPU 62 changes the four rhythms prepared in advance, as in the above-described tempo for the whole. For example, if the user touches, strokes, grasps, raises or lowers the stuffed toy 12, and there is a rhythmic input from any sensor, the sound is output from the speaker 24 along the rhythm (I). Further, when there is a rhythmical input, the sound is output from the speaker 24 along the rhythm (II).
[0084]
The speed of such rhythm change is defined by the overall tempo. If the tempo is fast, the rhythm changes immediately if there is a rhythmic input, but if the tempo is slow, the rhythm changes after the rhythmic input. Is changed.
[0085]
When the tempo is fast, if there is a rhythmical input, the rhythm (II) between rhythm (I) and rhythm (III) can be skipped and changed.
[0086]
In other words, when the interaction state is level 3, a musical element is operated (controlled) in accordance with the expression of the user, and a response (voice) by a simple human voice or a breathing sound is changed to a sound like a chord, In addition, music that follows the rhythm and melody will be played. That is, the user can have fun communication with the stuffed toy 12.
[0087]
Further, when the interaction state is level 4, the volume balance between the rhythm and the chord (original breathing sound) is simply adjusted, and the stuffed toy 12 (music communication device 10) is brought into a runaway state (uncontrollable state). In other words, keys and chords change in a mess (bullshit), and an unpleasant sound such as a dissonant sound is output. However, if the user soothes the stuffed animal 12, the interaction state returns to level 3 and control becomes possible.
[0088]
Specifically, as shown in FIG. 5, the role of each sensor is arranged. In the figure, the columns indicated by x indicate that the control is irrelevant.
[0089]
Regarding the whole, the CPU 62 randomly changes the type of the key according to a predetermined algorithm programmed in advance regardless of the input from each sensor.
[0090]
When there is an input from the left arm touch sensor 48 for the breathing sound, the CPU 62 changes the volume. For example, when the user touches the left arm 12g, the volume is increased. However, since the runaway state occurs at level 4, the volume may be increased when there is no input from the left arm touch sensor 48, and the volume may be decreased when there is an input from the left arm touch sensor 48.
[0091]
Further, similarly to the above-described change of the key type, the CPU 62 randomly constructs a harmony according to a predetermined algorithm programmed in advance regardless of an input from each sensor. For example, an unpleasant sound such as a dissonant sound can be output.
[0092]
Further, as for the rhythm, when there is an input from the right arm touch sensor 50, the CPU 62 increases the volume of the rhythm. However, at the level 4, since a runaway state occurs, the volume of the rhythm is increased when there is no input from the right arm touch sensor 50, and the volume of the rhythm is decreased when there is an input from the right arm touch sensor 50. You can also.
[0093]
However, when the level 4 is set as described above and a runaway state (crazy state) is reached, the user can return to the level 3 by soothing or playing the stuffed toy 12. If the runaway state occurs, the stuffed toy 12 can be left alone to return to the initial state (level 0) after a predetermined time (10 seconds in this embodiment) has elapsed. You can also.
[0094]
The level of the interaction state described above is maintained or transitioned depending on how the user touches the stuffed toy 12, and the sound arrangement according to the user's expression, that is, the input from each sensor is determined by the sensor role arrangement determined at each level. Is output.
[0095]
Specifically, the CPU 62 shown in FIG. 2 performs processing according to the flowcharts shown in FIGS. However, when the main power supply of the music communication apparatus 10 is turned on, the CPU 62 loads the program stored in the memory area 66a of the ROM 66 into the working area 68a of the RAM 68, and stores the roles of the sensors stored in the memory area 66b. Is loaded into the working area 68b, data corresponding to the audio signal stored in the memory area 66c is loaded into the working area 68c, and the following processing is executed.
[0096]
As shown in FIG. 6, when starting the process, the CPU 62 first sets the interaction state to level 0 in step S1. That is, the roles of the sensors as shown in the left column of FIG. 3 are arranged. Next, in step S3, the input from each sensor is checked.
[0097]
Subsequently, in a step S5, it is determined whether or not there is an input from each sensor. If “NO” in the step S5, that is, if there is no input, the process returns to the step S3 and checks the input again. On the other hand, if “YES” in the step S5, that is, if there is an input, in a step S7, the touch sensor (14, 22, 26, 32, 48, 50, 52, 54), the camera 18 or the head proximity sensor 20 Judge whether there is an input from. If “YES” in the step S7, that is, if there is an input from the touch sensor, the camera 18 or the head proximity sensor 20, the process proceeds to a step S11.
[0098]
However, in FIG. 6, the steps are omitted, but if the input is from the camera 18 or the head proximity sensor 20, the breathing sound is temporarily stopped and the process proceeds to step S11.
[0099]
On the other hand, if “NO” in the step S7, that is, if there is no input from the touch sensor, the camera 18 or the head proximity sensor 20, in the step S9, as described above, the output processing of the respiratory sound according to the tempo is performed. After executing, the process returns to step S3.
[0100]
In step S11, the interaction state is set to level 1 (transition). That is, each sensor is arranged in the role shown in the right column of FIG. In a succeeding step S13, an input from each sensor is checked, and it is determined whether or not the sensor is held in a step S15. Specifically, it is determined whether or not the input from the back proximity sensor 34 or the butt proximity sensor 36 has stopped.
[0101]
In this embodiment, since it is assumed that the stuffed animal 12 is placed so as to be seated or lying down on the back, a back proximity sensor 34 or a butt proximity sensor 36 is provided, and each input is checked. However, assuming that the stuffed animal 12 is placed on the prone, a proximity sensor may be further provided on the abdomen, and the input of the proximity sensor may be checked.
[0102]
If “YES” in the step S15, that is, if the input from the back proximity sensor 34 or the butt proximity sensor 36 is stopped, it is determined that the user is held up, and the process proceeds to the step S25 shown in FIG. On the other hand, if “NO” in the step S15, that is, if there is an input from the back proximity sensor 34 or the buttocks proximity sensor 36, it is determined that the user is not held up, and the process proceeds to the step S17.
[0103]
In step S17, it is determined whether there is an input from the head touch sensor 14, the abdomen touch sensor 26, the back touch sensor 32, the left arm touch sensor 48, the right arm touch sensor 50, the left leg touch sensor 52, and the right leg touch sensor 54. I do. If “YES” in the step S17, that is, if there is an input from any of the touch sensors except the mouth touch sensor 26, it is determined whether or not the input continues for 8 seconds or more in a step S19. If “YES” in the step S19, that is, if the input is continued for 8 seconds or more, the process proceeds to a step S25 shown in FIG. On the other hand, if “NO” in the step S19, that is, if the input from the touch sensors other than the mouth touch sensor 22 is less than 8 seconds, the process returns to the step S13 and checks the input again.
[0104]
If “NO” in the step S17, that is, if there is no input from the touch sensor, it is determined whether or not there is an input from a sensor other than the touch sensor (excluding the mouth touch sensor 22) in a step S21. If “NO” in the step S21, that is, if there is no input from the sensors other than the touch sensor except the mouth touch sensor 22, the process returns to the step S13 as it is.
[0105]
On the other hand, if “YES” is determined in the step S21, that is, if there is an input from a sensor other than the touch sensor except the mouth touch sensor 22, the function of the sensor corresponding to the level 1 as described above is determined in a step S23. After executing the process of outputting the sound, that is, replying to the expression of the user, the process returns to step S13.
[0106]
In step S25 shown in FIG. 7, the interaction state is set to level 2 (transition). Then, in a step S27, the input from each sensor is checked, and it is determined whether or not the sensor is lowered in the step S29. If "YES" in the step S29, that is, if there is an input from the back proximity sensor 34 or the buttocks proximity sensor 36, and if there is no input from other sensors, it is determined that the lowering has been performed, and after being lowered in the step S31, Determine if two seconds have elapsed.
[0107]
If “NO” in the step S31, that is, if two seconds have not elapsed after the lowering, the process returns to the same step S31 and waits for a lapse of two seconds. On the other hand, if “YES” in the step S31, that is, if two seconds have elapsed after the lowering, the process returns to the step S11 shown in FIG. 6, and sets the interaction state to the level 1. That is, the level of the interaction state changes.
[0108]
If “NO” in the step S29, that is, if there is no input from the back proximity sensor 34 or the buttocks proximity sensor 36, and if there is an input from another sensor, it is determined that it is not lowered, and in a step S33. Determine if there is any input. If “NO” in the step S33, that is, if there is no input, the process returns to the step S27 as it is. On the other hand, if “YES” in the step S33, that is, if there is an input, it is determined in a step S35 whether or not the input is a rhythmical (regular) input.
[0109]
If “NO” in the step S35, that is, if it is not a rhythmic input, a sound corresponding to the role of the sensor shown in the left column of FIG. 4 is output in a step S37, and the process returns to the step S27. On the other hand, if “YES” in the step S35, that is, if it is a rhythmical input, the interaction state is set (transition) to the level 3 in a step S39, and the input from each sensor is checked in a step S41.
[0110]
Subsequently, in a step S43, it is determined whether or not there is an input from each sensor. If “NO” in the step S43, that is, if there is no input from each sensor, it is determined whether or not 5 seconds have passed without any input in a step S45. If "NO" in the step S45, that is, if 5 seconds have not elapsed, the process returns to the step S43 to determine whether or not there is an input from each sensor. On the other hand, if “YES” in the step S45, that is, if 5 seconds have passed without any input, the process returns to the step S25 and sets the interaction state to the level 2. That is, the level of the interaction state changes.
[0111]
On the other hand, if “YES” in the step S43, that is, if there is an input from any of the sensors, it is determined whether or not the interaction state is set to the level 4 in a step S47 shown in FIG. However, in this embodiment, whether to set the interaction state to level 4 is determined by a certain rule (rule), probability, or random number.
[0112]
Note that, similarly to other level transitions, transition to level 4 may be made in response to an input from a predetermined sensor. For example, when the user hits any part of the stuffed toy 12 or touches any part of the stuffed toy 12 with an irregular rhythm, the interaction state is set to level 4.
[0113]
If "NO" in the step S47, that is, a sound is output in the role of each sensor as shown in the right column of FIG. 4, and then the process returns to the step S41 shown in FIG. 7 to check the input again. In other words, when the interaction state is level 3, the musical element is operated (controlled) in accordance with the expression of the user, the response from a simple human voice is changed to a sound like a chord, and further along with the rhythm and melody. Music can be played.
[0114]
On the other hand, if “YES” in the step S47, that is, if the interaction state is set to the level 4, the interaction state is set (transition) to the level 4 in the step S51. Then, in step S53, the input from each sensor is checked. In a succeeding step S55, it is determined whether or not the action is soothing or relaxing. For example, it is determined whether or not there is a rhythmical (regular) input from any of the sensors, so that the mother can comfortably call the child good. That is, it is determined whether there is an input similar to that in the case of transition from level 2 to level 3.
[0115]
It should be noted that voice data corresponding to voices used when playing or soothing, such as "Yoshiyoshi", is stored in advance, and the user's voice is recognized by, for example, the DP matching method or the HMM method, and "Yoshiyoshi" is used. It may be determined whether or not these words are continuously input.
[0116]
Alternatively, the determination may be made based on both a rhythmic input and an input of a word such as “yoshi”.
[0117]
If “YES” in the step S55, that is, if the input is a rhythmical input, it is determined that the action is a soothing or a soothing action, and the process returns to the step S39 shown in FIG. 7 to set the interaction state to the level 3. That is, the interaction state is changed.
[0118]
On the other hand, if “NO” in the step S55, that is, if the input is irregular, it is determined that the action is not soothing or easy, and it is determined in a step S57 whether or not the user is left. Specifically, it is determined whether there is an input from the back proximity sensor 34 or the buttocks proximity sensor 36, and whether there is an input from another sensor. If “YES” in the step S57, that is, if the device is left, it is determined whether or not 10 seconds have elapsed since the device was left in the step S59. If “NO” in the step S59, that is, if 10 seconds have not elapsed after being abandoned, the process returns to the same step S59 and waits for an elapse of 10 seconds. On the other hand, if “YES” in the step S59, that is, if ten seconds elapse after being abandoned, the process returns to the step S1 and sets the interaction state to the level 1 (transition). That is, the state returns to the initial state.
[0119]
On the other hand, if “NO” in the step S57, that is, if not left, the sound corresponding to the expression of the user is output in a step S61 in the role of the sensor illustrated in FIG. Check sensor input. However, since it is out of control, the codes and keys are randomly changed, so that an unpleasant sound such as a dissonant sound is output.
[0120]
According to this embodiment, the level of the interaction state is changed according to the expression of the user, and a sound corresponding to the input of the sensor is output in the role of the sensor determined by the level of the interaction state. It can be said that it is excellent. Therefore, when the music communication device is provided as a toy, an infant or the like does not get bored immediately.
[0121]
In this embodiment, the pressure sensor is used as the touch sensor. However, a contact sensor that can determine whether or not the user touches the touch sensor may be used.
[0122]
Further, the type, number, or arrangement of roles of the sensors used in this embodiment are merely examples, and can be arbitrarily changed according to the intention of the designer. That is, it should not be limited by this embodiment.
[0123]
Further, in this embodiment, it is determined only whether or not a person approaches in accordance with a change in the luminance level of the video signal input from the camera. However, by analyzing the video signal, the stuffed toy 12 (music communication device 10) is analyzed. It is also possible to specify the place where the is placed or the person approaching. For example, a plurality of types of role arrangements as shown in FIGS. 3 to 5 are prepared, and one type of role arrangement can be selected by specifying a place or a person. When a person can be specified, when the user (master) of the stuffed toy 12 is specified, the state transition from level 0 to level 3 is made faster than when another person is specified. You can also have fun communication in time.
[Brief description of the drawings]
FIG. 1 is an illustrative view showing one embodiment of the present invention;
FIG. 2 is an electrical configuration diagram of the music communication device shown in FIG. 1 embodiment.
FIG. 3 is an illustrative view showing one example of a role arrangement of each sensor shown in FIGS. 1 and 2;
FIG. 4 is an illustrative view showing another example of the role arrangement of each sensor shown in FIGS. 1 and 2;
FIG. 5 is an illustrative view showing another example of the role arrangement of each sensor shown in FIGS. 1 and 2;
FIG. 6 is a flowchart showing a part of the processing of the CPU shown in FIG. 2;
FIG. 7 is a flowchart showing another portion of the processing by the CPU shown in FIG. 2;
FIG. 8 is a flowchart showing another portion of the processing by the CPU shown in FIG. 2;
[Explanation of symbols]
10. Music communication device
12 stuffed animals
14, 22, 26, 32, 48, 50, 52, 54 ... touch sensor
16 ... microphone
18… Camera
20, 34, 36 ... Proximity sensor
24… Speaker
30, 38 ... temperature sensor
40, 42, 44, 46… Bend
60… Computer
62 ... CPU
66… ROM
68… RAM
70… Counter
72… Timer

Claims (7)

A music communication device comprising a plurality of sensors for detecting a user's expression, and controlling a plurality of musical elements by an input from the plurality of sensors,
Role setting means for setting a role of each sensor for the plurality of music elements in advance for each of a plurality of interaction state levels determined according to detection results of the plurality of sensors , wherein at least one sensor has a role of the interaction state Different depending on the level, moreover
State setting means for setting any of the plurality of interaction state levels according to the detection results of the plurality of sensors, and
A music communication device, comprising: output means for outputting a sound in which the musical element is controlled in response to an input from the plurality of sensors according to a role of the sensor at an interaction state level set by the state setting means . The music communication device according to claim 1 , wherein the role setting unit assigns the role to the at least one sensor at a certain interaction state level and does not assign the role at another interaction state level . The music communication device according to claim 1 , wherein the state setting unit sets one of the plurality of interaction state levels according to an input from a predetermined sensor among the plurality of sensors . And the output means, the detection results including sound changing means for changing the musical elements in accordance with music communication apparatus according to any one of claims 1 to 3. The sound changing means includes a scale changing means for changing a scale according to the detection result, a chord changing means for changing a chord according to the connection detection result, a rhythm changing means for changing a rhythm according to the detection result, The music communication device according to claim 4, further comprising at least one melody changing unit that changes a melody according to a detection result. Further comprising a stuffed animal in which the plurality of sensors are arranged at a predetermined site,
The music communication device according to claim 1, wherein the plurality of sensors detect the expression through the stuffed toy. The state setting means, in accordance with an input from a predetermined sensor among the plurality of sensors, the stuffed animal is determined to be held, lowered, soothed or cradled, or left, 7. The music communication device according to claim 6, wherein one interaction state level is set based on the determination.

Images