WO2017061368A1

WO2017061368A1 - Bodily sensation control device and bodily sensation system

Info

Publication number: WO2017061368A1
Application number: PCT/JP2016/079267
Authority: WO
Inventors: 石川　博一; 健文大塚
Original assignee: シャープ株式会社
Priority date: 2015-10-09
Filing date: 2016-10-03
Publication date: 2017-04-13
Also published as: JP6607952B2; JPWO2017061368A1

Abstract

In this bodily sensation control device (10), at least an acoustic signal is input into an input unit (12). A microcomputer (determining unit) (11) determines whether bodily sensation control data for controlling a bodily sensation appliance (30) have been input into the input unit (12) together with the acoustic signal. If bodily sensation control data have been input into the input unit (12) together with the acoustic signal, a drive signal for driving the bodily sensation appliance (30) is generated by a drive signal generating unit (17) on the basis of the bodily sensation control data, and if no bodily sensation control data have been input into the input unit together with the acoustic signal, the drive signal for driving the bodily sensation appliance (30) is generated by the drive signal generating unit (17) on the basis of the acoustic signal.

Description

Experience control device and experience system

This disclosure relates to a sensation control device that controls a sensation device that outputs vibration, light, or the like in accordance with sound, and further relates to a sensation system including the sensation control device and the sensation device. This application claims priority based on Japanese Patent Application No. 2015-200763 filed on Oct. 9, 2015, and incorporates all the content described in the Japanese Patent Application. .

Conventional sensation devices are mainly used to convert the bass component of music into mechanical vibrations (referred to as “sensory sensation”) so that the music listener can experience it, or to create a sense of reality in a home theater system. There are known things that allow viewers to feel a feeling of vibration and vibration.

In recent years, in conjunction with light, it is used for production at live concerts, etc. In addition to 3D images and sound at amusement parks or movie theaters, called 4D movies, front and back, left and right, up and down It has realized special effects that stimulate the five senses, such as movement, vibration, wind, fog, scent, strobe, and smoke.

Japanese Unexamined Patent Publication No. 2006-287483 (Patent Document 1) relates to a signal processing circuit for body sensation sound for producing a sense of reality in the above-mentioned home theater system. Specifically, the signal processing circuit of this document mixes the left signal, right signal, left surround signal, right surround signal, and low frequency sound effect signal of the 5.1 channel surround system, and performs signal processing on the mixed signal. By doing so, a bodily sensation sound signal is generated. Here, the signal processing circuit is configured to output the sensory sound signal only when the low frequency sound effect signal is large.

JP 2006-287483 A

The signal processing circuit described in Patent Document 1 automatically generates a bodily sensation signal based on an acoustic signal. Therefore, music / video software such as an existing CD (Compact Disc) or DVD (Digital Versatile Disc) is used. This is convenient for use. However, Patent Document 1 discloses a means for making the sensation device and its control device usable even when a composer himself creates a song that includes not only sound (or sound and video) but also a sensation effect. Not disclosed.

This disclosure has been made in consideration of the above-described problems, and the purpose of this disclosure is when only music is provided, and when control data of a sensation device that matches the music is provided along with the music. It is to realize a bodily sensation control device and a bodily sensation system that can be used in both cases.

This disclosure provides, as one aspect, a sensation control device for controlling a sensation module that gives a sensation specific to a user. The sensation control device according to this aspect includes at least an input unit to which an acoustic signal is input, a determination unit, and a drive signal generation unit. The determination unit determines whether or not sensation control data for controlling the sensation module is input to the input unit together with the acoustic signal. The drive signal generation unit generates a drive signal for driving the sensation module based on the sensation control data when the sensation control data is input to the input unit, and the sensation control data is input together with the sound signal. When not input to the unit, a drive signal for driving the sensation module is generated based on the acoustic signal.

According to the above aspect, it is possible to provide a sensation control device that can be used both when the sound data indicating the musical tone of the music is input and when the control data of the sensation device is input together with the sound data. Can do.

It is an external view showing typically an example of a bodily sensation system according to the first embodiment. It is a block diagram which shows the functional structure of the connection box of FIG. It is a block diagram which shows the functional structure of the sensation apparatus of FIG. It is a flowchart which shows operation | movement of the connection box of FIG. It is a flowchart which shows the procedure which discriminate | determines whether the sensation control data is contained in the input signal in the sensation control apparatus by 2nd Embodiment.

Hereinafter, embodiments will be described in detail with reference to the drawings. In addition, the same reference number is attached | subjected to the part which is the same or it corresponds, and the description may not be repeated.

<First Embodiment>
[Example of bodily sensation system]
FIG. 1 is an external view schematically showing an example of a bodily sensation system according to the first embodiment. FIG. 1 shows an example of using the sensation control device (connection box 10) for controlling the

sensation devices

30A and 30B when the

sensation devices

30A and 30B (wristbands) are used in a seat such as an airplane or a long-distance train. ing.

Referring to FIG. 1 (A), an armrest of a seat 50 such as an airplane is usually provided with a 3.5 mmφ stereo mini jack 52 for connecting an earphone or headphones. From the jack 52, an audio signal from an audio device such as a CD player, a DVD player, a Blu-ray (registered trademark) player and / or a video device, or an audio signal of a radio broadcast or a television broadcast is output.

In the present embodiment, there is provided a sensation control device that can be used when not only analog sound signals but also sensation control data created so as to match the sound signals is output from the jack 52 together. In this case, the sensation control data is in a specific frequency region that is predetermined in a non-audible region (a region higher than about 20 kHz, which is the upper limit frequency that a person can listen to) so as not to affect the acoustic signal. Modulation is performed, and the sensation control data after modulation is superimposed on the acoustic signal. Hereinafter, the acoustic signal and the sensation control data after modulation incorporated in the acoustic signal are collectively referred to as a superimposed signal.

As shown in FIG. 1A, a connection box 10 as a sensation control device is connected to the jack 52 in order to extract the sensation control data from the received superimposed signal. The connection box 10 uses a filter to extract a signal in a specific frequency region of the superimposed signal, and reproduces the sensation control data by demodulating the extracted signal. The connection box 10 generates a drive signal for driving a corresponding sensation module provided in the sensation device based on the sensation control data. The generated drive signal is transmitted from the connection box 10 to each sensation module provided in the sensation device 30 using a wireless communication method such as BLUETOOTH (registered trademark) (see FIG. 1 for an example of the sensation module). B)).

Furthermore, the connection box 10 is configured to be able to handle a case where only an acoustic signal that does not include sensation control data is received. In this case, the connection box 10 generates a plurality of drive signals for driving each sensation module based on the intensity of each frequency component of the acoustic signal extracted by the plurality of filters, and the generated drive signals are transmitted to the wireless communication device. Send to the bodily sensation device using the method.

In any of the above cases (when a superimposed signal is received or when only an acoustic signal is received), the acoustic signal is output to the earphone or headphones via a stereo mini jack (not shown) provided in the connection box 10. Is done. In the above description, the driving signal of each sensation module is transmitted to the sensation device by wireless communication, but may be transmitted to the sensation device by wired communication.

FIG. 1 (B) shows an example of a bodily sensation device. In the case of FIG. 1 (B), the sensation apparatus has the form of

wristbands

30A and 30B to be worn on both wrists of the user. Inside the wristband, a communication device for communicating with the connection box 10 of FIG. 1 (A), a vibrator as an experience module, and an air bag and a blower for inflating the air bag are attached. Yes. The vibrator can give a vibration to the user, and the blower can inflate the air bag to apply pressure to the user's wrist. Furthermore, LED (Light Emitted Diode) 37 for outputting light according to an acoustic signal is attached to

wristbands

30A and 30B as a body sensation module.

The on / off of the outputs of these sensation modules (ie, vibrator, blower, and LED) is controlled according to the drive signals of each sensation module received from the connection box 10 as the sensation control device. The output level of the sensation module may be changed according to the sound pressure level of the sound output. For example, the greater the sound pressure level of the sound output, the greater the output intensity of the sensation module.

In addition, the sensation device controlled by the connection box 10 is not limited to the wristband. For example, instead of the wristband, a band-like thing wound around a part of the body such as an arm, a leg, or an ankle, or a cushion-like thing pressed against a part of the body such as the back or the waist may be used. Furthermore, the sensation module attached to the sensation device is not limited to the vibrator, blower, and LED.

[Functional configuration of connection box]
FIG. 2 is a block diagram showing a functional configuration of the connection box of FIG. Referring to FIG. 2, a connection box 10 as a sensation control device includes a microcomputer 11 including a CPU (Central Processing Unit) and a memory, an acoustic signal input unit 12, a digital filter 16, and a sensation module drive signal generation. Unit 17, acoustic signal output unit 13, communication unit 14, antenna 21, and power supply unit 15 (battery).

The microcomputer 11 controls each component of the connection box 10. Furthermore, the microcomputer 11 functions as a determination unit that determines whether the input signal from the output unit 51 of the audio device includes sensation control data for controlling the sensation module.

The acoustic signal input unit 12 receives an analog acoustic signal input from the output unit 51 of the acoustic device. In the case of FIG. 1, an analog acoustic signal is input from a stereo mini jack 52 that is an output terminal of the output unit 51 to a stereo mini plug that is an input terminal of the acoustic signal input unit 12. As described above, the sensation control data modulated in the specific frequency region within the non-audible region may be input by being incorporated into the acoustic signal. The acoustic signal input unit 12 performs A / D (Analog to Digital) conversion of the input acoustic signal or superimposed signal into a digital signal. The digitally converted acoustic signal or superimposed signal is input to the microcomputer 11.

The digital filter 16 extracts a desired frequency band from the acoustic signal or the superimposed signal in accordance with a command from the microcomputer 11. The digital filter 16 is configured by, for example, a band pass filter, a low pass filter, a high pass filter, and a combination thereof. For example, when a superimposed signal is input to the acoustic signal input unit 12, sensation control data modulated to a specific frequency region within the non-audible region is extracted by a band pass filter or a high pass filter. When only the acoustic signal, not the superimposed signal, is input to the acoustic signal input unit 12, for example, in order to generate a driving signal for each body sensation module based on the acoustic signal, Sound ranges are extracted by a low-pass filter, a band-pass filter, and a high-pass filter, respectively.

When a superimposed signal (ie, an acoustic signal and modulated sensation control data) is input to the acoustic signal input unit 12, the drive signal generation unit 17 is in the non-audible region extracted by the digital filter 16. The sensation control data is reproduced by demodulating the signal in the specific frequency region. The drive signal generator 17 generates a drive signal for driving each sensation module based on the sensation control data. Specifically, in the case of FIG. 2, the drive signal generation unit 17 includes a vibrator drive signal generation unit 18 that generates a drive signal for a vibrator, a blower drive signal generation unit 19 that generates a drive signal for a blower, and an LED And an LED drive signal generation unit 20 that generates a drive signal for use. The output of each body sensation module is controlled according to the corresponding drive signal.

On the other hand, when only the acoustic signal, not the superimposed signal, is input to the acoustic signal input unit 12, the drive signal generation unit 17 detects each sensation based on a plurality of signal components of the acoustic signal extracted by the digital filter 16. Generate drive signals for the module. In this case, for example, a method similar to Patent Document 1 can be used.

The digital filter 16 and the drive signal generation unit 17 are configured by a dedicated digital circuit, for example, a programmable FPGA (Field Programmable Gate Array). When the microcomputer 11 is sufficiently fast, the functions of the digital filter 16 and the drive signal generation unit 17 may also be realized by the microcomputer 11.

The acoustic signal output unit 13 performs D / A (Digital to Analog) conversion on the acoustic signal A / D converted by the acoustic signal input unit 12, and outputs the analog acoustic signal after the D / A conversion to the acoustic output device 53. To do. The sound output device 53 is, for example, an earphone, a headphone, a speaker, or the like. The acoustic signal output unit 13 is configured to output the acoustic signal or superimposed signal input to the acoustic signal input unit 12 to the acoustic output device as it is (without performing A / D conversion and D / A conversion). Also good. Since the sensation control data is modulated in the non-audible region of the acoustic signal, even if the superimposed signal is output as it is, the acoustic output device 53 is not adversely affected.

The communication unit 14 generates a transmission signal by modulating the drive signal of each sensation module generated by the drive signal generation unit 17 according to a wireless communication method such as BLUETOOTH (registered trademark). The generated transmission signal is transmitted to the communication unit (32 in FIG. 3) provided in the sensation apparatus 30 via the antenna 21.

The power supply unit 15 supplies power necessary for the operation of each of the above-described elements constituting the connection box 10 by a battery.

[Functional structure of sensory equipment]
FIG. 3 is a block diagram showing a functional configuration of the sensation apparatus shown in FIG. With reference to FIG. 3, the sensation apparatus 30 includes a microcomputer 31 including a CPU and a memory, a communication unit 32, an antenna 38, a vibrator 34, a blower 35, an air bag 36, an LED 37, and a power supply unit 33. (Battery).

The microcomputer 31 controls each component of the sensation device 30. The communication unit 32 receives the transmission signal from the connection box 10 via the antenna 38 and demodulates the received transmission signal to reproduce the driving signal of each sensation module.

Each sensory module operates according to the reproduced corresponding drive signal. Specifically, the vibrator 34 gives a vibration feeling to the user by being turned on and off according to the drive signal for the vibrator 34. The blower 35 applies pressure to the user's wrist by inflating the air bag according to the drive signal for the blower 35. The LED 37 stimulates the user with light by turning on and off according to the drive signal for the LED 37. Thus, the user can experience a unique sense of reality that matches the sound.

The power supply unit 33 supplies electric power necessary for the operation of each of the above-described elements constituting the sensation device 30 with a battery.

[Operation of connection box]
FIG. 4 is a flowchart showing the operation of the connection box of FIG. Hereinafter, with reference to FIG. 2 to FIG. 4, the above description will be summarized and the operation of the connection box 10 as the sensation control device will be described.

The microcomputer 11 built in the connection box 10 determines whether or not the acoustic signal input unit 12 is connected to the output unit 51 of the acoustic device (that is, a plug that is an input terminal of the acoustic signal input unit 12 is connected to the output unit 51. It is detected whether or not it is inserted into the jack 52 which is an output terminal. The connection box 10 is in a standby state until the connection between the acoustic signal input unit 12 and the output unit 51 of the acoustic device is detected (NO in step S105) (step S100).

Various methods can be considered as a method for determining whether or not the plug is inserted into the jack. For example, insertion into a jack can be detected by detecting a change in impedance between each terminal of the plug.

When the microcomputer 11 detects that the acoustic signal input unit 12 and the output unit 51 of the acoustic device are connected (YES in step S105), the A / D converter built in the acoustic signal input unit 12 Thus, an input signal (acoustic signal or superimposed signal) from the output unit 51 of the acoustic device is A / D converted. Subsequently, the microcomputer 11 uses the digital filter 16 to extract a signal in a specific frequency region within the non-audible region, and determines whether or not sensation control data is included in the input signal based on the signal in the specific frequency region. (Step S110).

When the sensation control data is included in the input signal as a result of the determination in step S110 (that is, when the input signal is a superimposed signal) (YES in step S115), the process from step S120A to step S140A is performed. A series of steps is executed repeatedly. Specifically, it is as follows.

First, a superimposed signal is input to the acoustic signal input unit 12 (step S120A). The digital filter 16 filters a specific frequency region in the input superimposed signal. The drive signal generation unit 17 takes out the sensation control data by demodulating the filtered signal (step S125A).

Next, the drive signal generation unit 17 generates a drive signal (vibrator drive signal, blower drive signal, LED drive signal) for driving each sensation module according to the extracted sensation control data (step S130A). Each generated drive signal is output to the corresponding sensation module of the sensation device 30 by the communication unit 14 (step S135A). Each sensation module operates according to the corresponding drive signal, so that the user can feel a special sense of presence.

In parallel with the above steps S120A to S135A, the acoustic signal output unit 13 converts the digital superimposed signal (or acoustic signal) after A / D conversion into an analog signal by performing D / A conversion, and the analog conversion is performed. The superimposed signal (or acoustic signal) is output to the acoustic output device 53 (step S140A). Alternatively, the acoustic signal output unit 13 may be configured to output the superimposed signal before A / D conversion input to the acoustic signal input unit 12 to the acoustic output device 53 as it is.

On the other hand, as a result of determination in step S110, when the sensation control data is not included in the input signal (that is, when the input signal is only an acoustic signal) (NO in step S115), from step S120B to step S140B. A series of steps are repeatedly executed. Specifically, it is as follows.

First, an acoustic signal is input to the acoustic signal input unit 12 (step S120B). The digital filter 16 extracts data of a plurality of frequency bands from the input acoustic signal (step S125B).

Next, the driving signal generation unit 17 drives driving sensors (vibrator driving signal, blower driving signal, LED, etc.) based on the extracted acoustic signals of each band or a part of them. Drive signal) is generated (step S130B). Each generated drive signal is output to the corresponding sensation module of the sensation device 30 by the communication unit 14 (step S135B). Each sensation module operates according to the corresponding drive signal, so that the user can feel a special sense of presence.

In parallel with the above steps S120B to S135B, the acoustic signal output unit 13 converts the digital acoustic signal after A / D conversion into an analog signal by D / A conversion, and outputs the analog converted acoustic signal as an acoustic output. The data is output to the device 53 (step S140B). Alternatively, the acoustic signal output unit 13 may be configured to output the acoustic signal before A / D conversion input to the acoustic signal input unit 12 to the acoustic output device 53 as it is.

[effect]
As described above, the sensation control device and the sensation system according to the first embodiment both in the case where only the acoustic signal indicating the musical tone of the music is input and in the case where the control data of the sensation device is input together with the acoustic signal. The user can experience a sense of reality that matches the music. Since the control data of the sensation device (sensation sensation control data) is incorporated in the acoustic signal as a signal modulated in a specific frequency region in the non-audible region, there is no adverse effect on the acoustic output device. There is no need to change the recording medium standard and the communication interface standard.

<Second Embodiment>
In 2nd Embodiment, the more detailed procedure of step S110 of the flowchart of FIG. 4, ie, the step which determines whether sensation control data is contained in an input signal is shown.

Particularly in the second embodiment, the identification information (ID code: Identification Code) of the sensation module to be controlled is recorded in advance in the first silent part of each piece of music recorded on a CD, DVD, Blu-ray disc or the like. It is desirable to leave. By detecting the identification information recorded in the first silent part when each music piece is reproduced, it is possible to extract the sensation control data from the superimposed signal including the subsequent acoustic signal without delay.

Hereinafter, the detailed procedure of step S110 will be described in detail with reference mainly to FIG. 2 and FIG. Since parts other than step S110 in the configuration of the sensation device 30 and the sensation control apparatus 10 and the operation of the sensation control apparatus are the same as those in the first embodiment, description thereof will not be repeated.

FIG. 5 is a flowchart showing a procedure for determining whether or not sensation control data is included in an input signal in the sensation control apparatus according to the second embodiment.

As described with reference to FIG. 4, when the microcomputer 11 incorporated in the connection box 10 detects that the acoustic signal input unit 12 is connected to the output unit 51 of the acoustic device (YES in step S105 of FIG. 4). The input signal to the acoustic signal input unit 12 is A / D converted. Subsequently, the microcomputer 11 determines whether or not any signal is included in the specific frequency region by filtering the specific frequency region to which the sensation control code is assigned in the non-audible region by the digital filter 16. (Step S200 in FIG. 5). When the microcomputer 11 determines that the signal is not included in the specific frequency region (NO in step S210), the microcomputer 11 determines that the sensation control data is not included in the input signal (step S250).

On the other hand, when any signal is included in the specific frequency region, the microcomputer 11 identifies the identification information of the sensation module (for example, an ID code (Identification Code) set in advance for each sensation module) in the specific frequency region. Is included (step S220). As a result, when the microcomputer 11 determines that the identification information of the sensation module is not included in the specific frequency region (NO in step S230), the microcomputer 11 determines that the sensation control data is not included in the input signal ( If it is determined that the identification information of the sensation module is included in the specific frequency region (YES in step S230), it is determined that the sensation control data is included in the input signal (step S240).

When executing the above procedure, it is desirable to record the identification information of the sensation module in the first silent portion of each piece of music recorded on a CD or DVD. Then, based on the information recorded in the silent part, it is possible to extract the sensation control data from the superimposed signal including the subsequent acoustic signal without delay.

<Third Embodiment>
In the first and second embodiments, it is assumed that the acoustic signal output from the output unit 51 of the acoustic device is an analog signal. However, when a digital acoustic signal is input instead of the analog signal, It is also possible to configure the connection box 10 to be applicable. For example, the present invention can also be applied when a video signal is input as an input signal together with an audio signal as in the HDMI (registered trademark) standard. When a digital acoustic signal is input, A / D conversion in the acoustic signal input unit 12 and D / A conversion in the acoustic signal output unit 13 are unnecessary.

Also in the above case, as in the first and second embodiments, the sensation control data is desirably incorporated into the acoustic signal as a signal modulated in a specific frequency region within the non-audible region. As a result, the sensation control data can be incorporated into the acoustic signal without changing the standard of the existing communication interface.

The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

10 connection box (sensory control device), 11, 31 microcomputer, 12 acoustic signal input unit, 13 acoustic signal output unit, 14, 32 communication unit, 16 digital filter, 17 drive signal generation unit, 18 vibrator drive signal generation unit, 19 Blower drive signal generation unit, 20 LED drive signal generation unit, 30 bodily sensation devices, 30A, 30B wristbands (both bodily sensation devices), 34 vibrators, 35 blowers, 36 air bags, 37 LEDs, 50 seats, 51 sound device output units , 52 jack, 53 sound output device.

Claims

A bodily sensation control device for controlling a bodily sensation module that gives a bodily sensation to the user,
An input unit to which at least an acoustic signal is input; and
A determination unit that determines whether or not sensation control data for controlling the sensation module is input to the input unit together with the acoustic signal;
When the sensory control data is input to the input unit together with the acoustic signal, a driving signal for driving the sensory module is generated based on the sensory control data, and the sensory control data is generated together with the acoustic signal. A bodily sensation control device comprising: a drive signal generation unit configured to generate a drive signal for driving the bodily sensation module based on the acoustic signal when not input to the input unit.
The sensation control data is modulated into a specific frequency region in a non-audible region and incorporated into the acoustic signal,
The driving signal generation unit extracts the signal in the specific frequency region using a filter and demodulates the sensation control data when the sensation control data is input to the input unit together with the acoustic signal. The sensation control device according to claim 1, wherein the sensation control device generates the drive signal according to the reproduced sensation control data.
The acoustic signal indicates a musical tone of music, and a silent part is included at the beginning of the music, and identification information of the sensation module is incorporated in the specific frequency region of the silent part,
The sensation control device according to claim 2, wherein the determination unit detects the identification information of the sensation module by extracting the specific frequency region of the silent part using a filter.
A sensation device with a sensation module that gives the user a unique sensation,
A bodily sensation control device for controlling the bodily sensation module;
The sensation control device is:
An input unit to which at least an acoustic signal is input; and
A determination unit that determines whether or not sensation control data for controlling the sensation module is input to the input unit together with the acoustic signal;
When the sensory control data is input to the input unit together with the acoustic signal, a driving signal for driving the sensory module is generated based on the sensory control data, and the sensory control data is generated together with the acoustic signal. A drive signal generation unit that generates a drive signal for driving the sensory module based on the acoustic signal when not being input to the input unit;
A bodily sensation system including a communication unit capable of communicating with the bodily sensation apparatus and transmitting the drive signal to the bodily sensation apparatus.
The sensation device has a form of a band wound around a part of the user's body,
A vibrator is attached to the band as the sensory module, and an air bag and a blower for inflating the air bag are attached to the band as another sensory module.
The bodily sensation system according to claim 4, wherein each of the vibrator and the blower is driven by the corresponding driving signal.