JP4529528B2 - Sound localization terminal - Google Patents

Description

    The present invention relates to a sound image localization technique for localizing a sound image.

  2. Description of the Related Art Conventionally, an audio distribution system that receives audio data such as music streamed via a wireless communication network at a mobile terminal and outputs it as sound from headphones connected to the mobile terminal is known (for example, Patent Document 1). According to such an audio distribution system, the user can easily enjoy music even when going out.

Japanese Patent Laid-Open No. 9-181510 (FIG. 3)

  However, in the conventional audio distribution system, even if the distributed audio data can be faithfully reproduced on the mobile terminal, entertainment such as the user participating in the generation of the audio data is provided to the user. I couldn't.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to provide a highly amusement acoustic amusement.

In order to solve the above-described problem, the sound image localization terminal according to the present invention includes terminal identification information for identifying the other terminal from other terminals constituting the sound system, terminal position information indicating the position of the other terminal, Receiving means for receiving azimuth information representing the azimuth of another terminal, position / orientation detecting means for detecting the position and azimuth of the terminal constituting the sound system, a plurality of sound data, and each sound data Storage means for previously storing the terminal identification information corresponding to each sound data in association with each sound data as a selection condition , and the sound data stored in association with the received terminal identification information of the other terminal a selection means for selecting from said storing means, with reference to the position and orientation of the own terminal, received the terminal location information及The position and orientation of the other terminal shown in azimuth information, characterized by comprising a localization means for localizing a sound image of the sound corresponding to said selected sound data.

  According to this configuration, when communicating with another terminal, sound data that matches a selection condition (for example, terminal ID) is selected and selected from a plurality of sound data stored in advance in the terminal itself. The sound image of the sound corresponding to the recorded sound data is localized at the position and orientation of the other terminal. A user who owns the terminal can recognize what kind of user is approaching by grasping the type of musical sound to be output, the sound image localization position, and the like.

  As described above, according to the present invention, it is possible to provide a highly entertaining acoustic amusement.

  Embodiments according to the present invention will be described below with reference to the drawings.

<Schematic configuration of sound data distribution system>
First, a schematic configuration of a sound data distribution system according to the present embodiment will be described with reference to FIG. In this figure, a satellite group 400 is an artificial satellite corresponding to GNSS (Global Navigation Satellite System) such as GPS (Global Positioning System), is controlled and controlled by a ground control station (not shown), and transmits satellite signals to the ground. To do. This satellite signal includes information indicating the time transmitted from the satellite and the orbital position of the satellite that transmitted the signal.

  On the other hand, the mobile communication network 500 includes various devices for providing a data communication service, such as a base station control device, to which a large number of base stations 510 are connected. The sound data distribution server 300 streams sound data to the terminal 100 via the mobile communication network 500 and the base station 510. Here, the sound data is information representing sound output from a virtual sounding point defined by three-dimensional coordinates.

  The terminal 100 is a portable wireless communication terminal, and transmits terminal position information indicating the position of the terminal 100 to the sound data distribution server 300 via the base station 510 or from the sound data distribution server 300. Receive sound data. As will be described later, when the terminal 100 receives sound data, the terminal 100 generates an audio signal from the sound data in which a sound image of a sound that is supposed to be emitted from the position of the sound generation point corresponding to the sound data is localized.

The terminal 100 can be connected to a stereo headphone 200, and emits an audio signal generated from the sound data via the headphone 200. The headphone 200 has an orientation sensor 210 that detects a direction in which the user's face faces in a use state worn on the user's head, and outputs a signal indicating the detected orientation while the audio signal is input. It transmits to the terminal 100. The terminal 100 includes a receiver for receiving satellite signals transmitted from the satellite group 400.
In this figure, as a set of the terminal 100 and the headphones 200, two sets of two users, ie, a set used for the user U1 and a set used for the user U2, are illustrated. And headphones 200 may be one set or three or more sets.

<Configuration of sound data distribution server>
FIG. 2 is a block diagram illustrating a configuration of the sound data distribution server 300. In this figure, the control unit 310 controls each unit via a bus B3. In addition, the control unit 310 performs processing for selecting a sounding point and processing for converting the amount of sound data as described later.

  The communication unit 320 receives information such as terminal location information transmitted from the terminal 100 via the mobile communication network 500. In addition, the communication unit 320 transmits the sound data corresponding to the four sound generation points selected by the control unit 310 in parallel to the terminal 100 as will be described later. The storage unit 330 is composed of a magnetic disk or the like and stores various types of information.

  FIG. 3 is a diagram illustrating a part of information stored in the storage unit 330. As shown in this figure, the storage unit 330 stores a plurality of sound generation point IDs, sound generation position information and sound data associated with each sound generation point ID. Of these, the pronunciation point ID is for identifying the pronunciation point. In the present embodiment, each sound generation point is defined at a position corresponding to a structure (for example, a store or a building) in the city, and “mini store A” or “building A” indicating the structure. The structure name such as “” is registered as the pronunciation point ID. Note that the location of the sound generation point is not limited to the position corresponding to the structure, and can be arbitrarily set. The pronunciation position information includes, for example, latitude, longitude, altitude, etc., and defines the position of the pronunciation point by three-dimensional coordinates.

  The sound data is data representing a sound that is assumed to be emitted from the sound generation point, and is data sampled at a predetermined frequency (for example, 44.1 kHz). The sound data may be any data as long as it represents data such as music, music, and voice. In this embodiment, the sound data includes “shopping march (music)” and “taiko (music). ) "," Dog cries (voice) "," chimes (electronic sounds) ", etc. are recorded. The sound data distribution server 300 distributes four sound data among the sound data stored in the storage unit 330 to the terminal 100. The terminal 100 processes and mixes the four distributed sound data and then emits the sound through the headphones 200.

  The storage unit 330 stores a data amount conversion table used for converting the amount of sound data. FIG. 4 is a diagram showing the configuration of the data amount conversion table. As shown in this figure, the data amount conversion table TBL is a table in which the distance D between the terminal 100 and the sounding point is associated with the sampling frequency at which sound data corresponding to the sounding point is to be converted. For example, in FIG. 4, the sampling frequency “f1 (= 44.1 kHz)” is associated with the distance D of “0” or more and less than “L1 (> 0)”, and “L1” or more and “L2 ( > L1) ”is associated with the sampling frequency“ f2 (= 22 kHz) ”. The control unit 310 converts the data amount of the sound data in a data amount conversion process described later according to the data amount conversion table TBL. Note that “L1”, “L2”, “L3”, and “L4” in FIG. 4 satisfy the relationship of “0” <“L1” <“L2” <“L3” <“L4”.

<Terminal configuration>
FIG. 5 is a block diagram illustrating a configuration of the terminal 100. In this figure, the control unit 110 controls each unit via the bus B1. Satellite radio wave receiver 145 receives satellite signals in parallel from each of a plurality of satellites included in satellite group 400, and inputs each received signal to positioning unit 140. The positioning unit 140 generates terminal position information indicating the position of the terminal 100 using information such as a transmission time and an orbital position included in each satellite signal input from the satellite radio wave receiving unit 145. At this time, the positioning unit 140 measures the distance (pseudo distance) from the terminal 100 to the satellite to which each satellite signal is transmitted, and substitutes each measured distance into the positioning equation to determine the terminal using three-dimensional coordinates. Generate location information.
Here, the terminal 100 is used by being carried by a user. For this reason, the position of the terminal 100 measured by the positioning unit 140 can be regarded as being equal to the position (center position) of the user.

  The instruction input unit 120 includes operation buttons and the like, and inputs a generation start signal and the like for instructing the start of audio signal generation to the control unit 110. Here, the generation of an audio signal is a process of generating an audio signal obtained by localizing a sound image output from a sound generation point from sound data. The control unit 110 controls the entire terminal 100 in accordance with an instruction from the user given via the instruction input unit 120.

  The azimuth detecting unit 150 detects which direction the face of the user wearing the headphones is facing by using the azimuth sensor 210 provided in the headphones 200, and supplies the detected direction to the bus B1 as azimuth information. As the direction sensor 210 provided in the headphones 200, the following method can be used in addition to the method of detecting geomagnetism and the method of using a gyroscope. That is, it is also possible to provide a plurality of positioning units in the headphones 200 and detect the direction in which the user's face is directed using the relative change amount of the position detected by each positioning unit.

  The wireless communication unit 130 establishes a wireless link with the base station 510 in the area where the terminal 100 is located under the control of the control unit 110, and transmits the terminal location information to the sound data distribution server 300 via this. Or four sound data are received from the sound data distribution server 300 in parallel.

  The audio signal generation unit 160 generates a 2-channel audio signal from each of the four sound data input in parallel from the wireless communication unit 130, and supplies the generated audio signal to the bus B1. At this time, the audio signal generation unit 160 separately generates an L channel signal that is an audio signal for the left ear and an R channel signal that is an audio signal for the right ear, and supplies each of them to the bus B1. . Each of the L channel signal and the R channel signal supplied to the bus B1 is output as sound from the headphones 200 via the audio signal output unit 190.

  Next, a detailed configuration of the audio signal generation unit 160 will be described with reference to FIG. As shown in this figure, the audio signal generation unit 160 includes four processing units 170-1, 170-2, and 170-3 that are equal to the number of sound data that the terminal 100 receives in parallel from the sound data distribution server 300. , 170-4. Each of these processing units 170-1, 170-2, 170-3, 170-4 processes any one of the four sound data, and is assumed to be emitted from the sounding point. An audio signal in which the sound image is localized is generated. Which processing unit 170-1, 170-2, 170-3, and 170-4 processes which sound data is processed according to the distance between each sounding point and the terminal 100 is the processing unit 170-1. , 170-2, 170-3, and 170-4 may be configured in such a manner that sound data corresponding to the sound generation point where the distance from the terminal 100 becomes longer is assigned. In the following description, when it is not necessary to distinguish each of the processing parts 170-1, 170-2, 170-3, and 170-4, the reference numeral is simply denoted as 170.

  Here, prior to detailed description of the processing unit 170, a mechanism in which a listener who has actually listened to sound output from a certain point (sound source) perceives the direction of the sound source and the distance to the sound source, that is, sound image localization. Will be described. For example, when the sound source is located on the right side of the listener, the distance from the listener's right ear to the sound source is closer than the distance from the left ear to the sound source. For this reason, the time taken for the sound output from the sound source to reach the right ear at a certain time is shorter than the time taken to reach the left ear. The listener perceives the direction of the sound source by such a delay time generated between the left ear and the right ear. Further, it is assumed that there are two sound sources, a sound source located near the listener and a sound source located far away. In this case, when a sound of a certain volume (sound pressure) is output from each sound source, the sound volume of the sound source located nearer than the sound volume of the sound source located far from the listener at the position of the listener. The volume is higher. Due to such a difference in volume, the listener perceives the distance to the sound source.

  Therefore, each processing unit 170 in the present embodiment, for each sounding point, and the position of the sounding point and the user (listener) so that the user can feel as if the sounding point actually exists. ) In accordance with the positions of the left and right ears), an audio signal in which the delay time and volume produced by the left and right ears are defined is generated. Hereinafter, with reference to one processing unit 170, generation of an audio signal related to one sound generation point will be described.

  As shown in FIG. 7, when the center position P (XP, YP, ZP) of the user U indicated by the terminal position information and the user's face direction A indicated by the orientation information are given, the left ear and the right When the distance from the ear is e, the position L (XL, YL, ZL) of the left ear of the user U is e / 2 horizontally from the center position P (XP, YP, ZP) and perpendicular to the direction A. The right ear position R (X R, Y R, Z R) is determined by the distance of e / 2 in the horizontal direction and perpendicular to the direction A from the center position P (XP, YP, Z R). It is specified by the position on the right side. Here, assuming that the distance between the sound generation point S (XS, YS, ZS) and the user (center position P) is sufficiently large, the sound reaches the user's ear as a plane wave. Further, it is assumed that the sound generation point S is located on the right front as viewed from the user, and at this time, the angle formed by the direction A facing the user's face and the direction of the sound generation point S as viewed from the user is θ. At this time, when a sound is output from the sound generation point S, the time difference (delay time) Δt of the sound that occurs between the right ear and the left ear reaches the distance d of the arrival path and the sound speed c,

  It is expressed as Here, since d = e · sin θ holds, the delay time Δt is

It becomes.
Further, when the distance from the sound generation point S to the left and right ears of the user is DL and DR, the time is t, and the wave equation of the spherical wave is f, the sound pressure PL generated in the left ear and the right ear Each of the sound pressures PR generated in can be expressed as follows.
That is,

It can be expressed as
The processing unit 170 generates an audio signal expressing the delay time Δt expression (2), the sound pressure PL expression (3), and the sound pressure PR expression (4) from the sound data. As a result, the sound image at the virtual sounding point is localized as if the user is sounding from the position indicated by the sounding position information.

  The description returns to FIG. 6 again. Each processing unit 170 includes a parameter generation unit 172, a delay unit 176, and an amplifier 178. Among these, the parameter generation unit 172 further includes a delay parameter generation unit 173 and an amplifier parameter generation unit 174. The delay parameter generation unit 173 generates a parameter that defines the delay time Δt of each of the L channel signal and the R channel signal. More specifically, the delay parameter generation unit 173 includes the sound generation position information received from the sound data distribution server 300, the direction information detected by the direction detection unit 150, the terminal position information detected by the positioning unit 140, the left and right The distance e between both ears is input, a parameter DP that defines the delay time Δt between the left and right ears is generated by the equation (2), and the parameter DP is transmitted to the delay unit 176.

On the other hand, the amplifier parameter generation unit 174 generates a parameter representing the sound pressure when each of the L channel signal and the R channel signal is emitted. More specifically, the amplifier parameter generation unit 174 generates the sound generation position information received from the sound data distribution server 300, the direction information detected by the direction detection unit 150, the terminal position information detected by the positioning unit 140, the left and right By inputting the distance e between both ears, a parameter AL for defining the sound pressure PL generated in the left ear and a parameter AR for defining the sound pressure PR generated in the right ear are generated by the equations (3) and (4). The parameters AL and AR are transmitted to the amplifier 178.
The distance e between the user's left and right ears input to the parameter generation unit 172 may be stored in a ROM (Read Only Memory) included in the control unit 110 and read from the ROM or the like. A configuration in which the user inputs via the input unit 120 may be adopted. Further, the above-described methods for specifying the positions of the left and right ears, the delay time Δt, and the sound pressures PL and PR (2, 3, and 4) are merely examples, and further, the head-related transfer function and the frequency spectrum It is possible to add various changes and improvements such as incorporating the qualitative change of the sound due to the change of the sound and the influence of the ratio of the direct sound and the reverberant sound.

  The delay unit 176 generates an L channel signal SL1 for the left ear and an R channel signal SR1 for the right ear from the sound data input via the wireless communication unit 130, and transmits each of them to the amplifier 178. More specifically, the delay unit 176 generates each signal such that a delay occurs in the L channel signal SL1 and the R channel signal SR1 according to the delay parameter DP received from the delay parameter generation unit 173. As a result, the L channel signal SL1 and the R channel signal SR1 for one sound data, as if the arrival time difference from the sounding point is generated according to the position of the left and right ears of the user, that is, the user From the viewpoint of the sound, the sound is generated as if it were output from a sounding point located in a certain direction.

  The amplifier 178 amplifies the L channel signal SL1 received from the delay unit 176 by the parameter AL received from the amplifier parameter generation unit 174, while receiving the R channel signal SR1 received from the delay unit 176 from the amplifier parameter generation unit 174. Are amplified by the parameter AR and transmitted to the mixing unit 180 as an L channel signal SL2 and an R channel signal SR2, respectively. Thereby, each of the L channel signal SL2 and the R channel signal SR2 is generated as if the sound pressure level differs according to the distance between the position of the user's left and right ears and the respective sound generation points. Such adjustment of the sound pressure level by the amplifier 178 is performed for each of the processing units 170-1, 170-2, 170-3, and 170-4. Therefore, the user feels as if the distance to each sounding point is different depending on the audio signal generated in each of the processing units 170-1, 170-2, 170-3, and 170-4. Can be given.

The mixing unit 180 mixes the four L channel signals SL2 transmitted from the four processing units 170, and transmits the L channel signal SL3 to the audio signal output unit 190, while mixing the four R channel signals SR2. The R channel signal SR3 is transmitted to the audio signal output unit 190. At this time, the mixing unit 180 preferably limits the signal levels of the mixed L channel signal SL3 and R channel signal SR3 so as not to damage the user's ear. Each of the L channel signal SL3 and the R channel signal SR3 transmitted from the mixing unit 180 is D / A (Digital / Analog) converted by the audio signal output unit 190, and then output to the headphone 200 for the left ear. Sound is emitted through the sound emitting unit 220 and the sound emitting unit 230 for the right ear.
Although confirming, the processing by the audio signal generation unit 160 includes various processes such as reception of sound data by the wireless communication unit 130, generation of terminal position information by the positioning unit 140, generation of direction information by the direction detection unit 150, and the like. Performed in parallel with the processing, the audio signal is generated from the sound data in a stream format. For this reason, when the user moves, the terminal position information, the direction information, and the like are updated accordingly, regardless of the position the user moves to, or in which direction the face faces, An audio signal is generated so that the sound image of the sound output from the sound generation point is localized.

<Operation of sound data distribution system>
Next, the operation of the sound data distribution system will be described with reference to FIG. This operation is a process of distributing sound data from the sound data distribution server 300 to the terminal 100 and generating an audio signal from the distributed sound data in the terminal 100 while updating the terminal position information and orientation information. This operation is a process in which a process is started by using a generation start signal input from the instruction input unit 120 of the terminal 100 as a trigger, and then a timer interrupt is performed by the terminal 100. Further, various processes executed in a general mobile communication system, such as connection authentication between the sound data distribution server 300 and the terminal 100 and terminal authentication, are not directly related to the present invention, and thus description thereof will be omitted. And

First, in step SA1, the control unit 110 of the terminal 100 receives the satellite signal transmitted from the satellite group 400 by the satellite radio wave reception unit 145, and acquires the satellite signal. Next, the control part 110 of the terminal 100 produces | generates the terminal position information SP which shows the position of the terminal 100 by the positioning part 140 according to the acquired satellite signal in step SA2. Next, the control unit 110 of the terminal 100 transmits the generated terminal location information SP to the base station 510 in step SA3.
When receiving the terminal location information SP from the terminal 100, the base station 510 transfers the terminal location information SP to the sound data distribution server 300 in step SA4.

  When the control unit 310 of the sound data distribution server 300 receives the terminal position information SP transferred from the base station 510, the control unit 310 executes sound generation point selection processing in step SA5. The sounding point selection process is a process of selecting sounding points until a predetermined number is reached according to the position of the terminal 100 based on the received terminal position information SP and the position of the sounding point based on the sounding position information. Here, the sound generation point selection process executed by the control unit 310 of the sound data distribution server 300 will be described with reference to FIG.

  First, in step SA51, the control unit 310 sets a selection number n indicating the number of selected pronunciation points to “0” and initializes the selection number n. Next, in step SA52, the control unit 310 selects the nearest sounding point among the sounding points not selected at this time. At this time, the control unit 310 selects a sound generation point using the received terminal position information SP of the terminal 100 and the sound generation position information of each sound generation point stored in the storage unit 330. For example, as shown in FIG. 11, it is assumed that eight sound generation points S1, S2,..., S8 are arranged around the terminal 100. These sound generation points S1, S2,..., S8 are arranged so as to be away from the terminal 100 in this order. At this time, if any of the sound generation points S1, S2,..., S8 is not selected (the number of selections n = 0), the control unit 310 selects the sound generation point S1 in step SA52.

  Next, in step SA53, the control unit 310 increments the selection number n by “1”. Next, in step SA54, the controller 310 determines whether or not the selection number n has reached a predetermined number (four in this embodiment). If the determination result is negative, control unit 310 returns the processing procedure to step SA52, and repeats the processing from step SA52 to step SA54 until the selected number n reaches a predetermined number.

  On the other hand, if the determination result in step SA54 is affirmative, control unit 310 ends the sounding point selection process because a predetermined number of sounding points have been selected. For example, in FIG. 11, the control unit 310 selects four sound generation points S1, S2, S3, and S4 indicated by black circles that are close to the terminal 100 among the eight sound generation points S1, S2,. In the present embodiment, four sound generation points are selected by the control unit 310, but the number of sound generation points to be selected can be arbitrarily set.

  Now, in FIG. 8 again, the control unit 310 of the sound data distribution server 300 performs the data amount conversion process in step SA6 after the sounding point selection process (step SA5) is completed. This data amount conversion process is a process of converting the data amount of sound data corresponding to the selected sounding point, that is, the data amount of sound data distributed to the terminal 100. Here, the data amount conversion processing executed by the control unit 310 of the sound data distribution server 300 will be described with reference to FIG. In this description, it is assumed that the sampling frequency of sound data recorded in advance in the storage unit 330 of the sound data distribution server 300 is 44.1 kHz or higher.

  First, in step SA61, the control unit 310 obtains the distance D between each sound generation point selected by the sound generation point selection process and the terminal 100 using the sound generation position information and the terminal position information SP. Next, in step SA62, the control unit 310 refers to the data amount conversion table TBL shown in FIG. 4, and the sound data of each sounding point is converted according to the distance D from each sounding point to the terminal 100. Specify the sampling frequency to be used. For example, in FIG. 11, the distance D between the terminal 100 and the pronunciation point S1 is “0” or more and less than “L1”, and the distance D between the terminal 100 and the pronunciation point S2 is “L1” or more and less than “L2”. Yes, it is assumed that the distance D between the terminal 100 and the pronunciation point S3 is not less than “L2” and less than “L3”, and the distance D between the terminal 100 and the pronunciation point S4 is not less than “L3” and less than “L4”. At this time, the control unit 310 refers to the data amount conversion table TBL, specifies the sampling frequency for the sound data at the sound generation point S1 as f1 (44.1 kHz), and sets the sampling frequency for the sound data at the sound generation point S2. The sampling frequency for the sound data at the sound generation point S3 is specified as f3 (10 kHz), and the sampling frequency for the sound data at the sound generation point S4 is specified as f4 (5 kHz).

Next, in step SA63, the control unit 310 generates sound data of the sampling frequency specified in step SA62 from the sound data of each sounding point recorded in advance in the storage unit 330. As a result, the generated sound data has a lower sampling frequency as the sound data of the sounding point farther from the terminal 100 decreases. Therefore, the amount of data decreases as the sampling data of the sounding point farther from the terminal 100 Is done. As a result, the total amount of sound data distributed from the sound data distribution server 300 is reduced, and as a result, the network traffic in the mobile communication network 500 related to the distribution of the sound data and the sound data distribution related to the transmission of the sound data. The load on the server 300 will be reduced.
In general, when the sampling frequency of the audio signal is lowered, the sound quality when the audio signal is emitted is deteriorated. However, in the present embodiment, the sound data distributed to the terminal 100 is the processing unit of the terminal 100. According to 170, the sound data of the sound generation point located farther from the terminal 100 is processed so as to have a smaller volume. For this reason, even if the sampling frequency of the sound data at the sound generation point located far from the terminal 100 is lowered, the sound quality when the audio signal generated in the terminal 100 is emitted is hardly affected. In other words, according to the data amount conversion process, the amount of sound data can be reduced without unduly impairing the sound quality, and the network traffic due to the distribution of sound data and the load on the sound data distribution server 300 can be reduced.

Now, in FIG. 8 again, when the data amount conversion process (step SA6) is completed, the control unit 310 of the sound data distribution server 300 next, in step SA7, the four sound data SD1 and SD2 whose data amounts have been converted. , SD3, and SD4 are transmitted to base station 510 in parallel. At this time, the control unit 310 adds sound position information recorded in the storage unit 330 to each sound data SD1, SD2, SD3, SD4, and then transmits the sound data SD1, SD2, SD3, SD4 in a stream format. . For example, in FIG. 3, if the sound data to be transmitted to the base station 510 is “shopping march” corresponding to the pronunciation point ID “ministore A”, the control unit 310 transmits the pronunciation position information ( After adding x1, y1, z1), "shopping march" is transmitted.
When the base station 510 receives the sound data SD1, SD2, SD3, SD4 transmitted from the sound data distribution server 300, the base station 510 transfers the sound data SD1, SD2, SD3, SD4 to the terminal 100 in step SA8.

  On the other hand, when the control unit 110 of the terminal 100 transmits the terminal position information SP to the base station 510 in step SA3, next, in step SA9, the azimuth detection unit 150 generates azimuth information indicating the user's face direction. Next, in step SA10, the control unit 110 of the terminal 100 generates an audio signal from the sound data SD1, SD2, SD3, SD4 received from the base station 510. At this time, the control unit 110 of the terminal 100 generates an audio signal in a stream format by the audio signal generation unit 160 according to the terminal position information SP, the direction information, and the sound generation position information. Next, the control unit 110 of the terminal 100 outputs an audio signal from the audio signal output unit 190 in step SA11. The audio signal output from the terminal 100 is output as sound via the headphones 200.

  For example, as shown in FIG. 12, the positions of the sound generation points S1, S2, S3, and S4 are set, and the position of the terminal 100 (user) is determined by the terminal position information SP, and the face of the user is directed by the direction information. It is assumed that direction A is given. At this time, it is assumed that the distance from the user to each of the sound generation points S1, S2, S3, S4 increases in this order for both the left and right ears. At this time, the sound pressure (volume) of the sound data at each sounding point output from the headphones 200 has the highest sound pressure of the sound data corresponding to the sounding point S1, and the sound pressure of the sound data corresponding to the sounding point S4. Becomes the smallest. As a result, the user perceives that the sounding point S1 is located closest and the sounding point S4 is located farthest. Further, since the distance from the sound generation point S1 to the user's right ear is shorter than the distance to the left ear, the L channel signal for the sound data at the sound generation point S1 is delayed from the R channel signal. Thereby, the user perceives that the pronunciation point S1 is located on the right side. Similarly, the user perceives that the sound generation point S4 is located on the left side by the delay amount (delay time Δt) between the L channel signal and the R channel signal.

  Next, as shown in FIG. 13, it is assumed that the user faces the sounding point S4. At this time, since the direction A toward the user's face is updated by the azimuth detection unit 150, the delay amount (delay time Δt) of each sound data between the L channel signal and the R channel signal is updated. Thereby, the user perceives that the sound generation point S2 is located on the right side and the sound generation point S3 is located on the left side.

  Next, as shown in FIG. 14, the user approaches the sounding point S4, and the distance from the user to each sounding point S1, S2, S3, S4 is the sounding point S4, the sounding point S2, the sounding point S3, the sounding point. It is assumed that it has moved to a position farther away in the order of S1. When the user moves in this manner, the sound pressure (sound volume) of the sound data at the sound generation point S1 decreases because the user moves away from the sound generation point S1, whereas the sound pressure of the sound data at the sound generation point S4 increases because the sound pressure approaches the sound generation point S4. Become. Accordingly, the user perceives that the user has left the sound generation point S1 and has approached the sound generation point S4.

  As described above, according to the present embodiment, it is as if the user is sounding from the position indicated by the sound generation position information according to the position of the user, the direction of the user, and the position of the sound generation point. An audio signal localized in is generated. As a result, the user can obtain a feeling as if each of the sound generation points actually exists at the specified position. For example, if each part of an orchestra is placed as a sounding point in a certain area, if the user moves within the area, the user gets a feeling as if moving in the space where each part is placed. This enables a user to participate in the generation of audio data and provide a varied and enjoyable acoustic amusement.

Further, in this embodiment, since the sound image of the sound generation point is localized, when applied to a voice information system that indicates the position of the target object of the user by voice, if the target object is positioned on the right side, “right direction, You will hear a sound like "It is right before the gas station." This makes it possible to more intuitively give information on the direction to the user as compared with a conventional voice information system that does not consider a sound image, and the efficiency of voice instructions is improved.
Furthermore, the sound data distribution system can be used as a guide for visually impaired people. For example, it is good also as a structure which arrange | positions the pronunciation point which outputs the audio | voice showing each of them in the ticket vending machine in a station, a station staff room, a ticket gate, etc. With such a configuration, the user can voluntarily approach the target because the position can be guided by voice in the same manner as when viewed with the eyes.

  In addition, the sound generation point may be arranged at a location such as a store and a sound indicating an advertisement of the store may be output. As a result, since it sounds as if sound is output from the store, the user can easily find a store located in a place that is difficult to see, such as the second floor of a building. On the other hand, in the store, an advertising effect can be expected, and the business is activated. In addition, the advertisement that works on hearing is converted from an actual sound such as a call to an advertisement of a virtual pronunciation point, thereby reducing noise in the city.

<Modification of First Embodiment>
In the first embodiment described above, an example in which the data amount of sound data is reduced in the data amount conversion processing has been shown in order to reduce the load on the network traffic and the sound data distribution server due to the distribution of the sound data. However, if these do not cause a problem, the data amount conversion process can be omitted.

<Second Embodiment>
<Configuration of sound data distribution system>
In the first embodiment described above, the sound data distribution system that distributes sound data from one sound data distribution server 300 to the terminal 100 has been described. In contrast, in the second embodiment, a sound data distribution system that distributes sound data from each of a plurality of sound data distribution servers to the terminal 100 will be described.
In addition, the same code | symbol is attached | subjected about the thing which is common in the system which concerns on 1st Embodiment among the structures of the sound data delivery system in 2nd Embodiment.

FIG. 15 is a diagram showing a schematic configuration of a sound data distribution system in the second embodiment. As shown in this figure, the mobile communication network 500 is roughly connected with two types of server devices. That is, the control server 600 and the sound data distribution servers 610A, 610B, and 610C. Among these, each of the sound data distribution servers 610A, 610B, and 610C is sound data distributed to the terminal 100, and stores sound data corresponding to different sound generation points. The control server 600 manages the distribution of sound data from the sound data distribution servers 610A, 610B, and 610C to the terminal 100. More specifically, the control server 600 selects a predetermined number (for example, two) of sound generation points according to the position of the terminal 100 and the position of each sound generation point. The sound data distribution servers 610A, 610B, and 610C distribute sound data corresponding to the sound generation point selected by the control server 600 to the terminal 100. Each of the control server 600 and the sound data distribution servers 610A, 610B, and 610C is assigned a server ID for identifying each in the mobile communication network 500.
For convenience of explanation, in the second embodiment, the sound data distribution servers 610A, 610B, and 610C are directly connected to the mobile communication network 500, but the sound data distribution servers 610A, 610B, and 610C are connected to the Internet or the like. It is good also as a structure connected to the mobile communication network 500 via. In the second embodiment, an example in which the number of sound data distribution servers 610A, 610B, and 610C is three will be described. However, the number of sound data distribution servers 610A, 610B, and 610C is limited to three. Instead, other numbers may be used.

  First, the configuration of the sound data distribution servers 610A, 610B, and 610C will be described. The sound data distribution servers 610A, 610B, and 610C in the second embodiment have the same configuration as the sound data distribution server 300 (see FIG. 2) in the first embodiment, a control unit that controls each unit, and mobile communication A communication unit that exchanges data with the network and a storage unit that stores various types of information are provided.

  16 is a diagram showing main information stored in the storage unit of the sound data distribution server 610A, and FIG. 17 is main information stored in the storage unit of the sound data distribution server 610B. FIG. Although not shown, the storage unit in the sound data distribution server 610C stores the same information as the sound data distribution servers 610A and 610B. As shown in these drawings, the storage units of the sound data distribution servers 610A and 610B store a sound generation point ID and sound data associated with each sound generation point ID. This storage unit is characteristic in comparison with the storage unit 330 of the sound data distribution server 300 in the first embodiment in that it does not store pronunciation position information.

  Next, the configuration of the control server 600 will be described. The control server 600 has the same configuration as that of the sound data distribution server 610, a control unit that controls each unit, a communication unit that exchanges data with the mobile communication network 500, and a storage unit that stores various types of information. It has.

  FIG. 18 is a diagram showing main information stored in the storage unit of the control server 600. As shown in this figure, the storage unit stores a sound generation point ID, sound generation position information and a server ID associated with each sound generation point ID. The storage unit of the control server 600 is different from the storage unit 330 of the sound data distribution server 300 in the first embodiment in that it does not store sound data and stores a server ID. It is in. The pronunciation point ID is used to specify the pronunciation point stored in the sound data distribution servers 610A, 610B, and 610C. The server ID is information indicating in which sound data distribution server 610A, 610B, 610C the sound data of the sound generation point specified by the sound generation point ID is stored. In this figure, the server ID “A” indicates the sound data distribution server 610A, the server ID “B” indicates the sound data distribution server 610B, and the server ID “C” indicates the sound data distribution server 610C. For example, in this figure, the pronunciation point of the pronunciation point ID “ministore A” is located at the coordinates (x1, y1, z1), and the sound data is stored in the sound data distribution server 610A. The sounding point of the point ID “ministore B” is located at coordinates (x2, y2, z2), and indicates that the sound data is stored in the sound data distribution server 610B.

<Operation of sound data distribution system>
The operation of the sound data distribution system in the second embodiment will be described with reference to FIG. This operation is a process of distributing sound data corresponding to the sounding point selected by the control server 600 from the sound data distribution server 610 to the terminal 100, and generating an audio signal from the distributed sound data in the terminal 100. . Note that this operation starts when a user inputs a generation start signal instructing the start of audio signal generation via the instruction input unit 120 of the terminal 100, and then a process in which a timer interrupt is performed by the terminal 100. It is. Further, various processes executed in a general mobile communication system, such as connection authentication and terminal authentication between the control server 600 and the terminal 100, are not directly related to the present invention, and thus description thereof will be omitted. And

  First, in step SB1, the control unit 110 of the terminal 100 receives the satellite signal transmitted from the satellite group 400 by the satellite radio wave reception unit 145, and acquires the satellite signal. Next, in step SB2, the control unit 110 of the terminal 100 uses the positioning unit 140 to generate terminal position information SP indicating the three-dimensional position of the terminal 100 according to the acquired satellite signal.

  Next, in step SB3, control unit 110 of terminal 100 transmits terminal location information SP to base station 510 together with a server ID indicating control server 600 that is the transmission destination of the information. When receiving the terminal location information SP from the terminal 100, the base station 510 transfers the information to the control server 600 in step SB4.

  When receiving the terminal position information SP, the control unit of the control server 600 executes a pronunciation point selection process according to the received terminal position information SP and the pronunciation position information stored in the storage unit in step SB5. This sounding point selection process is the same process as the sounding point selection process (see FIG. 9) in the first embodiment described above, and in order from the sounding points close to the terminal 100 until the predetermined number (two) is reached. Is the process of selecting. In this description of the operation, for example, the control unit of the control server 600 selects two sound generation points “ministore A” and “building B” from the sound generation point IDs shown in FIG. The “shopping march A (see FIG. 16)” stored in the sound data distribution server 610A and the “taiko B (see FIG. 17)” stored in the sound data distribution server 610B are distributed to the terminal 100. Will be described. Note that the number of sound generation points selected by the control unit of the control server 600 is not limited to two and can be arbitrarily set.

Next, the control unit of the control server 600 transmits the server ID_SID of the sound data distribution server 610 having the sound data of the selected pronunciation point to the base station 510 in step SB6. At this time, the control unit of the control server 600 adds the pronunciation point ID and the pronunciation position information stored in the storage unit to the server ID_SID, and then transmits the server ID_SID. That is, the control unit of the control server 600 adds the pronunciation point ID “ministore A” and the pronunciation position information (x1, y1, z1) to the server ID “A” in FIG. 18 and also adds to the server ID “B”. After adding the pronunciation point ID “Bill B” and the sound generation position information (x2, y2, z2), the server ID “A” and the server ID “B” are transmitted to the base station 510.
When receiving each server ID_SID from the control server 600, the base station 510 transfers them to the terminal 100 in step SB7.

  When receiving the server ID_SID transferred by the base station 510, the control unit 110 of the terminal 100 requests the sound data distribution servers 610A and 610B specified by the server ID_SID to distribute sound data in step SB8. The distribution requests DRA and DRB are transmitted to the base station 510. That is, the control unit 110 of the terminal 100 transmits the “shopping march A” distribution request DRA to the sound data distribution server 610A and the “taiko B” distribution request DRB to the sound data distribution server 610B to the base station 510. .

  When receiving the distribution requests DRA and DRB from the terminal 100, the base station 510 transfers the received distribution requests DRA and DRB to the corresponding sound data distribution servers 610A and 610B in step SB9 and step SB10. That is, the base station 510 transfers the distribution request DRA to the sound data distribution server 610A in step SB9, and transfers the distribution request DRB to the sound data distribution server 610B in step SB10.

  When receiving the distribution request DRA from the base station 510, the control unit of the sound data distribution server 610A streams the sound data SDA (here, “shopping march A”) indicated by the distribution request DRA to the base station 510 in step SB11. Send in format. Upon receiving the sound data SDA from the sound data distribution server 610A, the base station 510 transfers the sound data SDA to the terminal 100 in step SB12.

  On the other hand, when receiving the distribution request DRB from the base station 510, the control unit of the sound data distribution server 610B receives the sound data SDB (here, “taiko B”) indicated by the distribution request DRB in step SB13. Send in stream format. Upon receiving the sound data SDB from the sound data distribution server 610B, the base station 510 transfers the sound data SDB to the terminal 100 in step SB14. The processes of step SB13 and step SB14 are executed in parallel with the processes of step SB11 and step SB12 described above.

  Similar to the first embodiment, the data amount conversion processing may be executed in the sound data distribution servers 610A and 610B before the sound data SDA and SDB are transmitted. That is, each of the sound data distribution servers 610A and 610B is configured to transmit the sound data to the base station 510 after reducing the data amount of each of the sound data SDA and SDB according to the distance between the terminal 100 and the sounding point. It is also good.

  The control unit 110 of the terminal 100 transmits the sound data SDA transmitted from the sound data distribution server 610A (here “shopping march A”) and the sound data SDB transmitted from the sound data distribution server 610B (here “drum sound B”). ”) Is received in parallel via the base station 510, in step SB15, the sound data is input to the audio signal generation unit 160 to generate left and right two-channel audio signals. At this time, the sound data of “shopping march A” and “drum sound B” is processed by one of the two processing units 170 included in the audio signal generation unit 160. And the control part 110 of the terminal 100 outputs an audio signal from the audio signal output part 190 in step SB16. The audio signal output from the terminal 100 is output as sound via the headphones 200.

  As described above, in the second embodiment, the sound data is distributed from the sound data distribution servers 610A, 610B, and 610C to the terminal 100 under the control of the control server 600. In the terminal 100, the position of the user and the direction of the face In accordance with A, an audio signal in which a sound image from a virtual sound generation point is localized is generated. As a result, as in the first embodiment described above, the user can move in a space as if the sound generation point actually exists, and can provide an unprecedented acoustic amusement.

  In the second embodiment, sound data is distributed from a plurality of sound data distribution servers 610A, 610B, and 610C. For this reason, the situation in which the plurality of terminals 100 receive sound data distribution from one sound data distribution server is avoided, and the load on the sound data distribution server is distributed. Furthermore, since the distribution of the sound data is collectively managed by the control server 600, the sound data can be easily managed, and the sound data can be easily increased. As a result, the types and contents of the sound data distributed to the terminal 100 are enriched.

<Modification of First and Second Embodiments>
In the first and second embodiments described above, the example in which the terminal location information SP is generated in the terminal 100 by GNSS is shown, but the present invention is not limited to this. For example, the sound data distribution server 300 or the control server 600 may generate the terminal position information SP according to the position of the base station 510 where the terminal 100 establishes a wireless link.

  In the first and second embodiments described above, the terminal 100 including the wireless communication unit 130 that wirelessly communicates with the base station 510 has been described. However, the present invention is not limited to this. For example, a portable terminal that does not have a wireless communication function such as PDA (Personal Digital Assistants) and a removable communication module may be attached to the portable terminal so that data can be exchanged with the sound data distribution server 300 or the like.

  In the first and second embodiments described above, the example in which the sound data is distributed in the stream format has been shown. However, the storage unit that stores the sound data is provided in the terminal 100, and the audio is generated by the sound data cached in the storage unit. It is good also as a structure which produces | generates a signal.

  In addition, in the first and second embodiments described above, the amount of sound data distributed to the terminal 100 is converted by the sound data distribution server 300, 610A or the like according to the distance between the terminal 100 and the sound generation point. An example is shown, but the present invention is not limited to this. For example, the storage unit 330 of the sound data distribution server 300 stores in advance a plurality of sound data representing the same sound and having different data amounts (for example, a plurality of sound data having different sampling frequencies). Any one of these may be selected according to the distance between the terminal 100 and the sound generation point, and the selected sound data may be transmitted to the terminal 100. Thereby, network traffic will be reduced like each embodiment mentioned above.

<Third Embodiment>
In the first and second embodiments described above, the aspect in which sound data is received from the sound data distribution servers 300 and 610 via the wireless communication network 500 and an audio signal is generated from the received sound data has been described. On the other hand, in the following third embodiment, an aspect of generating an audio signal from sound data stored in a storage unit in the terminal will be described.

  FIG. 20 is a diagram illustrating a schematic configuration of a sound system according to the third embodiment. The sound system includes a plurality of terminals 700 that can exchange terminal IDs for identifying terminals, terminal position information, and direction information.

  Each terminal 700 selects and selects suitable sound data from among a plurality of sound data stored in the storage unit of its own terminal 700 based on the terminal ID, terminal position information, and direction information received from the other terminal 700 It has a function to generate an audio signal from the sound data. In the following description, it is described as the own terminal 700A and the other terminal 700B for convenience.

FIG. 21 is a diagram showing a configuration of a terminal (sound image localization terminal) 700. The terminal 700 is provided with a sound data management unit 720, a receiving unit 730, and a transmitting unit 740 with respect to the terminal 100 shown in FIG. Since other configurations are the same as those in FIG. 5, corresponding portions are denoted by the same reference numerals, and description thereof is omitted.
The transmission unit 740 is means for transmitting the terminal ID, terminal position information, and direction information of the own terminal 700A to the other terminal 700B. Here, the terminal position information supplied to the transmission unit 740 is information indicating the position of the own terminal 700 </ b> A and information generated in the positioning unit (position / orientation detection means) 140. On the other hand, the orientation information supplied to the transmission unit 740 is information representing the orientation of the face of the user wearing the headphone 200 of the terminal 700A (in other words, information representing the orientation of the terminal), and the orientation detection unit (position / This is information generated in (azimuth detecting means) 150.

  Specifically, when communicating with the other terminal 700B, the control unit 110 reads the terminal ID of the own terminal 700A from a memory (not shown) and transfers it to the transmission unit 740. Further, the control unit 110 transfers the terminal position information of the own terminal 700 </ b> A supplied from the positioning unit 140 and the direction information of the own terminal 700 </ b> A supplied from the direction detection unit 150 to the transmission unit 740. The transmission unit 740 transmits the terminal ID, terminal position information, and direction information received from the control unit 110 to the other terminal 700B using Bluetooth (registered trademark), IrDA (Infrared Data Association), or the like.

  The receiving unit 730 receives terminal ID (attribute information) for identifying the other terminal 700B transmitted from the other terminal 700B, terminal position information indicating the position of the other terminal 700B, and direction information indicating the direction of the other terminal 700B. Means for receiving. Upon receiving these pieces of information from the receiving unit 730, the control unit 110 supplies the received terminal ID of the other terminal 700B to the sound data management unit 720, while transmitting the received terminal position information and direction information of the other terminal 700B as an audio signal. The data is supplied to the generation unit 160.

  The sound data management unit 720 is means for managing various types of sound data (for example, music 1, music 2,...). FIG. 22 is a diagram illustrating the contents of the sound data management table TA registered in the sound data management unit 720.

  In the sound data management table (storage means) TA, a plurality of sound data and condition information (terminal ID for identifying each terminal in FIG. 22) indicating the selection condition of each sound data are registered in association with each other. Yes. In this embodiment, the terminal ID (that is, terminal identification information corresponding to the sound data) is set as the selection condition of the sound data. However, other parameters may be set (described later). ).

  When the sound data management unit (selection unit) 720 receives the terminal ID (for example, terminal ID-3) of the other terminal 700B from the control unit 110, each terminal registered in the received terminal ID and the sound data management table TA. Compare the ID. Then, based on the comparison result, the sound data management unit 720 acquires (selects) the sound data (for example, the music piece 3) registered in association with the terminal ID from the sound data management table (storage means) TA. This is supplied to the audio signal generator 160.

  The audio signal generation unit 160 receives the terminal position information and direction information of the other terminal 700B from the reception unit 730, and receives the terminal position information and direction information of the own terminal 700A from the positioning unit 140 and the direction detection unit 150. The audio signal generation unit 160 generates an audio signal based on the received information and the sound data supplied from the sound data management unit 720. That is, the audio signal generation unit (localization means) 160 uses the position and orientation of the own terminal 700A as a reference, and the sound corresponding to the sound data is displayed at the position and orientation indicated in the terminal position information and orientation information of the other terminal 700B. Audio signal is generated and processed so that the sound image is localized. Since the details of the operation of the audio signal generation unit 160 have been clarified in the first embodiment and the like, further explanation is omitted.

  As described above, according to the sound system according to the third embodiment, each terminal 700 generates various audio signals using sound data stored in advance in its own terminal 700A. Thereby, like the first embodiment and the second embodiment, an unprecedented acoustic amusement can be provided. Furthermore, since a configuration for distributing sound data is not required, the configuration of the system can be simplified.

  In the present embodiment, the case where the terminal ID is set as the sound data selection condition has been described. However, instead of the terminal ID, the group to which each terminal belongs (for example, group A, group B,...). A group ID for identification may be set. In such a case, the receiving unit 730 of the own terminal 700A receives the group ID (attribute information) of the group to which the other terminal 700B belongs, instead of (or in addition to) the terminal ID of the other terminal 700B. According to such a configuration, since the same type of musical sound is output for each group, each user who owns the terminal 700 determines which group the user belongs to from the type of musical sound to be listened to through the headphones 200. Etc. can be grasped.

<Fourth embodiment>
FIG. 23 is a diagram showing a configuration of a terminal (sound image localization terminal) 800 according to the fourth embodiment. The terminal 800 is provided with a storage unit 810 instead of the sound data management unit 720 shown in FIG. Since other configurations are the same as those in FIG. 21, corresponding portions are denoted by the same reference numerals and description thereof is omitted. In the following description, it is described as own terminal 800A and other terminal 800B for convenience.
The storage unit 810 includes a local sound data storage unit 811 and another sound data storage unit 812. The own sound data storage unit 811 stores sound data (for example, music 1) unique to the own terminal 800A. On the other hand, the other sound data storage unit 812 has an identification ID (for example, terminal ID-2, terminal ID-3, etc.) for identifying the other terminal 800B and sound data (for example, music 2) unique to each other terminal 800B. , Music 3 etc.) are stored in association with each other. Each terminal 800 performs the following operation when communicating with another terminal 800B.

  The control unit 110 of the own terminal 800 </ b> A reads unique sound data from the own sound data storage unit 811 and sends it to the transmission unit 740. Further, the control unit 110 includes terminal position information of the own terminal 800A generated by the positioning unit (position / orientation detection means) 140, and direction information of the own terminal 800A generated by the direction detection unit (position / orientation detection means) 150. Are sent to the transmission unit 740, respectively. Upon receiving these pieces of information from the control unit 110, the transmission unit 740 adds a terminal ID for identifying the own terminal 800A to these pieces of information, and transmits the information to the other terminal 800B. On the other hand, the receiving unit (receiving unit) 730 receives the terminal ID, terminal position information, orientation information, and sound data of the other terminal 800B transmitted from the other terminal 800B. Upon receiving these pieces of information from the receiving unit 730, the control unit 110 transfers the terminal position information, direction information, and sound data of the other terminal 800B to the audio signal generation unit 160, while the terminal ID and sound data of the other terminal B are transferred. Stored in association with the other sound data storage unit 812.

  The audio signal generation unit 160 receives the terminal position information, direction information, and sound data of the other terminal 800B from the reception unit 730, and receives the terminal position information and direction information of the own terminal 800A from the positioning unit 140 and the direction detection unit 150. . The audio signal generation unit 160 generates an audio signal based on these pieces of information. That is, the audio signal generation unit (localization means) 160 receives the sound data received from the other terminal 800B at the position and orientation indicated in the terminal position information and orientation information of the other terminal 800B with reference to the position and orientation of the own terminal 800A. The audio signal is generated and processed so that the sound image corresponding to the sound is localized. The audio signal generation unit 160 outputs the audio signal thus generated to the outside via the headphones 200. Each user can grasp the type of musical sound output from the headphones 200, the sound image localization position, and the like, thereby recognizing which user is approaching. As described above, unique sound data may be exchanged between the terminals 800, and an audio signal may be generated from the exchanged sound data.

<Modification common to the first to fourth embodiments>
(Modification 1)
In each of the above-described embodiments, an example in which the position of the terminal is obtained using GPS has been illustrated. However, depending on a place where a radio wave transmitted from a GPS satellite is received (for example, indoors), a large error may occur. In some cases, position measurement is impossible. In such a case, the position of the terminal itself may be obtained using an ultrasonic sensor, an infrared sensor, or the like that is effective for position measurement within a limited range.

(Modification 2)
Also, instead of measuring the position and orientation of the terminal itself, only the relative distance and orientation with other terminals may be measured. FIG. 24 is a diagram illustrating a configuration of a terminal (sound image localization terminal) 900 according to the second modification. The terminal 900 is provided with an ultrasonic generator 910 and a relative distance / direction calculation unit 920 instead of the positioning unit 140, the sanitary radio wave reception unit 145, and the direction detection unit 150 of the terminal 700 shown in FIG. Since other configurations are substantially the same as those of the terminal 700 shown in FIG. 21, the corresponding parts are denoted by the same reference numerals and description thereof is omitted. Further, in the following description, for the sake of convenience, it is described as the own terminal 900A and the other terminal 900B.

As shown in FIG. 25, the ultrasonic generator 910 is a means for outputting ultrasonic waves f1 to fp (1 ≦ p) having different wavelengths around the terminal 900A in each direction (eight directions in FIG. 25). . Note that the direction and the number of ultrasonic waves to be output can be appropriately changed according to the required position detection accuracy and the like.
The receiving unit 730 receives the terminal ID and ultrasonic information (described later) of the other terminal 900B instead of receiving the terminal ID, terminal position information, and direction information of the other terminal 900B from the other terminal 900B.

  For example, as shown in FIG. 25, when the other terminal 900B exists diagonally forward to the right when viewed from the own terminal 900A, the other terminal 900B mainly receives the ultrasonic wave f2. When the other terminal 900B determines that the reception intensity of the ultrasonic wave f2 is the highest by comparing the reception intensity of each ultrasonic wave or the like, the other terminal 900B generates ultrasonic information indicating the type and reception intensity of the ultrasonic wave having the highest reception intensity, This is transmitted to the own terminal 900A. Upon receiving the terminal ID and the ultrasonic information from the other terminal 900B, the receiving unit 730 of the own terminal 900A supplies the received terminal ID to the sound data management unit 720, while the received ultrasonic information is used as a relative distance / azimuth calculating unit. Send to 920.

  When the sound data management unit (selection unit) 720 receives the terminal ID (attribute information) via the reception unit 730, the sound data management table TA (storage unit) stores the sound data (for example, the music piece 3) corresponding to the terminal ID. ) Obtained (selected) and supplied to the audio signal generator 160.

  On the other hand, the relative distance / direction calculating unit (relative distance / direction calculating unit) 920 obtains the relative distance and direction between the terminal 900A and the other terminal 900B based on the ultrasonic information supplied from the receiving unit 730. . More specifically, the relative position / direction calculation unit 920 obtains the relative direction (for example, the right front side) of the other terminal from the type of the ultrasonic wave (for example, the ultrasonic wave f2) indicated in the ultrasonic information, A relative distance (for example, ○ × m) is obtained from the reception intensity indicated in the sound wave information. Then, the relative distance / direction calculation unit 920 supplies the obtained relative direction and relative distance to the audio signal generation unit 160. The audio signal generation unit 160 generates an audio signal based on the relative direction and the relative distance supplied from the relative distance / azimuth calculation unit 920 and the sound data supplied from the sound data management unit 720. That is, the audio signal generation unit (localization means) 160 uses the position and orientation of the terminal 900A as a reference, and the sound image of the sound corresponding to the sound data is located at a position separated by the distance obtained in the direction obtained as described above. Generate and process audio signals to localize. Since the subsequent operation can be described in the same manner as each embodiment, it is omitted.

  Thus, you may make it provide the acoustic amusement similar to each embodiment by measuring a relative distance and direction with another terminal. In the above example, the case of obtaining the relative distance and direction in the own terminal 900A has been described. However, in the other terminal 900B, the relative distance and direction from the own terminal 900A are directly obtained, and the obtained relative distance, The direction may be notified to the own terminal 900A. Further, in the above example, an example in which the relative distance and direction with respect to the other terminal 900B are obtained using only the ultrasonic wave is illustrated, but for example, the relative distance and direction with respect to the other terminal using infrared rays or the like can be determined. Further, the relative distance and direction from the other terminal 900B may be obtained by using ultrasonic waves and infrared rays together and using the arrival time difference between the ultrasonic and infrared signals.

(Modification 3)
Since the various functions of each terminal according to this embodiment and the modification described above are realized by the CPU (or DSP) executing the program stored in the memory, the recording medium such as a CD-ROM is used for such a program. It may be recorded and distributed on the Internet or distributed via a communication network such as the Internet.

It is a figure which shows the structure of the sound data delivery system in 1st Embodiment of this invention. It is a figure which shows the structure of the sound data delivery server contained in the sound data delivery system. It is a figure which shows the information memorize | stored in the memory | storage part of the sound data delivery server. It is a figure which shows the data amount conversion table memorize | stored in the memory | storage part of the sound data delivery server. It is a figure which shows the structure of the terminal contained in the sound data delivery system. It is a figure which shows structures, such as an audio signal production | generation part contained in the terminal. It is a figure for demonstrating the process by the audio signal generation part. It is a flowchart which shows operation | movement of the sound data delivery system. It is a flowchart which shows the sounding point selection process which the sound data delivery server performs. It is a flowchart which shows the data amount conversion process which the sound data delivery server performs. It is a figure which shows the mode of the sounding point selected by the sound data delivery server. It is a figure for demonstrating the sound image localization by the audio signal which the terminal produces | generates. It is a figure for demonstrating the same sound image localization. It is a figure for demonstrating the same sound image localization. It is a figure which shows the structure of the sound data delivery system in 2nd Embodiment of this invention. It is a figure which shows the information memorize | stored in the sound data delivery server contained in the sound data delivery system. It is a figure which shows the information memorize | stored in the sound data delivery server. It is a figure which shows the information memorize | stored in the control server contained in the sound data delivery system. It is a flowchart which shows operation | movement of the sound data delivery system. It is a figure which shows the structure of the sound data system in 3rd Embodiment of this invention. It is a figure which shows the structure of the terminal. It is the figure which illustrated the contents of the sound data management table. It is a figure which shows the structure of the terminal in 4th Embodiment of this invention. It is a figure which shows the structure of the terminal in the modification 2 of this invention. It is a figure for demonstrating the ultrasonic wave generate | occur | produced from the ultrasonic generator.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100,700,800,900 ... Terminal, 110 ... Control part, 120 ... Instruction input part, 130 ... Wireless communication part, 140 ... Positioning part, 145 ... Satellite radio wave receiving part, 150 ... Direction detection part, 160 ... Audio signal generation 170, processing unit, 172 ... parameter generation unit, 173 ... delay parameter generation unit, 174 ... amplifier parameter generation unit, 176 ... delay unit, 178 ... amplifier, 180 ... mixing unit, 190 ... audio signal output unit, 200 ... Headphone, 210 ... Direction sensor, 220, 230 ... Sound emitting unit, 300, 610A, 610B, 610C ... Sound data distribution server, 310 ... Control unit, 320 ... Communication unit, 330 ... Storage unit, 400 ... Satellite group, 500 ... Mobile communication network, 510 ... base station, 600 ... control server, 710, 810 ... storage unit, 720 ... sound Data management unit, 730 ... receiver, 740 ... transmitting unit storage section, 811 ... own sound data storage section, 812 ... other sound data storage section, 910 ... ultrasonic generator, 920 ... the relative distance and orientation calculation unit.

Claims (2)

Receiving means for receiving terminal identification information for identifying the other terminal, terminal position information representing the position of the other terminal, direction information representing the direction of the other terminal, from another terminal constituting the sound system;
Position / orientation detection means for detecting the position and orientation of the terminal constituting the sound system;
Storage means for storing a plurality of sound data in advance and storing the terminal identification information corresponding to each sound data in advance as a selection condition for each sound data in association with each sound data;
Selecting means for selecting, from the storage means, sound data stored in association with the received terminal identification information of the other terminal ;
Localization means for localizing the sound image of the sound corresponding to the selected sound data at the position and orientation of the other terminal indicated in the received terminal location information and orientation information with reference to the location and orientation of the terminal itself And a sound image localization terminal comprising: Receiving means for receiving terminal identification information for identifying the other terminal from another terminal constituting the sound system;
Storage means for storing a plurality of sound data in advance and storing the terminal identification information corresponding to each sound data in advance as a selection condition for each sound data in association with each sound data;
Selecting means for selecting, from the storage means, sound data stored in association with the received terminal identification information of the other terminal ;
A relative distance / direction calculating means for obtaining a relative distance and direction between the own terminal and another terminal constituting the sound system;
Localization means for localizing a sound image of the sound corresponding to the selected sound data at a position separated by the determined distance in the determined direction with reference to the position and orientation of the terminal. Sound image localization terminal.

Images