JP2010152570A - Pointing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a pointing system in which an object can be pointed more intuitively. <P>SOLUTION: A receiver 2 detects an azimuth (deviation angle from an absolute azimuth) directed with own device, based on a modulated voice output from a transmitter 1, and detects a relative azimuth between the own device and an operator 3, based on a voice output from the operator 3. the receiver 2 detects an azimuth of the operator 3 (directed azimuth of the operator 3 with respect to the receiver 2), based on the azimuth of the own device and the relative azimuth, and detects an azimuth indicated by the operator 3. The modulated voice output from the transmitter 1 includes information about an azimuth of the object, and the receiver 2 provides information (for example, voice guide or the like) about the object to a user, when the object exists in the azimuth indicated by the operator 3. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a pointing system including an operation element (pointing device) for pointing an object.

2. Description of the Related Art Conventionally, an apparatus that provides a user with information (for example, image information, text information, voice information, etc.) corresponding to an object (real object) pointed to by the user has been proposed (see, for example, Patent Document 1).
JP 2000-194726 A

  However, in the apparatus of Patent Document 1, an object in the field of view is recognized by image recognition using a CCD camera. Further, the position of the object is detected using an infrared beacon and an IR sensor. For this reason, there are many erroneous detections, and there is a problem that the processing load is high. There is also a problem that the number of parts related to the optical sensor is large and the cost is high.

  Furthermore, the operation for pointing the object is performed by moving the pointer on the screen using the mouse, and the object cannot be pointed intuitively.

  Accordingly, an object of the present invention is to provide a pointing system capable of pointing an object more intuitively.

  The pointing system of the present invention includes a transmitter, a receiver, and an operator for a user to point to an object. The transmitter includes a transmission-side sound emitting unit that emits a transmission-side sound on which information indicating a sound emitting direction is superimposed, and the operator includes an operator-side sound emitting unit that emits an operator-side sound. . The information indicating the sound emitting direction is a deviation angle from a certain reference direction.

  The receiver includes a plurality of sound collecting means (microphones) for collecting sound and a demodulating means for demodulating information indicating the sound emitting direction from the collected transmission side sound. Further, the receiver includes own device direction detecting means for detecting the direction in which the own device is directed based on the information indicating the demodulated sound emitting direction and the sound collection time difference of the transmission side sound of each microphone. For example, by obtaining the difference in sound collection time between two microphones, the relative orientation of the transmitter and the receiver can be obtained, and the direction of the device (the deviation angle from the reference orientation) is obtained from information indicating the sound emitting direction of the transmitter. ).

  In addition, the receiver includes operating element direction detecting means for detecting the relative direction between the own apparatus and the operating element based on the sound collection time difference between the operating element side sounds of the microphones. Then, the receiver detects the azimuth pointed to by the manipulator based on the azimuth that the device is facing and the relative azimuth of the manipulator. That is, the orientations of the own device and the operation element can be detected, and the direction indicated by the operation element can be detected.

  In this way, since the azimuth pointed by the operator can be detected only by voice, there are few false detections, the processing load is low, and the cost can be further reduced. In addition, the object can be pointed directly with an operator held by the user, and the object can be pointed more intuitively.

  Moreover, the transmission-side sound emitting means of the transmitter may emit the sound by superimposing the direction of the object and information on the object on the transmission-side sound. In this case, the demodulating means of the receiver demodulates information about the direction of the object and the object. Then, the receiver reproduces information about the target object when the direction indicated by the operation element matches the direction of the target object.

  According to the present invention, the position of the pointing device can be detected only by voice, and the object can be pointed more intuitively.

  An embodiment according to the pointing system of the present invention will be described. FIG. 1 is a diagram illustrating an overall configuration of a pointing system.

  The pointing system includes a transmitter 1, a receiver 2, and an operator 3. The transmitter 1 is fixedly installed indoors and emits modulated sound (for example, ultrasonic waves) having directivity in a predetermined direction. The receiver 2 is a headset terminal device attached to the user's head and includes a plurality of microphones. The operation element 3 is a pointing device such as an electronic pen, and is used by the user to point to the object 41 or the object 42 with the hand. A speaker is attached to the operation element 3 and emits ultrasonic waves having a frequency different from that of the transmitter 1.

  The receiver 2 detects the azimuth (deviation angle from the reference azimuth) of the own device based on the modulated sound output from the transmitter 1 and operates with the own device based on the sound output from the operator 3. Relative orientation with respect to the child 3 (which orientation the operation element 3 faces with respect to the receiver 2) is detected. The receiver 2 detects the absolute azimuth (shift angle from the reference azimuth) indicated by the operator 3 from the azimuth of the device itself and the relative azimuth. The modulated sound output from the transmitter 1 includes information related to the direction of the object. If the object exists in the absolute direction indicated by the operation element 3, the receiver 2 is related to the object. (For example, voice guidance) is provided to the user. Therefore, the pointing system of the present embodiment can obtain information on an object simply by the user pointing to the object with the operation element 3.

Hereinafter, a detailed configuration and operation of the pointing system of the present embodiment will be described.
The receiver 2 includes a speaker 21 </ b> L and a speaker 21 </ b> R that are in contact with both ears of the user, and the speaker 21 </ b> L and the speaker 21 </ b> R are connected by a head strap 22. On the head strap 22, a microphone 23, a microphone 24, and a microphone 25 are provided. The microphone 23 is attached to the left side, and the microphone 24 is attached to the right side. The microphone 25 is attached to the central portion (the top position). The receiver 2 detects the azimuth of its own device based on the difference in the sound collection times of the modulated sounds in the three microphones. Details will be described later.

  FIG. 2A is a block diagram showing the configuration of the transmitter 1, and FIG. 2B is a block diagram showing the configuration of the operator 3. The transmitter 1 outputs two-channel audio, and includes an object information storage unit 11, a reading unit 12, a modulator 13, and a PN code generator 14 for each channel. The sound of each channel is added by the adder 15 and emitted from the speaker 16. The PN code generator 14 for each channel generates a different series of PN codes. Note that the number of channels is not limited to two. The PN code generator 31 of the operator 3 also generates a different series of PN codes and emits sound from the speaker 33.

  The object information storage unit 11 of the transmitter 1 stores information related to the object. FIG. 3 is a diagram illustrating an example of the content of information stored in the object information storage unit 11. Information about the object 41 is stored in the object information storage unit 11 of the channel 1, and information about the object 42 is stored in the object information storage unit 11 of the channel 2. As shown in FIG. 3, information on the object includes a name, a category, a priority, detailed information, and the like. Of the information related to the object, the name is text data of the name of the object. In this embodiment, “apple” is stored as the name of the object 41, and “mandarin orange” is stored as the name of the object 42. The category is information representing the type of object. In the present embodiment, the category “fruit” is stored as the category of the object 41 and the object 42. The priority is information indicating the importance of the object. When two objects are pointed at the receiver 2 at the same time, information is provided in descending order of importance. The detailed information is guide information or the like that shows details of the object, and is, for example, text data or voice data.

  The object information storage unit 11 of the channel 1 stores information related to the sound emitting direction of the transmitter 1 and the directions of the object 41 and the object 42. These azimuths are represented by deviation angles from a reference azimuth (for example, true north). It should be noted that not only channel 1 but also information regarding these directions may be stored in all channels. Moreover, the information regarding the azimuth | direction of a target object may be contained in the information regarding the target object of each corresponding channel.

  The readout unit 12 of each channel reads out the various information stored in the object information storage unit 11 and inputs the information to the modulator 13. The modulator 13 modulates the output ultrasonic wave and superimposes various input information. The modulator 13 superimposes and outputs the PN code input from the PN code generator 14 on the ultrasonic wave.

  FIG. 4 is a block diagram illustrating a configuration of the receiver 2. Audio signals collected by the microphone 23, the microphone 24, and the microphone 25 are input to the demodulator 53, the demodulator 54, and the demodulator 55, respectively. These audio signals are also input to the demodulator 63, demodulator 64, and demodulator 65.

  The demodulator 53, the demodulator 54, and the demodulator 55 receive the PN code of channel 1 from the PN code generator 51. The demodulator 53, the demodulator 54, and the demodulator 55 correlate the PN code input from the PN code generator 51 and the input voice signal, and extract the peak timing of the correlation. The demodulator 53, the demodulator 54, and the demodulator 55 output information indicating the peak timing to the own apparatus direction detection unit 56. This time difference in peak timing corresponds to the sound collection time difference of each microphone. Further, the demodulator 55 demodulates information related to the direction (sound emitting direction) among various information superimposed on the audio signal collected by the microphone 25 and outputs the demodulated information to the own apparatus direction detection unit 56. Information about the direction may be demodulated by another demodulator 53 or demodulator 54. However, since the microphone 25 provided at the center of the head strap 22 is considered to have the best sound collection characteristics, the demodulator 55 It is desirable to demodulate with.

  The own apparatus direction detection unit 56 calculates the direction (relative angle Δθ) of the own apparatus with respect to the sound emitting direction of the transmitter 1 based on the time difference between the peak timings input from the demodulator 53, the demodulator 54, and the demodulator 55. To do. When the sound emitting azimuth of the transmitter 1 is added to the calculated relative angle Δθ, the azimuth of the receiver 2 (shift angle from the reference azimuth) can be obtained.

The direction detection method of the receiver 2 will be described with reference to FIG. The difference in sound collection time of each microphone corresponds to the distance from the transmitter. FIG. 5A illustrates the distance based on the time difference. If the distance based on the time difference between the sound collection timing T24 of the microphone 24 and the sound collection timing 23 of the microphone 23 is ΔL, and the distance based on the time difference between the sound collection timing T24 of the microphone 24 and the sound collection timing 25 of the microphone 25 is ΔR. ,
ΔL = (T23−T24) × V
ΔR = (T25−T24) × V
(Where V is the speed of sound). When the distance between the microphone 23 and the microphone 24 is L, the relative angle Δθ between the sound emitting direction of the transmitter 1 and the direction of the receiver 2 is
sin Δθ = ΔL / L
And Δθ is
Δθ = sin ⁻¹ (ΔL / L) Equation 1
It becomes.

However, Formula 1 is established when 0 ° ≦ Δθ ≦ 90 ° and 270 ° ≦ Δθ ≦ 360 °. When the relative angle Δθ is 90 ° ≦ Δθ ≦ 270 °,
Δθ = sin ⁻¹ (ΔL / L) + 180 ° Formula 2
It becomes. Whether Δθ is in the range of Formula 1 or Formula 2 can be determined by the magnitude relationship between ΔR and ΔL / 2. As shown in FIG. 4B, when “0 ° ≦ Δθ ≦ 90 ° and 270 ° ≦ Δθ ≦ 360 °”, ΔR> ΔL / 2, and when “90 ° ≦ Δθ ≦ 270 °”, ΔR <ΔL / 2. By this determination, it is possible to distinguish two modes having the same ΔL and different directions as shown in FIG.

  The own apparatus direction detection unit 56 can obtain the direction θ of the receiver 2 by adding the sound emitting direction θs to the relative angle Δθ calculated as described above. Information on the calculated orientation θ of the receiver 2 is input to the control unit 58.

  In FIG. 4, the demodulator 63, the demodulator 64, and the demodulator 65 also correlate the PN code input from the PN code generator 50 and the input voice signal, and extract the correlation peak timing. . The PN code output from the PN code generator 50 is the same PN code as the PN code generator 31 of the operator 3. The demodulator 63, demodulator 64, and demodulator 65 output information indicating the extracted peak timing to the operator orientation detection unit 66. The operating element direction detection unit 66 detects the relative direction θp between the receiver 2 and the operating element 3 using the method shown in FIG. It is assumed that the control 3 is present in front of the receiver 2 at θp = 0 °. Information on the relative orientation θp is also input to the control unit 58.

  The control unit 58 displays a target icon as shown in FIG. 6 on the monocular display 26 according to the input relative orientation θp. The center of the screen of the monocular display 26 corresponds to the relative orientation θp = 0 °, the right end of the screen corresponds to the relative orientation θp = 280 °, and the left end of the screen corresponds to the relative orientation θp = 80 °. is doing. Therefore, the control unit 58 displays the target icon at a position in the screen corresponding to the input relative orientation θp. When the operation element 3 moves, the control unit 58 also moves and displays the target icon.

  In the present embodiment, an example in which only the azimuth is detected by three microphones is shown, but the angle in the height direction can also be detected by the method shown in FIG. In that case, the target icon in the screen of the monocular display 26 may be moved in the vertical direction in accordance with the movement of the operator 3.

  Thus, by displaying the target icon in the screen of the monocular display 26, it becomes a standard when the user points to the object using the operation element 3. The size of the target icon is set as an object search range (for example, ± 3 °).

  Next, the control unit 58 adds the relative orientation θp input from the operator orientation detection unit 66 and the orientation θ of the own device input from the own device orientation detection unit 56, and the absolute orientation of the operator 3 Determine the deviation angle from the reference orientation. Information on the orientation of each object is input from the demodulator 55 to the control unit 58. That is, the demodulator 55 demodulates information related to the orientation of the object among various information superimposed on the audio signal collected by the microphone 25 and outputs the demodulated information to the control unit 58. The control unit 58 compares the information regarding the orientation of each object input from the demodulator 55 with the absolute orientation of the operation element 3, and near the absolute orientation of the operation element 3 (± 3 ° within the range of the target icon). ), The channel of the object (for example, channel 2 for the object 42) is selected.

  The control unit 58 outputs information regarding the selected channel to the demodulator 59. The demodulator 59 demodulates information on the selected channel. The demodulated information is input to the control unit 58.

  The control unit 58 displays various information on the monocular display 26 from the input information. For example, as shown in FIG. 6B, when the orientation of the object 41 corresponds to the vicinity of the absolute orientation of the operation element 3, channel 1 is selected, and various information of channel 1 is input, information on channel 1 Display the window. For example, “There are“ apples ”in this direction. Do you want to display detailed information? Or the like.

  Here, when the user operates the switch 32 of the operation element 3, the control unit 58 displays detailed information of the object 41. When the detailed information is audio data, the control unit 58 outputs the audio data to the audio processing unit 71, and the audio processing unit 71 emits sound from the speaker 21L and the speaker 21R. In addition, when detailed information is text data, not only it displays on the monocular display 26, but the audio | voice processing part 71 may synthesize | combine audio | voice from the said text data, and you may make it emit a sound.

  In addition, although any detection method of the switch operation of the operation element 3 may be used, for example, it is performed as follows. That is, the PN code generator 31 of the operator 3 changes the frequency of the sound to be output when the switch 32 is operated. The receiver 2 detects the frequency change of the sound output from the operation element 3 and determines that the switch 32 has been operated when the frequency change occurs.

  Further, it is possible to detect a user operation without a switch. For example, in the receiver 2, the frequency change of the sound output from the operation element 3 is detected, and when the frequency becomes higher, it is determined that the operation element 3 has approached. This is because the frequency increases due to the Doppler effect when the operating element 3 approaches. Similarly, when the frequency becomes low, it is determined that the operation element 3 has moved away. By detecting the speed, amount, time, and the like of this frequency shift, the operation mode of the operation element 3 can be detected. For example, it is only necessary that the user performs a selection operation when the user quickly shakes away.

  As shown in FIG. 6C, when the object 41 and the object 42 are present in the same direction, information is provided in order from the most important channel. In this case, the monocular display 26 may be configured to display information windows in descending order of importance, and display information windows having low priority in order (cyclic display).

  As described above, in the pointing system according to the present embodiment, it is possible to detect the azimuth indicated by the operation element only by voice. Therefore, there are few false detections, the processing load is low, and the cost can be suppressed. In addition, the object can be pointed directly with an operator held by the user, and the object can be pointed more intuitively.

It is a figure which shows the whole structure of a pointing system. It is a block diagram which shows the structure of a transmitter. It is the figure which showed the example of the content of the information memorize | stored in the target object information storage part. It is a block diagram which shows the structure of a receiver. It is a figure explaining the direction detection method of the receiver. It is the figure which showed the content of the various information displayed on the monocular display.

Explanation of symbols

1-transmitter 2-receiver 3-controller

Claims (2)

A pointing system comprising a transmitter, a receiver, and an operator for a user to point to an object,
The transmitter includes a transmission-side sound emitting means for emitting a transmission-side sound on which information indicating a sound emission direction is superimposed,
The operation element includes an operation element side sound emitting means for emitting an operation element side sound,
The receiver includes a plurality of sound collecting means for collecting sound;
Demodulating means for demodulating information indicating the sound emitting direction from the collected transmission side voice;
Self-device azimuth detecting means for detecting the azimuth in which the self-device is directed based on information indicating the sound emitting direction demodulated by the demodulating means and the sound collection time difference of the transmitting side sound collected by the plurality of sound collecting means When,
Based on the sound collection time difference of the operator side sound collected by the plurality of sound collection means, the operator orientation detection means for detecting the relative orientation of the own device and the operator,
Operating element position detecting means for detecting an azimuth directed by the operating element based on an azimuth facing the own device and a relative azimuth of the operating element;
Pointing system with The transmitter-side sound emitting means of the transmitter emits a sound by superimposing the direction of the object and information on the object on the transmitter-side voice,
The demodulating means of the receiver demodulates the direction of the object and the information on the object from the transmission side sound collected by the sound collecting means,
The pointing system according to claim 1, wherein the receiver further includes a reproducing unit that reproduces information related to the object when the direction indicated by the operator matches the direction of the object.

Images