JP5462656B2 - Pointing system - Google Patents

Description

  The present invention relates to a pointing system that estimates a position on a screen indicated by a pointing device, transmits the estimated position information to a display device, and displays an indication point.

  Conventionally, a pointer or a laser pointer is used to indicate an object in a presentation. In addition, there is a system in which an indication bar or a laser pointer itself is photographed by an infrared camera, a direction indicated by image processing is calculated, and an indication point is displayed.

  In the technique disclosed in Patent Literature 1, ultrasonic waves are transmitted and received between the tip of the pointer and a plurality of positions on the screen, the distance is obtained based on the time required for the transmission and reception, and the plane of the pointer The position coordinates are estimated and the indicated point is displayed. In the technique disclosed in Patent Document 2, infrared and ultrasonic waves are simultaneously output from a pointing device, and the infrared rays are received by a photodiode on the screen, and the ultrasonic waves are received by two microphones on the screen, respectively. . Then, the distance from the microphone to the device is obtained based on the time difference from when the infrared ray is received until the ultrasonic wave is received, and the indication point is displayed.

JP 2001-125740 A JP 2005-122534 A

  However, with pointing bars and laser pointers, it is necessary to lay a light receiving element on the surface of the screen or photograph the pointing point with a camera in order to detect the pointing point, and the price of the pointing system will increase significantly. . It is difficult to construct a system for identifying a plurality of pointing devices by embedding identifiers in light waves. Furthermore, when there is a shield, it is impossible to indicate the object itself.

  In addition, in a system in which the pointing rod or the laser pointer itself is photographed with an infrared camera, the pointing direction is calculated by image processing, and the pointing point is displayed, the infrared camera can perform position measurement by image processing in a dark indoor place. When there is an overlap, such as when there is an obstacle, the estimation accuracy is likely to deteriorate.

  In addition, the technique disclosed in Patent Document 1 estimates the plane position coordinates on the screen where the tip of the pointing bar contacts and displays the pointing point, so that the usage scene is limited to the case where the pointing bar is close to the screen. End up. For this reason, the user must approach the screen, and this technique cannot be used when the user is away from the screen. In the technique disclosed in Patent Document 2, infrared rays are used for time synchronization in transmission and reception, but it is necessary to attach an infrared transceiver to the pointing system. For this reason, the pointing device becomes large, and the pointing system becomes expensive. Furthermore, if there is a shield or the like between the pointing device and the light receiving unit, the estimation accuracy is likely to deteriorate.

  Although a method of transmitting sound waves from a plurality of transmitting devices at the same time is also conceivable, sound waves generated from different transmitting devices cannot be distinguished by the receiving device, and the distance between the transmitting device and the receiving device is accurate. Cannot be calculated. On the other hand, if sound waves are transmitted from a plurality of transmission devices at different times, it is highly conceivable that the device moves during that time, and in that case, the estimation accuracy of the indication point is lowered. Furthermore, unmodulated sound waves tend to change depending on the surrounding environment, and there is a problem in the accuracy of calculating the arrival time difference between sound waves, particularly in a noisy environment.

  The present invention has been made in view of such circumstances, and by obtaining the orientation (direction) of the pointing device, the pointer function is realized even when the pointing device is away from the screen. In addition, by not using infrared rays, a small, inexpensive and highly convenient pointing system is provided. In particular, in a pointing system that receives sound waves on the pointing device side, an inexpensive and power-saving pointing device is realized. Furthermore, it is possible to detect the position coordinates to be passed to the computer as the indication point without laying down the light receiving elements or installing a camera. Further, a system for identifying a plurality of pointing devices can be constructed by adding identifiers to the pointing devices. Furthermore, the pointer function can be exhibited even if there is a shield.

  In addition, by obtaining the sound wave arrival time using the modulated sound wave, the measurement accuracy is improved even in a noisy environment or in a situation where the pointing device is moving, and the modulated sound wave is simultaneously transmitted from a plurality of sound wave transmitting devices. By enabling the transmission, it is possible to display the indication point with high accuracy and high speed.

  (1) In order to achieve the above object, the present invention takes the following measures. That is, the pointing system of the present invention is a pointing system for estimating the position on the screen pointed to by the pointing device, and is provided at a plurality of positions different from each other in the space, and the sound wave modulated by the different pseudo-random signals. A plurality of sound wave transmitting devices that transmit the sound wave, a plurality of sound wave receiving devices that are provided at arbitrary positions on the pointing device, and receive the sound waves, and perform a correlation process on the received modulated sound waves received by each of the sound wave receiving devices, A distance calculation unit that calculates an arrival time of the modulated sound wave, calculates a distance between each sound wave transmission device and each sound wave reception device based on the arrival time, and each sound wave transmission device and each sound wave reception device Based on the distance, a position estimating unit that estimates the position of each sound wave receiving device in space, and each coordinate of the estimated sound wave receiving device A straight line connecting two arbitrary points, or a straight line connecting an arbitrary point of each coordinate of the estimated sound wave receiving device and the center of gravity of each coordinate of the estimated sound wave receiving device, and the direction of the pointing device which is known A direction vector that specifies the pointing direction of the pointing device is estimated based on the angle difference from the vector shown, and passes through an arbitrary point on the pointing device and is a straight line and plane parallel to the estimated direction vector. An instruction point estimation unit for estimating an intersection with the screen.

  In this way, the ultrasonic wave is modulated with a pseudo-random signal, the modulated sound wave transmitted from each sound wave transmitting device is received by each sound wave receiving device, and correlation processing is performed on the received modulated sound wave to determine the arrival time of the modulated sound wave. And calculating the distance between each sound wave transmitting device and each sound wave receiving device based on the obtained arrival time, and each sound wave receiving device based on the calculated distance between each sound wave transmitting device and each sound wave receiving device. And estimating a direction vector for specifying the pointing direction of the pointing device based on the estimated position in the space of each sound wave receiving device, and calculating a straight line parallel to the direction vector and the screen. Since the intersection point is estimated, it is possible to accurately grasp the point indicated by the pointing device even if there is an obstacle. In addition, since the sound wave receiving device is provided on the pointing device side, it is possible to reduce the size of the pointing device. Furthermore, by calculating the sound wave arrival time using the modulated sound wave, the sound wave arrival time can be accurately obtained even in a noisy environment or a situation where the pointing device is moving, and the sound waves can be simultaneously transmitted from a plurality of sound wave transmitting devices. Since it is possible to make a call, it is possible to display the indication point with high accuracy and high speed.

  (2) The pointing system of the present invention is a pointing system for estimating the position on the screen pointed to by the pointing device, which is provided at a plurality of positions different from each other in the space, and is modulated by mutually different pseudo-random signals. A plurality of sound wave transmitting devices for transmitting the generated sound waves, a plurality of sound wave receiving devices for receiving sound waves provided at arbitrary positions on the pointing device, and correlation processing on the received modulated sound waves received by each of the sound wave receiving devices. A distance calculating unit that calculates an arrival time of the modulated sound wave, calculates a distance between each of the sound wave transmission devices and each of the sound wave reception devices based on the arrival time, and each of the sound wave transmission devices and each of the sound wave receptions A position estimation unit for estimating a position of each sound wave receiving device in space based on a distance to the device, and each coordinate of the estimated sound wave transmitting device A straight line connecting any two points, or a straight line connecting any one point of each coordinate of the estimated sound wave transmitter and the center of gravity of each coordinate of the estimated sound wave transmitter, and the direction of the known pointing device A direction vector that specifies the pointing direction of the pointing device is estimated based on an angle difference from the vector shown, and passes through an arbitrary point that is not on the pointing device, as a straight line and a plane parallel to the estimated direction vector And a pointing point estimation unit for estimating an intersection with the screen.

  In this way, the ultrasonic wave is modulated with a pseudo-random signal, the modulated sound wave transmitted from each sound wave transmitting device is received by each sound wave receiving device, and correlation processing is performed on the received modulated sound wave to determine the arrival time of the modulated sound wave. And calculating the distance between each sound wave transmitting device and each sound wave receiving device based on the obtained arrival time, and each sound wave receiving device based on the calculated distance between each sound wave transmitting device and each sound wave receiving device. And estimating a direction vector for specifying the pointing direction of the pointing device based on the estimated position in the space of each sound wave receiving device, and calculating a straight line parallel to the direction vector and the screen. Since the intersection point is estimated, it is possible to accurately grasp the point indicated by the pointing device even if there is an obstacle. In addition, since the sound wave receiving device is provided on the pointing device side, it is possible to reduce the size of the pointing device. Furthermore, by calculating the sound wave arrival time using the modulated sound wave, the sound wave arrival time can be accurately obtained even in a noisy environment or a situation where the pointing device is moving, and the sound waves can be simultaneously transmitted from a plurality of sound wave transmitting devices. Since it is possible to make a call, it is possible to display the indication point with high accuracy and high speed.

  (3) The pointing system of the present invention is a pointing system for estimating the position on the screen pointed to by the pointing device, which is provided at an arbitrary position on the pointing device and is modulated with different pseudo-random signals. Correlation processing is performed on a plurality of sound wave transmitting devices that transmit sound waves, a plurality of sound wave receiving devices that are provided at a plurality of different positions in the space and that receive sound waves, and reception modulation sound waves received by each of the sound wave receiving devices. To calculate the arrival time of the modulated sound wave, and based on the arrival time, calculate a distance between each sound wave transmission device and each sound wave reception device, each sound wave reception device and each sound wave transmission device, A position estimation unit that estimates the position of each sound wave transmission device in space based on the distance of each, and an arbitrary coordinate of each of the estimated sound wave transmission devices A straight line connecting two points, or a straight line connecting any one point of each coordinate of the estimated sound wave transmitting device and the center of gravity of each coordinate of the estimated sound wave transmitting device, and a vector indicating the direction of the known pointing device The direction vector specifying the pointing direction of the pointing device is estimated on the basis of the angle difference between the pointing device and the screen as a straight line and a plane passing through an arbitrary point on the pointing device and parallel to the estimated direction vector. And an indicated point estimation unit for estimating an intersection with.

  In this way, the ultrasonic wave is modulated with a pseudo-random signal, the modulated sound wave transmitted from each sound wave transmitting device is received by each sound wave receiving device, and correlation processing is performed on the received modulated sound wave to determine the arrival time of the modulated sound wave. The distance between each sound wave transmitting device and each sound wave receiving device is calculated based on the obtained arrival time, and each sound wave transmitting device is calculated based on the calculated distance between each sound wave transmitting device and each sound wave receiving device. And estimating a direction vector for specifying the pointing direction of the pointing device based on the estimated position in the space of each sound wave transmission device, and calculating a straight line parallel to the direction vector and the screen. Since the intersection point is estimated, it is possible to accurately grasp the point indicated by the pointing device even if there is an obstacle. In addition, by calculating the sound wave arrival time using the modulated sound wave, the sound wave arrival time can be accurately obtained even in a noisy environment or in a situation where the pointing device is moving. Since it is possible to transmit simultaneously, it is possible to display the indication point with high accuracy and high speed.

  (4) The pointing system of the present invention is a pointing system for estimating the position on the screen pointed to by the pointing device, which is provided at an arbitrary position on the pointing device and is modulated with different pseudo-random signals. Correlation processing is performed on a plurality of sound wave transmitting devices that transmit sound waves, a plurality of sound wave receiving devices that are provided at a plurality of different positions in the space and that receive sound waves, and reception modulation sound waves received by each of the sound wave receiving devices. To calculate the arrival time of the modulated sound wave, and based on the arrival time, calculate a distance between each sound wave transmission device and each sound wave reception device, each sound wave reception device and each sound wave transmission device, A position estimation unit that estimates the position of each sound wave transmission device in space based on the distance of each, and an arbitrary coordinate of each of the estimated sound wave transmission devices A straight line connecting two points, or a straight line connecting any one point of each coordinate of the estimated sound wave transmitting device and the center of gravity of each coordinate of the estimated sound wave transmitting device, and a vector indicating the direction of the known pointing device And a direction vector for specifying the pointing direction of the pointing device based on the angle difference between the straight line and the plane as a plane and a plane parallel to the estimated direction vector, passing through an arbitrary point not on the pointing device. An instruction point estimation unit for estimating an intersection with the screen.

  In this way, the ultrasonic wave is modulated with a pseudo-random signal, the modulated sound wave transmitted from each sound wave transmitting device is received by each sound wave receiving device, and correlation processing is performed on the received modulated sound wave to determine the arrival time of the modulated sound wave. The distance between each sound wave transmitting device and each sound wave receiving device is calculated based on the obtained arrival time, and each sound wave transmitting device is calculated based on the calculated distance between each sound wave transmitting device and each sound wave receiving device. And estimating a direction vector for specifying the pointing direction of the pointing device based on the estimated position in the space of each sound wave transmission device, and calculating a straight line parallel to the direction vector and the screen. Since the intersection point is estimated, it is possible to accurately grasp the point indicated by the pointing device even if there is an obstacle. In addition, by calculating the sound wave arrival time using the modulated sound wave, the sound wave arrival time can be accurately obtained even in a noisy environment or in a situation where the pointing device is moving. Since it is possible to transmit simultaneously, it is possible to display the indication point with high accuracy and high speed.

  (5) Moreover, in the pointing system of the present invention, each of the sound wave transmitting devices transmits the sound wave at at least one of a simultaneous time and a different time.

  As described above, each sound wave transmitting device transmits the sound wave at the same time, thereby improving the indication point display speed and realizing a pointing system with quick response. On the other hand, since each sound wave transmitting device transmits the sound wave at a different time, the sound wave receiving device side can easily distinguish which sound wave transmitting device is the sound wave transmitted from the sound wave receiving device side. It becomes possible to improve.

  (6) The pointing system of the present invention further includes a display unit that displays the intersection point on the screen as an instruction point indicated by the pointing device.

  In this way, the intersection point is displayed on the screen as the indication point indicated by the pointing device, so that the user can easily grasp the indication point.

  (7) In addition, the pointing system of the present invention further includes a projection device that projects the screen displayed on the display unit onto the screen.

  In this way, since the screen displayed on the display unit is projected onto the screen, the convenience in presentation can be improved.

  According to the present invention, an ultrasonic wave is modulated with a pseudo-random signal, the modulated sound wave transmitted from each sound wave transmitting device is received by each sound wave receiving device, and the received modulated sound wave is subjected to correlation processing to reach the modulated sound wave. The time is obtained, the distance between each sound wave transmitting device and each sound wave receiving device is calculated based on the obtained arrival time, and each sound wave is calculated based on the calculated distance between each sound wave transmitting device and each sound wave receiving device. Estimating the position of the receiving device in space, and estimating a direction vector for specifying the pointing direction of the pointing device based on the estimated position of the sound wave receiving device in space, and a straight line and a plane parallel to the direction vector Since the point of intersection with the screen is estimated, it is possible to accurately grasp the point indicated by the pointing device even if there is an obstacle. In addition, since the sound wave receiving device is provided on the pointing device side, it is possible to accurately grasp the point to be pointed even when the pointing device is away from the screen. In addition, since the sound wave receiving device is provided on the pointing device side, it is possible to reduce the size of the pointing device. Furthermore, a system for identifying a plurality of pointing devices can be constructed by adding identifiers to the pointing devices. In addition, it is possible to accurately grasp the point indicated by the pointing device even if there is an obstacle. Furthermore, by calculating the arrival time using modulated ultrasound, the ultrasound arrival time can be obtained accurately even in a noisy environment or when the pointing device is moving. Since it is possible to simultaneously transmit from the transmitting device, it is possible to display the indication point with high accuracy and high speed.

1 is a diagram illustrating a schematic configuration of a pointing system according to a first embodiment. It is a block diagram which shows the function of the pointing system 1 which concerns on 1st Embodiment. It is a flowchart which shows operation | movement of the pointing system 1 which concerns on 1st Embodiment. It is a figure which shows a cross correlation value. It is a figure which shows schematic structure of the pointing system which concerns on 2nd Embodiment.

  A pointing system according to an embodiment of the present invention transmits and receives ultrasonic waves modulated between a plurality of speakers installed at known positions and a microphone mounted on a pointing device used by a user, and receives the received ultrasonic waves. Correlation processing is performed on the ultrasonic wave, the arrival time of the ultrasonic wave is obtained, the distance between the speaker and the microphone is calculated, and the position of the microphone attached to the pointing device is estimated. In addition, by attaching a plurality of microphones to the pointing device, it becomes possible to obtain a plurality of position coordinates on the pointing device, estimate the direction (pointing method) of the pointing device, and based on these information, A pointer function for displaying an indication point on a PC screen or screen is realized. As a result, the pointing function can be exhibited even when the pointing device is away from the screen.

  In the present embodiment, only the ultrasonic waves are used without using other wireless means such as infrared rays. As a result, since it is not necessary to attach an infrared transceiver or the like to the pointing device, the pointing device can be reduced in size. Further, in the present embodiment, transmission / reception is performed by modulating an ultrasonic wave with a pseudo-random signal. The pseudo-random signal is a periodic signal artificially generated according to a certain rule, is reproducible, and its value is completely determined if the rule is known. For this reason, the sound wave arrival time can be accurately obtained even in a noisy environment or in a situation where the pointing device is moving, and it is possible to simultaneously transmit sound waves from multiple sound wave transmitting devices. The indication point can be displayed. Even if the speaker and the microphone are interchanged, the essence of the invention does not change.

  In the pointing system according to the embodiment of the present invention, sound waves are used instead of light (electromagnetic waves) such as infrared rays. The sound wave is a concept including any sound range, but preferably uses a band of 18 kHz or more. In this specification, a pointing device using so-called ultrasonic waves is shown as an example, but the technical idea of the present invention is not limited to ultrasonic waves.

  Furthermore, in the pointing system according to the embodiment of the present invention, an M-sequence that is one of pseudo-random signals is used for ultrasonic modulation. In the present specification, M-sequence modulation is shown as an example, but the technical idea of the present invention is not limited to M-sequence modulation.

[First Embodiment]
FIG. 1 is a diagram illustrating a schematic configuration of a pointing system according to the first embodiment. As shown in FIG. 1, the pointing system 1 includes a pointing device 10, a screen 20, a PC (Personal Computer) 30, and a projector 40. The pointing device 10 is provided with a plurality of microphones as sound wave receiving devices at arbitrary positions.

  Specifically, in FIG. 1, two microphones 10 a and 10 b are provided at the distal end portion of the pointing device 10 in the longitudinal direction. In the vicinity of the screen 20, there are provided a plurality of speakers as sound wave transmitting devices that intermittently transmit ultrasonic waves modulated by different M series for each transmitting device. Specifically, in FIG. 1, three speakers 20 a, 20 b, and 20 c are provided at three vertex portions of the screen 20. The microphones 10a and 10b receive M-sequence modulated ultrasonic waves transmitted from the three speakers 20a, 20b, and 20c.

  The pointing device 10 and the PC 30 are connected by a wireless interface or a wired interface, and when M-sequence modulated ultrasonic waves are received by the microphones 10a and 10b of the pointing device 10, data indicating a received wave is transmitted to the PC 30. It is configured as follows. The PC 30 performs correlation processing on the received wave to obtain the arrival time of the M-sequence modulated ultrasonic waves, calculates the distances between the speakers 20a, 20b, and 20c and the microphones 10a and 10b, and the position of the pointing device 10 and its The pointing direction (indicated by P in FIG. 1) and the position of the pointing point of the pointing device 10 on the screen 20 are estimated. The estimated indication point (A1) is displayed on the screen of the PC 30. The PC 30 may instruct the three speakers 20a, 20b, and 20c to transmit M-sequence modulated ultrasonic waves, and when the three speakers 20a, 20b, and 20c transmit M-sequence modulated ultrasonic waves, Information indicating the transmission time may be input to the PC 30.

  The PC 30 is connected to a projector 40 as a projection device via a cable 30a. An image displayed on the screen of the PC 30 is projected onto the screen 20 via the projector 40. At that time, the pointing point A1 of the pointing device 10 estimated in the PC 30 is displayed on the screen 20 as the pointing point A2.

  FIG. 2 is a block diagram illustrating functions of the pointing system 1 according to the first embodiment. The distance calculation unit 30-0 of the PC 30 performs correlation processing on the M-sequence modulated ultrasound received by the microphones 10a and 10b of the pointing device 10 from the speakers 20a, 20b, and 20c, and the arrival time of the M-sequence modulated ultrasound And the distances between the speakers 20a, 20b and 20c and the microphones 10a and 10b are calculated. Further, the position estimation unit 30-1 of the PC 30 estimates the positions of the microphones 10a and 10b in space based on the distances between the speakers 20a, 20b and 20c and the microphones 10a and 10b calculated by the distance calculation unit 30-0. To do.

  Further, the pointing point estimation unit 30-2 specifies a direction vector for specifying the pointing direction (P direction) of the pointing device 10 based on the spatial positions of the microphones 10a and 10b estimated by the position estimation unit 30-1. Is estimated. In addition, the indication point estimation unit 30-2 estimates an intersection A2 between a straight line (P) that passes through an arbitrary point on the pointing device 10 and is parallel to the direction vector and the screen 20 as a plane. Other configurations are the same as described with reference to FIG.

FIG. 3 is a flowchart showing the operation of the pointing system 1 according to the first embodiment. First, in the PC 30, M series for modulating the ultrasonic waves transmitted from the three speakers 20a, 20b, and 20c are respectively generated (step S1). If i = 1,..., 3 are the identifiers of the three speakers 20a, 20b, and 20c, the M series: M _i is expressed by Expression (1).

一方、ｍ_ｉ（ｍ）を生成させる漸化式は、式（２）で表される。

On the other _hand, recursion formula to generate _m i (m) is expressed by equation (2).

ここで、ｇ_ｉ（ｌ）をフィードバック係数と呼び、その値は、０または１である。フィードバック係数は、原始多項式：Ｇ_ｉ（ｘ）として、式（３）で与えられる。

Here, g _i (l) is called a feedback coefficient, and its value is 0 or 1. The feedback coefficient is given by Equation (3) as a primitive polynomial: G _i (x).

ここで、ｎをＭ系列の次数と呼ぶ。Ｍ系列の周期は２^ｎ−１で表せるから、生成するＭ系列の要素数Ｎとは、式（４）で示される関係が成り立つ。

Here, n is called the order of the M sequence. Since the period of the M sequence can be expressed by 2 ⁿ −1, the relationship represented by the equation (4) is established with the number N of elements of the generated M sequence.

本実施形態では、式（４）の関係を満せば、任意の原始多項式を与えても十分である。ただし、３つのスピーカ２０ａ、２０ｂ、２０ｃから発信させる変調超音波は全て異なる信号である必要があるから、式（５）で示される全条件を同時に満たさなければならない。

In the present embodiment, it is sufficient to give an arbitrary primitive polynomial as long as the relationship of Expression (4) is satisfied. However, since all the modulated ultrasonic waves transmitted from the three speakers 20a, 20b, and 20c need to be different signals, all the conditions expressed by the equation (5) must be satisfied at the same time.

次に、ＰＣ３０において、生成したＭ系列：Ｍ_ｉを用い、搬送波である超音波を変調する（ステップＳ２）。本明細書では、一例として、振幅変調法を示すが、本発明の技術的思想は、振幅変調に限定されるわけではない。

Next, the PC 30, the generated M-sequence: with _{M i,} modulates the ultrasonic wave is a carrier wave (step S2). In this specification, an amplitude modulation method is shown as an example, but the technical idea of the present invention is not limited to amplitude modulation.

The carrier waveform used in this embodiment is U (t), the carrier sampling frequency is Q, and the carrier time length is L. Here, for each θ sample of the carrier wave U (t), amplitude modulation is performed using one element of the M series. Therefore, the modulated ultrasonic waves: F _i (t) transmitted from the three speakers 20a, 20b, and 20c are expressed by Expression (6).

ただし、式（７）は、ａを超える最大の整数を表す。

However, Formula (7) represents the largest integer exceeding a.

従って、式（４）より、式（８）が成り立つように、Ｍ系列の次数ｎを決定し、原始多項式を与える。

Therefore, from Equation (4), the order n of the M sequence is determined so that Equation (8) is satisfied, and a primitive polynomial is given.

次に、周波数帯域が１８ｋＨｚ以上になるように、Ｆ_ｉ（ｔ）に帯域制限を施した変調超音波を３つのスピーカ２０ａ、２０ｂ、２０ｃから断続的にそれぞれ発信する（ステップＳ３）。さらに、２つのマイク１０ａ、１０ｂで、３つのスピーカ２０ａ、２０ｂ、２０ｃから発信された変調超音波をそれぞれ受信する（ステップＳ４）。ここで、ｊ＝１，２を２つのマイク１０ａ、１０ｂの識別子として、受信された変調超音波の波形をＰ_ｊ（ｘ）で表す。そして、ポインティングデバイス１０からＰＣ３０に対して、受信波形Ｐ_ｊ（ｘ）を送信する（ステップＳ５）。

Next, modulated ultrasonic waves in which F _i (t) is band-limited are intermittently transmitted from the three speakers 20a, 20b, and 20c so that the frequency band becomes 18 kHz or more (step S3). Further, the two microphones 10a and 10b receive the modulated ultrasonic waves transmitted from the three speakers 20a, 20b and 20c, respectively (step S4). Here, j = 1 and 2 are used as identifiers of the two microphones 10a and 10b, and the received modulated ultrasonic waveform is represented by P _j (x). Then, the received waveform P _j (x) is transmitted from the pointing device 10 to the PC 30 (step S5).

Next, in the PC 30, the received waveform P _j (x) transmitted from the pointing device 10 is subjected to correlation processing, and the modulated ultrasonic waves transmitted from the three speakers 20a, 20b, and 20c are transmitted by the two microphones 10a and 10b. The arrival time _Tij until reception is calculated (step S6). Here, the cross-correlation value: D _ij (τ) is obtained using Expression (9).

ただし、前述のとおりＰＣ３０は変調超音波の発信時刻を別途入手しているため、τをスピーカ２０ａ、２０ｂ、２０ｃから変調超音波が発信されてから経過した時間とすることができる。また、相関処理に用いるＭ系列：ｍ_ｉ（ｍ）の値は、その直流成分をほぼ０にするため、０または１のかわりに−１または１を用いる。そして、Ｄ_ｉｊ（τ）を最大とするτを算出し、その値を到達時間Ａ_ｉｊとする。

However, since the PC 30 separately obtains the transmission time of the modulated ultrasonic wave as described above, τ can be the time that has elapsed since the modulated ultrasonic wave was transmitted from the speakers 20a, 20b, and 20c. In addition, as the value of the M sequence m _i (m) used for the correlation processing, −1 or 1 is used instead of 0 or 1 in order to make the direct current component almost zero. Then, τ that maximizes D _ij (τ) is calculated, and the value is set as arrival time A _ij .

  Here, the M sequence is reproducible, and its value is completely restored if the rules are known. As an example, let us consider a fifth order M sequence M1 and M2 generated from two different primitive polynomials as shown in equation (10).

このとき、Ｍ１、Ｍ２を結合した搬送波：Ｕ（ｔ）を式（１１）で表す。

At this time, a carrier wave U (t) obtained by combining M1 and M2 is expressed by Expression (11).

式（１１）に対し、Ｍ系列：Ｍ１、Ｍ２をそれぞれ用いて相互相関値：Ｄ_ｉ（τ）を求めた結果を図４に示す。

FIG. 4 shows the result of obtaining the cross-correlation value: D _i (τ) for the expression (11) using the M series: M1 and M2, respectively.

As shown in FIG. 4, it can be seen that the cross-correlation value: D _i (τ) has a sharp peak only when the M sequence: M _i is received. Therefore, even in an environment where noise and other sound waves exist, the arrival time A _ij can be calculated with high accuracy. Therefore, even if the modulated ultrasonic waves F _i (t) are simultaneously transmitted from the three speakers 20a, 20b, and 20c, the essence of the invention is not changed as long as the ultrasonic waves are modulated with different M series for each speaker.

Next, the PC 30 calculates the distance r _ij between the three speakers 20a, 20b, and 20c and the two microphones 10a and 10b based on the arrival time A _ij . Here, assuming that the sound speed is C, r _ij is obtained by the equation (12).

式（１３）は、既知の位置に設置されたスピーカの３次元空間座標をそれぞれ（ｕ_ｉ，ｖ_ｉ，ｗ_ｉ）（ｉ＝１，…，ｍ）とし、ポインティングデバイス装着されたマイクの３次元空間座標をそれぞれ（ｘ_ｊ，ｙ_ｊ，ｚ_ｊ）（ｊ＝１，…，ｎ）として、各スピーカと各マイクとの距離ｒ_ｉｊの関係を示す連立方程式を表している。スピーカ及びマイクの数が一般的な場合を示しており、スピーカの数をｍ，マイクの数をｎとする。

Equation (13) is obtained by assuming that the three-dimensional spatial coordinates of the speaker installed at a known position are (u _i , v _i , w _i ) (i = 1,..., M), and 3 of the microphone mounted with the pointing device. The dimensional space coordinates are (x _j , y _j , z _j ) (j = 1,..., N), respectively, and represent simultaneous equations indicating the relationship between the distances r _ij between each speaker and each microphone. This shows a case where the number of speakers and microphones is general, where m is the number of speakers and n is the number of microphones.

ここで、式（１３）の連立方程式は、式数と未知数の関係から、ｍ≧３であれば解くことができる。これにより、既知の位置に設置されたスピーカの数が３つ以上であれば、十分である。

Here, the simultaneous equations of Equation (13) can be solved if m ≧ 3 from the relationship between the number of equations and the unknown. Thus, it is sufficient if the number of speakers installed at known positions is three or more.

The three-dimensional spatial coordinates of the speakers 20a, 20b, and 20c installed at known positions are (u _i , v _i , w _i ) (i = 1, 2, 3), respectively, and the microphone 10a attached to the pointing device 10 is used. 10b is set to (x _j , y _j , z _j ) (j = 1, 2), and based on the equation (13), each speaker 20a, 20b, 20c and each microphone 10a, 10b A simultaneous equation showing the relationship of the distance r _ij is established. By solving these nonlinear simultaneous equations, (x _j , y _j , z _j ) are respectively calculated for the three-dimensional spatial coordinates of the microphones 10a, 10b attached to the pointing device 10 (step S7).

ここで、既知の位置に設置されたスピーカの数が４つ以上ある場合、式（１３）で示した連立方程式において、式数が未知数を上回り、方程式の解として複数の候補（ｘ_ｊ，ｙ_ｊ，ｚ_ｊ）＝（α_ｋ，β_ｋ，γ_ｋ）（ｋ＝１，…，ｍ！／３！（ｍ−３）！）が得られるが、式（１５）で示す誤差δ_ｋが最小となる（α_ｋ，β_ｋ，γ_ｋ）を、ポインティングデバイス１０に装着された各マイクの３次元空間座標（ｘ_ｊ，ｙ_ｊ，ｚ_ｊ）として採用する。

Here, when there are four or more speakers installed at known positions, in the simultaneous equations shown in Equation (13), the number of equations exceeds the unknown, and a plurality of candidates (x _j , y _j , z _j ) = (α _k , β _k , γ _k ) (k = 1,..., m! / 3! (m−3)!), the error δ _k shown in equation (15) is The minimum (α _k , β _k , γ _k ) is adopted as the three-dimensional spatial coordinates (x _j , y _j , z _j ) of each microphone attached to the pointing device 10.

On the other hand, when the time is not synchronized between the speaker and the microphone, a certain distance difference is generated in the distance between each speaker and each microphone. In this case, a simultaneous equation showing the relationship of the distance r _ij is established with T as the time difference between the speaker and the microphone. This is expressed by equation (15).

ここで、式（１５）の連立方程式は、式数と未知数の関係から、ｍ≧４であれば解くことができる。これにより、スピーカとマイクとの間で時刻の同期が取れていない場合は、既知の位置に設置されたスピーカの数が４つ以上であれば、十分である。

Here, the simultaneous equations of Equation (15) can be solved if m ≧ 4 from the relationship between the number of equations and the unknown. Thereby, when the time synchronization is not taken between the speaker and the microphone, it is sufficient if the number of speakers installed at known positions is four or more.

  Next, a direction vector representing the direction of the pointing device 10 is obtained from the three-dimensional space coordinates of the plurality of microphones 10a and 10b attached to the pointing device 10, and the direction of the pointing device 10 is estimated (step S8). For example, as shown in FIG. 1, when two microphones 10a and 10b are attached to both ends of the pointing device 10, the direction vector is given by Expression (16).

ここで、ポインティングデバイス１０に装着されているマイク１０ａ、１０ｂの数が３つ以上の場合や、ポインティングデバイス１０の両先端以外に装着されている場合であっても、ポインティングデバイス１０上でのそれらの相対的位置関係は既知であるため、それらを考慮して方向ベクトルを求めれば十分である。

Here, even when the number of microphones 10a and 10b attached to the pointing device 10 is three or more, or even when attached to other than both ends of the pointing device 10, those on the pointing device 10 are used. Since the relative positional relationship is known, it is sufficient to obtain the direction vector in consideration of them.

Then, as shown in Expression (17), a linear equation that passes through an arbitrary point (x _p , y _p , z _p ) in space and is parallel to the direction vector given in Expression (16) is established.

ここで、（ｘ_ｐ，ｙ_ｐ，ｚ_ｐ）は指示点を得るために算出する方向ベクトルに平行な直線の起点である。例えば、ポインティングデバイス１０の利用者側の端にマイク１０ａがある場合は、（ｘ_ｐ，ｙ_ｐ，ｚ_ｐ）＝（ｘ_１，ｙ_１，ｚ_１）とすれば良い。なお、ポインティングデバイス１０から外れた点を（ｘ_ｐ，ｙ_ｐ，ｚ_ｐ）としても、発明の本質は変わらない。

Here, (x _p , y _p , z _p ) is the starting point of a straight line parallel to the direction vector calculated to obtain the indication point. For example, if there is a microphone 10a at the end of the pointing device 10 on the user side, (x _p , y _p , z _p ) = (x ₁ , y ₁ , z ₁ ) may be used. Note that even if the point deviated from the pointing device 10 is (x _p , y _p , z _p ), the essence of the invention does not change.

Finally, the pointing point (X, Y) of the pointing device 10 is estimated by obtaining the intersection of the straight line equation given by the equation (17) and the pointing screen plane (screen 20) (step S9). For example, when (x _p , y _p , z _p ) = (x ₁ , y ₁ , z ₁ ) and the instruction screen (screen 20) is at z = 0, the intersection point can be obtained by equation (18). It is.

そして、推定された指示点（Ｘ，Ｙ）をＰＣ３０の画面上に表示すると共に、スクリーン２０上に表示する（ステップＳ１０）。

Then, the estimated indication point (X, Y) is displayed on the screen of the PC 30 and also on the screen 20 (step S10).

  As described above, according to the first embodiment, the ultrasonic waves are modulated in the M series by the PC 20, the modulated ultrasonic waves are transmitted by the speakers 20a, 20b, and 20c, and the modulated ultrasonic waves are received by the microphones 10a and 10b. Then, the received modulated ultrasonic wave is subjected to correlation processing to obtain the arrival time of the modulated ultrasonic wave, and the distance between the speakers 20a, 20b, 20c and the microphones 10a, 10b is calculated based on the obtained arrival time, Based on the calculated distance between the speakers 20a, 20b, 20c and the microphones 10a, 10b, the position of the microphones 10a, 10b is estimated in space, and the pointing device 10 is determined based on the estimated position of the microphones 10a, 10b in space. A direction vector that specifies the designated direction of the pointing device 10 is estimated, and an arbitrary point on the pointing device 10 is Since estimates the intersection of the screen 20 as a straight line and a plane, it is possible to accurately grasp the point instructing pointing device 10 even when there is an interrupting object.

  Further, since the microphones 10a and 10b are provided on the pointing device 10 side, the pointing device 10 can be downsized. Further, by calculating the arrival time using the modulated ultrasonic wave, the ultrasonic arrival time can be accurately obtained even in a noisy environment or a situation where the pointing device is moving, and the modulated ultrasonic wave is obtained from the speaker 20a, Since it is possible to simultaneously transmit from 20b and 20c, it is possible to display the indication point with high accuracy and high speed.

[Second Embodiment]
FIG. 5 is a diagram illustrating a schematic configuration of a pointing system according to the second embodiment. As shown in FIG. 5, the pointing system 2 according to the second embodiment is different from the first embodiment in that a microphone and a speaker are interchanged. That is, the pointing device 50 is provided with a plurality of speakers as sound wave transmitting devices at arbitrary positions. Specifically, in FIG. 5, two speakers 50 a and 50 b are provided at the distal end portion of the pointing device 50 in the longitudinal direction. In the vicinity of the screen 20, a plurality of microphones are provided as sound wave receiving devices that receive modulated ultrasonic waves. Specifically, in FIG. 5, three microphones 60 a, 60 b, and 60 c are provided at three vertex portions of the screen 20. Generation of modulated ultrasonic waves transmitted from the two speakers 50a and 50b is performed in the PC 50 in the same manner as in the first embodiment. The three microphones 60a, 60b, and 60c receive the modulated ultrasonic waves transmitted from the two speakers 50a and 50b.

  The pointing device 50 and the PC 30 are connected by a wireless interface or a wired interface so that when modulated ultrasonic waves are transmitted from the speakers 50a and 50b of the pointing device 50, data indicating the transmission time is transmitted to the PC 30. It is configured. Although not shown, when the modulated ultrasonic waves transmitted from the speakers 50a, 50b are received by the microphones 60a, 60b, 60c, data indicating the received waveform of the modulated ultrasonic waves is transmitted to the PC 30. . After these pieces of information are input to the PC 30, operations similar to those in the first embodiment are performed.

  As described above, according to the second embodiment, the PC 20 modulates ultrasonic waves with different M-sequences for each transmitter, intermittently transmits the modulated ultrasonic waves with the speakers 50a and 50b, and the microphones 60a and 60b. 60c, the modulated ultrasonic waves are received, correlation processing is performed on the received modulated ultrasonic waves to obtain the arrival time of the modulated ultrasonic waves, and the speakers 50a and 50b and the microphones 60a and 60b are obtained based on the obtained arrival times. The distance between the speakers 50a and 50b is calculated based on the calculated distance between the speakers 50a and 50b and the microphones 60a, 60b and 60c, and the estimated space between the speakers 50a and 50b is calculated. Based on the above position, a direction vector that specifies the pointing direction of the pointing device 50 is estimated, and a straight line and a plane parallel to the direction vector are estimated. Since estimates the intersection with the lean 20, it is possible to accurately grasp the point at which an instruction is a pointing device 50 even when there is an interrupting object.

  Further, by calculating the arrival time using the modulated ultrasonic wave, the ultrasonic arrival time can be accurately obtained even in a noisy environment or a situation where the pointing device is moving, and the modulated ultrasonic wave is obtained from the speaker 50a, Since it is possible to simultaneously transmit from 50b, it is possible to display the indication point with high accuracy and high speed.

  In the above description, the case of three speakers and two microphones and the case of two speakers and three microphones are shown, but the present invention is not limited to this number. Any aspect is possible as long as the number of speakers and microphones is two or more.

  In addition to the contents described above, the present invention can also be detected as a click operation on a computer system by detecting a certain operation of the pointing device or by moving the pointing device in a certain direction.

  The present invention can also identify a plurality of pointing devices by adding identifiers to the pointing devices.

  In the present invention, the pointing device can be any of a pointing device, a mobile phone, a telephone, a camera, a microphone, a ruler, a pen, glasses, headphones, a ring, a necklace, earrings, earrings, and a motion measuring device. It is.

  In the present invention, the screen can be constituted by any of a screen, a display, a television, a building wall, a window, glass, and a mirror.

1, 2 Pointing system 10 Pointing device 10a, 10b, 10c Microphone (sound wave receiving device)
20 Screen 20a, 20b, 20c Speaker (Sound wave transmission device)
30 PC
30-0 Distance Calculation Unit 30-1 Position Estimation Unit 30-2 Point Point Estimation Unit 40 Projector 50 Pointing Devices 50a, 50b, 50c Speaker (Sound Wave Transmitting Device)
60a, 60b, 60c Microphone (Sound wave receiving device)

Claims (4)

A pointing system for estimating a position on a screen indicated by a pointing device,
A plurality of sound wave transmitting devices that are provided at a plurality of positions different from each other in space and that emit sound waves modulated with different pseudo random signals;
A plurality of sound wave receiving devices that are respectively provided at a plurality of tip portions of the pointing device and receive sound waves;
Correlation processing is performed on the received modulated sound wave received by each sound wave receiving device to obtain the arrival time of the modulated sound wave, and the distance between each sound wave transmitting device and each sound wave receiving device is calculated based on the arrival time A distance calculation unit to perform,
A position estimation unit that estimates the position of each acoustic wave receiving device in space based on the distance between each acoustic wave receiving device and each acoustic wave transmitting device;
A straight line connecting any two points of each coordinate of the estimated sound wave receiving device, or a straight line connecting any one point of each coordinate of the estimated sound wave receiving device and the center of gravity of each coordinate of the estimated sound wave receiving device Estimating a direction vector for specifying the pointing direction of the pointing device based on an angular difference from a known vector indicating the direction of the pointing device, and at any one point on the pointing device or on the pointing device through no desired point, pointing system, wherein the designated point estimating unit, that Ru provided with estimating an intersection point between the screen as the direction vector and a straight line parallel to the plane in which the estimated. A pointing system for estimating a position on a screen indicated by a pointing device,
A plurality of sound wave transmitting devices that are provided at a plurality of tip portions of the pointing device and that emit sound waves modulated by mutually different pseudo-random signals;
A plurality of sound wave receiving devices which are provided at a plurality of different positions in the space and receive sound waves;
Correlation processing is performed on the received modulated sound wave received by each of the sound wave receiving devices to obtain the arrival time of the modulated sound wave, and the distance between each of the sound wave transmitting devices and each of the sound wave receiving devices is calculated based on the arrival time. A distance calculator;
On the basis of the distance between each wave receiving device and the respective wave emitting device, and the position estimation unit for estimating the spatial position of each wave emitting devices,
A straight line connecting any two points of each coordinate of the estimated sound wave transmitting device, or a straight line connecting any one point of each coordinate of the estimated sound wave transmitting device and the center of gravity of each coordinate of the estimated sound wave transmitting device. Estimating a direction vector for specifying the pointing direction of the pointing device based on an angular difference from a known vector indicating the direction of the pointing device, and at any one point on the pointing device or on the pointing device through no desired point, pointing system, wherein the designated point estimating unit, that Ru provided with estimating an intersection point between the screen as the direction vector and a straight line parallel to the plane in which the estimated. The position estimation unit uses a time difference between each sound wave transmission device and each sound wave reception device when time synchronization is not established between each sound wave transmission device and each sound wave reception device. The pointing system according to claim 1 or 2, wherein a position coordinate in a space of each of the sound wave receiving devices is estimated . 4. The pointing system according to claim 1, wherein each of the sound wave transmitting devices transmits the sound wave at least one of a simultaneous time and a different time .

Images