JP2018034221A - Robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To estimate a position of a sound wave generation source existing in an optional position on a three-dimensional space based on a voice signal of the sound wave generation source inputted to a plurality of microphones.SOLUTION: A robot system for estimating a position of a generation source of a sound wave, comprises a robot 20 provided in a three-dimensional space, a plurality of sound wave receivers provided in an optional position of the robot 20 and receiving the sound wave, an angle calculation part 30-1 for calculating an angle formed by a reference position on the robot 20 and the generation source of the sound wave based on an arrival time difference by calculating the arrival time difference of the sound wave respectively received by any two sound wave receivers, a position estimation part 30-2 for estimating a position of the generation source of the sound wave by solving an equation with position information of a calculated angle and the respective sound wave receivers as a parameter and a front face estimation part 30-3 for estimating a direction vector and a vertical surface to the direction vector for indicating the direction of the robot 20 to a position of the generation source of the estimated sound wave.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a robot system having a function of estimating a position of a sound wave generation source.

  Conventionally, there are techniques and systems for estimating the position of an object or a user by signal processing such as acoustic processing and optical processing. For example, in the technique disclosed in Patent Document 1, ultrasonic transmission / reception is performed between the tip of an object and a plurality of positions on a display device, a distance is obtained based on the time required for the transmission / reception, and the tip of the object is detected. Estimate the plane position coordinates.

  In the technique disclosed in Patent Document 2, infrared rays and ultrasonic waves are simultaneously output from an object, the infrared rays are received by a photodiode on a display device, and the ultrasonic waves are received by two microphones on a screen, respectively. . Then, based on the time difference between when the infrared rays are received and when the ultrasonic waves are received, the distance from the microphone to the device is obtained, and the position of the object is estimated.

  Moreover, in the technique disclosed in Patent Document 3, the direction of speech of a seated speaker is estimated based on an audio signal input to a fixed microphone arranged.

JP 2001-125740 A JP 2005-122534 A Japanese Patent No. 05235725

  However, in the technique disclosed in Patent Document 1, since the plane position coordinates on the display device with which the tip of the object (user's mouth) contacts is estimated, the usage scene is close to the display device (robot). It will be limited to. For this reason, the user must approach the robot, and this technique cannot be used when the user is away from the robot.

  In the technique disclosed in Patent Document 2, infrared rays are used for time synchronization in transmission and reception. However, it is necessary to attach an infrared transceiver to an object (user), and the apparatus becomes expensive. . If there is a shield or the like between the user and the light receiving unit, the estimation accuracy tends to deteriorate.

  In the technique disclosed in Patent Document 3, the utterance direction of a seated speaker is estimated based on an audio signal input to a microphone that is fixed in advance, and the microphone is attached to a robot or the like. When the value changes dynamically or when the user is moving, the estimation accuracy tends to deteriorate.

  The present invention has been made in view of such circumstances, and based on the sound signals of the sound wave generation sources input to the plurality of microphones, the position of the sound wave generation source existing at an arbitrary position in the three-dimensional space is determined. An object of the present invention is to provide a robot system that can be estimated.

  (1) In order to achieve the above object, the present invention takes the following measures. That is, the robot system of the present invention is a robot system that estimates the position of a sound wave generation source, and a robot provided in a three-dimensional space and a plurality of sound waves that are provided at arbitrary positions of the robot. And the arrival time difference between the sound waves received by each of the two sound wave reception devices, and the angle between the reference position on the robot and the sound wave generation source is calculated based on the arrival time difference. An angle calculation unit, a position estimation unit for estimating the position of the sound wave generation source by solving an equation using the calculated angle and the positional relationship between the sound wave receiving devices as parameters, and generation of the estimated sound wave A direction vector indicating the orientation of the robot with respect to the position of the source, and a front estimation unit for estimating a plane perpendicular to the direction vector.

  In this manner, the arrival time difference between the sound waves received by any two of the sound wave reception devices provided at arbitrary positions of the robot and receiving the sound waves is calculated, and based on the arrival time difference. The position of the sound wave source is estimated by calculating the angle between the reference position on the robot and the sound wave source, and solving the equation using the calculated angle and the positional relationship of each sound wave receiver as parameters. Since the direction vector indicating the orientation of the robot with respect to the estimated position of the sound wave source and the plane perpendicular to the direction vector are estimated, even if the sound wave source is far from the robot, the position is determined. It is possible to estimate. In addition, since sound waves are used, for example, it is possible to reduce the size and cost compared to the case of using light such as infrared rays, and it is possible to improve convenience. In addition, since sound waves are diffracted more than light, there is an advantage that the sound waves are not easily affected by a shielding object. Furthermore, since the position of the sound wave generation source can be estimated even when the position of the sound wave receiving device is not fixed, the present invention can be applied even when the robot moves. Furthermore, since the arrival time difference between sound waves is used, the position estimation accuracy can be maintained high without increasing the number of sound wave receiving devices even in a noise environment or in a situation where the position of the sound wave generation source moves. it can.

  (2) Further, in the robot system of the present invention, the angle calculation unit selects a plurality of combinations from two or more sound wave receiving devices as a set, and sets each sound wave in each combination. The receiving device calculates the arrival time difference between the received sound waves, calculates the angle between the reference position on the robot and the sound wave generation source, and the position estimation unit is based on the calculated plurality of angles. Then, the position of the sound wave generation source is estimated.

  In this way, among the three or more sound wave receivers, two sound wave receivers are taken as a set, a plurality of combinations are selected, and the arrival time difference between the sound waves received by each sound wave receiver in each combination is calculated. The angle between the reference position on the robot and the sound wave generation source is calculated, and the position of the sound wave generation source is estimated based on the calculated multiple angles. Can be estimated with high accuracy.

  (3) In the robot system of the present invention, the robot includes a speaker that outputs an audio signal, a light source that outputs an optical signal, or a drive unit that drives a main body or a part of the main body, and the estimated surface has On the other hand, it further includes a feedback execution unit that executes at least one feedback operation of outputting an audio signal from the speaker, outputting an optical signal from the light source, or changing the direction of the main body or a part of the main body. To do.

  As described above, since at least one feedback operation is performed on the sound wave generation source, it is possible to give the robot a function of reacting to sound.

  According to the present invention, it is possible to estimate the position of a sound wave generation source even when the sound source is away from the robot. In addition, since sound waves are used, for example, it is possible to reduce the size and cost compared to the case of using light such as infrared rays, and it is possible to improve convenience. In addition, since sound waves are diffracted more than light, there is an advantage that the sound waves are not easily affected by a shielding object. Furthermore, since the position of the sound wave generation source can be estimated even when the position of the sound wave receiving device is not fixed, the present invention can be applied even when the robot moves. Furthermore, since the arrival time difference between sound waves is used, the position estimation accuracy can be maintained high without increasing the number of sound wave receiving devices even in a noise environment or in a situation where the position of the sound wave generation source moves. it can.

It is a figure which shows schematic structure of the robot system which concerns on this embodiment. It is a block diagram which shows the function of the robot system which concerns on this embodiment. It is a flowchart which shows operation | movement of the robot system which concerns on this embodiment.

  The present inventors recalled the realization of a robot that reacts to sound, and attached a plurality of microphones to the robot, and based on the sound signal of the sound wave source input from these microphones, an arbitrary position in the three-dimensional space It is found that the function of reacting to sound can be given to the robot by estimating the position of the sound wave transmission source in the robot and performing a feedback operation on the sound wave transmission source using the estimated position information. It came to make this invention.

  That is, the robot system of the present invention is a robot system that estimates the position of a sound wave generation source, and a robot provided in a three-dimensional space and a plurality of sound waves that are provided at arbitrary positions of the robot. And the arrival time difference between the sound waves received by each of the two sound wave reception devices, and the angle between the reference position on the robot and the sound wave generation source is calculated based on the arrival time difference. An angle calculation unit, a position estimation unit that estimates the position of the sound wave generation source by solving an equation using the calculated angle and position information of each of the sound wave receiving devices as a parameter, and A direction vector indicating the orientation of the robot with respect to the position of the generation source, and a front estimation unit for estimating a plane perpendicular to the direction vector.

  Thus, the present inventors have made it possible to estimate the position even when the sound wave generation source is away from the robot. In addition, by using sound waves, for example, it is possible to reduce the size and cost as compared with the case of using light such as infrared rays, and it is possible to improve convenience. Furthermore, if the positions of a plurality of sound wave receiving devices mounted on the robot are known, the position of the sound wave generation source can be estimated even when the sound wave receiving device moves with the robot. Furthermore, since the difference in arrival time of sound waves is used, it is possible to maintain high position estimation accuracy without increasing the number of sound wave receiving devices even in a noise environment or in a situation where the position of the sound wave generation source moves. It was possible. Embodiments of the present invention will be specifically described below with reference to the drawings.

  FIG. 1 is a diagram illustrating a schematic configuration of a robot system according to the present embodiment. As shown in FIG. 1, the robot system 1 includes a robot 20 and a PC 30 as a position estimation device that estimates the position of a user 10 as a sound wave generation source. More specifically, it includes a robot 20, a PC (Personal Computer) 30, and a plurality of microphones 20a, 20b, and 20c as sound wave receiving devices provided at arbitrary positions of the robot 20. In FIG. 1, three microphones 20 a, 20 b, and 20 c are provided at each end of the robot 20. The microphones 20a, 20b, and 20c receive sound waves transmitted from the user 10 located at an arbitrary position in the three-dimensional space.

  Here, in the robot system according to the present embodiment, a sound wave is used. The sound wave is a concept including any sound range, but preferably “audible sound” in a band of 100 Hz to 2000 Hz which is a human voice. Is used. However, the technical idea of the present invention is not limited to audible sounds.

  In the present embodiment, as an example, three microphones 20a, 20b, and 20c are attached to the robot 20, but the technical idea of the present invention is not limited to three microphones, and more than that. Alternatively, when estimating an arbitrary position on the two-dimensional plane of the user 10, two microphones may be used.

  The PC 30 is built in the robot 20 or is connected to the robot 20 through a wireless interface or a wired interface, and when the sound waves are received by the microphones 20a, 20b, and 20c of the robot 20, data indicating a received wave is received. It is comprised so that it may be transmitted to PC30. The PC 30 performs correlation processing on the received wave to determine the arrival time difference between the sound waves received by each combination of any two microphones, calculates the angle formed by each combination of the microphones and the position of the user 10, and uses The position of the person 10 is estimated.

  For this reason, in this embodiment, even if the timing which the user 10 transmits a sound wave is arbitrary time, an arrival time difference is computable. If the microphones 20a, 20b, and 20c are attached so as not to be on the same straight line, the position coordinates of the user 10 in the three-dimensional space can be obtained, and the robot with respect to the position of the user 10 in the space. It is possible to estimate a direction vector specifying the direction from 20 and a plane perpendicular to the direction vector.

  The PC 30 is a direction vector that specifies the estimated position of the user 10 in the space, or the direction from the robot 20 with respect to the estimated position of the user 10 in the space, or is perpendicular to the estimated direction vector. The surface is transmitted to the robot 20 and feedback to the user 10 is executed.

  FIG. 2 is a block diagram illustrating functions of the robot system 1 according to the present embodiment. The angle calculation unit 30-1 of the PC 30 performs correlation processing on the sound waves received from the user 10 by the microphones 20a, 20b, and 20c of the robot 20, and arrives at the sound waves transmitted by each combination of any two microphones. A time difference is obtained, and an angle formed by each combination of the microphones and the position of the user 10 is calculated.

  In addition, the position estimation unit 30-2 of the PC 30 subordinates and solves each angle calculated by the angle calculation unit 30-1 and an equation reflecting the positional relationship of the user as a constraint condition, so Is uniquely estimated. Further, the front estimation unit 30-3 estimates a direction vector that specifies the direction from the robot 20 with respect to the position in the space of the user 10 estimated by the position estimation unit 30-2. Further, the front estimation unit 30-3 estimates a plane that passes through an arbitrary point on the robot 20 and perpendicularly intersects with a straight line parallel to the direction vector. Other configurations are the same as described with reference to FIG.

  The feedback execution unit 20-1 of the robot 20 includes a speaker that outputs an audio signal, a light source that outputs an optical signal, or a drive unit that drives the main body of the robot 20 or a part of the main body. The drive unit has a function of driving the movable unit of the robot 20 by obtaining power from a motor, an actuator, or the like. The feedback execution unit 20-1 is a direction vector that specifies the position of the user 10 in the space transmitted from the PC 30, or the direction from the robot 20 with respect to the position of the user 10 in the space, or is perpendicular to the direction vector. Based on this aspect, feedback to the user 10 is executed. For example, at least one feedback operation of outputting an audio signal from a speaker, outputting an optical signal from a light source, or changing the direction of the main body of the robot 20 or a part of the main body is performed on the estimated surface.

FIG. 3 is a flowchart showing the operation of the robot system 1 according to this embodiment. First, the user 10 at an unknown position in space intermittently transmits sound waves in a band of 100 Hz to 2000 Hz (step S1). Next, the sound waves transmitted from the user 10 are received by the three microphones 20a, 20b, and 20c attached to the robot 20 (step S2). Here, i = 1, 2, and 3 are set as identifiers of the three microphones 20a, 20b, and 20c, and the waveform of the received sound wave is represented by P _i (x).

ただし、ｔは任意の時刻とする。そして、Ｄ_ｈｉ（τ）を最大とするτを算出し、その値を到達時間差Ａ_ｈｉとする。 Next, the received waveform P _i (x) is transmitted from the robot 20 to the PC 30 (step S3). Next, the PC 30 performs a correlation process on the received waveform P _i (x) transmitted from the robot 20 and arrives until the sound waves transmitted from the user 10 are received by the three microphones 20a, 20b, and 20c. A time difference is calculated (step S4). Here, for each combination (h, i) of any two microphones, the arrival time difference is obtained using the cross-correlation value: D _hi (τ) represented by the equation (1).

However, t is an arbitrary time. Then, τ that maximizes D _hi (τ) is calculated, and this value is defined as an arrival time difference A _hi .

Next, the PC 30 calculates the distance difference r _hi from the user 10 for each combination (h, i) of any two microphones (step S5). Here, assuming that the sound speed is C, r _hi is obtained by the equation (2).

Next, the PC 30 calculates the angle formed by the position of the user 10 for each combination (h, i) of any two microphones (step S6). Here, when the microphones 20a, 20b, and 20c are away from the user 10, it may be considered that the sound waves transmitted from the user reach the microphones 20a, 20b, and 20c as parallel waves. Therefore, if the distance between each combination (h, i) of any two microphones attached to the robot 20 is 2L _hi , the straight line connecting each combination (h, i) of any two microphones and the user 10 The angle φ _hi (j) formed with the position can be obtained by Expression (3).

Next, in the PC 30, the three-dimensional space coordinates of the user 10 are estimated for each combination (h, i) of any two microphones (step S7). Here, the three-dimensional space coordinates of the user 10 can be considered to be on the plane of the angle φ formed with the midpoint of each combination (h, i) of any two microphones. The center of the robot 20 is the origin, and the three-dimensional spatial coordinates of the microphone mounted on the robot 20 are (x _h , y _h , z _h ) and (x _i , y _i , z _i ), respectively (h, i = 1, ..., a n), the three-dimensional coordinates of the user _{10 (u j, v j,} w j), provided that (j = 1, ..., When m), from the cosine theorem, equation (4) Given.

Here, it is assumed that the user 10 exists at w _j > 0. That is, it is assumed that the user 10 exists in front of the robot 20. Equation (4) shows a general case where the number of microphones and users 10 is general, where n is the number of microphones and m is the number of users.

Here, if the number of microphones attached to the robot 20 is three or more, the number of equations in Equation 3 satisfies the unknown and the simultaneous equations can be solved. That is, the three-dimensional spatial coordinates of the microphones 20a, 20b, and 20c attached to the robot 20 are respectively (x _i , y _i , z _i ), where (i = 1, 2, 3), and based on the equation (4). , phi _hi the simultaneous equations erected using (j), by solving subordinate nonlinear simultaneous equations, to uniquely calculate the three-dimensional coordinates of the user 10 _{(u 1, v 1, w} 1) Can do.

Further, when the number of microphones attached to the robot 20 is four or more, in the simultaneous equations shown in the equation (4), the number of equations exceeds the unknown, and a plurality of candidates (u _i , V _i , w _i ) can be obtained, but arbitrarily extracted (w _i , v _i , w _i ) = (α _k , β _k , γ _k ), where (k = 1,..., N) Therefore, the error (δ _k , β _k , γ _k ) in which the error δ shown in the equation (5) is small is adopted as the three-dimensional space coordinates (u _i , v _i , w _i ) of the user 10.

Next, in the PC 30, a direction vector that specifies the direction from the robot 20 with respect to the three-dimensional space coordinates of the user 10 is estimated (step S8). That is, since the position of each microphone provided in the robot 20 is arbitrary, it is necessary to specify the front surface of the user 10 with respect to the robot 20, and thus the direction vector is estimated. Here, since the origin of the center of the robot 20 is the origin, the direction vector from the robot 20 with respect to the three-dimensional space coordinates of the user 10 is given by equation (6).

ここで、（ｘ_ｐ，ｙ_ｐ，ｚ_ｐ）は、利用者１０のロボット２０から向きを得るために算出する方向ベクトルに平行な直線の起点である。例えば、マイク２０ａの装着位置を起点にする場合は、（ｘ_ｐ，ｙ_ｐ，ｚ_ｐ）＝（ｘ_１，ｙ_１，ｚ_１）とすれば良い。なお、ロボット２０から外れた点を（ｘ_ｐ，ｙ_ｐ，ｚ_ｐ）としても、発明の本質は変わらない。 Here, even when the number of users is two or more, if the sound waves received by the microphones 20a, 20b, and 20c do not overlap at an arbitrary time, the direction vector of each user can be obtained. An equation of a straight line that passes through an arbitrary point (x _p , y _p , z _p ) in the space with the center of the robot 20 as the origin and is parallel to the direction vector given by the equation (6) is expressed by the equation (7). Stand like this.

Here, (x _p , y _p , z _p ) is the starting point of a straight line parallel to the direction vector calculated to obtain the direction from the robot 20 of the user 10. For example, when starting from the mounting position of the microphone 20a _{is, (x p, y p,} z p) = (x 1, y 1, z 1) and may be. Note that even if the point deviated from the robot 20 is (x _p , y _p , z _p ), the essence of the invention does not change.

Next, in PC30, the front with respect to the user 10 is estimated. That is, the intersection point with the plane perpendicular to the linear equation given by equation (7) is estimated (step S9). Here, a plane that passes through (x _p , y _p , z _p ) and is perpendicular to equation (7) is obtained by equation (8).

  Finally, a direction vector that specifies the orientation from the robot 20 with respect to the position of the user 10 in space estimated in step S6 or the position of the user 10 in space estimated in step S7, or step S8 A plane perpendicular to the direction vector estimated in (1) is transmitted to the robot 20, and feedback to the user 10 is executed (step S10).

  The feedback operation by the robot 20 is performed by, for example, irradiating light from the robot 20 based on a direction vector that specifies the direction from the robot 20 with respect to the position of the user 10 in space estimated in step S7. The user 10 can be notified of feedback. Further, it is possible to radiate the voice of the robot 20 in a direction that specifies the direction from the robot 20 with respect to the position of the user 10 in the space estimated in step S7, and to notify the user 10 of feedback. Furthermore, if the face (display or the like) of the robot 20 is directed based on a plane perpendicular to the direction vector estimated in step S8, the robot 20 can turn around the user 10.

  Here, when the sound wave is continuously transmitted from the user 10, the process may return to step S2, receive the sound wave again, and estimate the position of the user 10 in the space. Thereby, for example, when the robot 20 performs a feedback operation that turns the face toward the user 10, the robot 20 can follow the movement of the user 10 and turn the face. Even if the movement of the user 10 is high speed, the position in the space of the user 10 can be accurately obtained by giving the maximum threshold value of τ in the equation (1), and the high-speed tracking of the user 10 can be obtained. Can be realized.

In addition, for example, when the microphone 10a, 20b, 20c is an omnidirectional microphone, and another directional microphone is fixed to the robot, the user 10 becomes w _j ≦ 0, that is, the user Even when 10 is on the back side of the robot 20, the position of the user 10 can be estimated.

  As described above, according to the present embodiment, the sound waves transmitted by the user 10 are received by the microphones 20a, 20b, 20c, and the arrival time difference between the sound waves received by each combination of any two microphones is obtained. Calculate the angle between each microphone combination and each microphone based on the obtained arrival time difference, and subtract the equation that reflects the positional relationship of each microphone as the calculated angle and constraint condition. The position of the user 10 in the space is uniquely estimated, the direction vector specifying the orientation of the robot 20 with respect to the estimated position in the space of the user 10 is estimated, and the plane perpendicular to the direction vector is estimated. Even when the person 10 is away from the robot 20, this can be realized. Further, by not using infrared rays, a small, inexpensive, and highly convenient device can be provided, which can be realized even if there is a shielding object. Furthermore, the position of the user can be estimated even with an unfixed microphone, and the microphone can be attached to a robot with dynamic control. In addition, by using the arrival time difference between sound waves, measurement accuracy can be improved even in a noisy environment or a situation where the position of the user is moving, and this can be realized without increasing the number of microphones. .

DESCRIPTION OF SYMBOLS 1 Robot system 10 User 20 Robot 20-1 Feedback execution part 20a, 20b, 20c Microphone 30 PC
30-1 Angle calculation unit 30-2 Position estimation unit 30-3 Front estimation unit

Claims (3)

A robot system for estimating the position of a sound wave source,
A robot provided in a three-dimensional space;
A plurality of sound wave receiving devices which are provided at arbitrary positions of the robot and receive sound waves;
An angle calculation unit that calculates an arrival time difference between the sound waves received by any two sound wave receiving devices, and calculates an angle formed by a reference position on the robot and the sound wave generation source based on the arrival time difference;
A position estimation unit that estimates the position of the sound wave generation source by solving an equation using the calculated angle and the positional relationship between the sound wave receiving devices as parameters;
A robot system comprising: a direction vector indicating a direction of the robot with respect to the estimated position of the sound wave generation source; and a front estimation unit that estimates a plane perpendicular to the direction vector. The angle calculation unit selects a plurality of combinations of two or more sound wave receiving devices as a set, and calculates a difference in arrival time of sound waves received by each sound wave receiving device in each combination. And calculating the angle formed by the reference position on the robot and the sound wave generation source,
The robot system according to claim 1, wherein the position estimation unit estimates a position of the sound wave generation source based on the calculated plurality of angles. The robot includes a speaker that outputs an audio signal, a light source that outputs an optical signal, or a drive unit that drives a main body or a part of the main body,
A feedback execution unit that executes at least one feedback operation of outputting an audio signal from the speaker, outputting an optical signal from the light source, or changing the direction of the main body or a part of the main body on the estimated surface; The robot system according to claim 1, comprising: a robot system according to claim 1.

Images