JP2010044028A - Tactile information detector and tactile information detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a tactile information detector for simply, easily and accurately detecting tactile information indicating a touch when a human strokes a to-be-detected plane. <P>SOLUTION: The tactile information detector includes: a contact element 11 contacting the to-be-detected plane; a frictional sound detecting microphone 12 for detecting a frictional sound generated when the contact element 11 moves while it contacts the to-be-detected plane; an information storage 17 for storing frictional sound information generated when the contact element 11 moves while it contacts at least one type of a to-be-contacted plane, and the tactile information on the to-be-contacted plane so as to be mapped; and an analysis section 20 for calculating a correlation between the frictional sound information detected by the frictional sound detecting microphone 12 and the frictional sound information stored in the information storage 17, reading the tactile information mapped to the frictional information having the correlation higher than a predetermined value from the information storage 17, and outputting it as the tactile information on the to-be-detected plane. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a haptic information detection apparatus and a haptic information detection method for detecting haptic information on a surface to be detected.

  The tactile information indicating the feeling of touch when a human touches an object is very important information indicating the texture of the object. As a technique for obtaining such tactile information, a technique for obtaining information based on pressure information obtained from a piezoelectric element, a strain sensor, etc., or a surface of a contacted surface by applying light to the contacted surface and measuring the reflected light. Techniques for obtaining the state have been proposed.

  For example, in Patent Document 1, an output signal corresponding to the magnitude of pressure sensitivity on an object surface is generated and analyzed by a piezoelectric element that is moved on the body surface, so that it corresponds to the tactile sensation on a human object surface A surface texture measuring device for obtaining the measured results is described. Patent Document 2 describes a distributed pressure detection sensor in which piezoelectric elements are arranged in an array.

JP 2002-181893 A JP-A-2-291901

  However, the pressure information used in the inventions according to Patent Document 1 and Patent Document 2 merely measures the pressure at the contact point with the contacted surface, and accurately detects the hardness of the contacted surface. However, it is very difficult to acquire tactile information such as a touch that humans feel. In addition, when the surface state of the contacted surface is obtained by applying light to the contacted surface and measuring the reflected light, a means for irradiating the light and a means for imaging the reflected light are required, which is obtained from pressure information. There is a problem that the system scale becomes very large compared to such a technique.

  The present invention has been proposed in view of such a situation, and tactile information that can accurately detect tactile information indicating a touch when a human subject strokes the surface to be detected by a simple method. It is an object to provide a detection device and a tactile information detection method.

  As means for solving the above-described problem, the haptic information detection device according to the present invention occurs when the contactor that contacts the detection target surface and the contactor moves in contact with the detection target surface. A frictional sound detection unit that detects a frictional sound with a microphone, frictional sound information generated when the contactor is moved in contact with at least one contacted surface, and tactile information of the contacted surface Is calculated in association with the information storage unit, the frictional sound information detected by the frictional sound detection unit, and the frictional sound information stored in the information storage unit, and the correlation value is greater than a predetermined value. An analysis unit that reads out the haptic information associated with the high frictional sound information from the information storage unit and outputs the haptic information as the haptic information of the detection target surface.

  Further, the tactile information detection method according to the present invention includes a friction sound detection step of detecting, with a microphone, a friction sound generated when a contactor in contact with the detection target surface moves in a state of contact with the detection target surface, and at least one Friction sound information generated when the contactor is moved in a state of contact with the contacted surface of the type and the tactile information of the contacted surface are stored in an associated information storage unit. And calculating the correlation value between the frictional sound information and the frictional sound information detected by the frictional sound detection step, and reading out the tactile information associated with the frictional sound information whose correlation value is higher than a predetermined value from the information storage unit. And an analysis step for outputting as tactile information of the surface to be detected.

  The present invention calculates a correlation value between a frictional sound generated when a contactor in contact with a surface to be detected moves in contact with the surface to be detected and frictional sound information stored in an information storage unit. The tactile information associated with the frictional sound information having the correlation value higher than the predetermined value is read from the information storage unit and output as the tactile information of the detection target surface. By doing so, the present invention uses the correlation characteristic between the frictional sound generated when the surface to be detected and the contact rub against each other and the contact information on the surface to be detected, and thus the surface to be detected is human. It is possible to accurately detect tactile information indicating the touch when the character is stroked by a simple method.

  The tactile information detection device to which the present invention is applied detects tactile information indicating a touch when a person strokes the detection target surface. Such a tactile information detection device is incorporated in a robot 100 as shown in FIG. As shown in FIG. 1B, the robot 100 includes a pressure sensor 101 and a frictional sound detection microphone 102 provided in an array on the arm part 100a, and a frictional sound detection microphone on a tip part 100b of the arm part 100a. 103 is provided. For example, the robot 100 touches an arbitrary detection target object in accordance with a user's instruction, and detects and analyzes the friction sound generated when the detection target microphone 102 and 103 detects the friction sound. Can be notified to the user. In addition, as shown in FIG. 2A, the robot 100 is provided with contacts 105a to 105c in which a plurality of microphones for detecting frictional sound are built in the fingertip portion of the robot hand 105. Further, in the tactile information detection device to which the present invention is applied, for example, a contact having a built-in microphone for detecting a frictional sound is attached to a human fingertip as shown in FIG. By detecting and analyzing the frictional sound generated when the target object is rubbed with the frictional sound detection microphone, it is possible to detect tactile information of the target object.

  A specific configuration of such a tactile information detection device will be described below with reference to a mode for carrying out the present invention using a detection device 1 as shown in FIG.

  As shown in FIG. 3, the detection device 1 includes a contact 11 that contacts the surface to be detected, and a friction sound detection that collects a friction sound that is generated when the contact 11 moves in contact with the surface to be detected. And a noise detecting microphone 13 for detecting a noise component included in the frictional sound. The detection device 1 also includes a pressure sensor 14 that detects the pressure at the contact point between the surface to be detected and the contact 11. The detection apparatus 1 includes an amplifying unit 15 that amplifies detection signals output from the frictional sound detecting microphone 12, the noise detecting microphone 13, and the pressure sensor 14, and an analog for the detection signal amplified by the amplifying unit 15. A / D conversion unit 16 for converting into detection data by digital conversion processing. The detection device 1 also includes an information storage unit 17 that stores frictional sound information and tactile information in association with each other, a voice input unit 18 that receives voice information from a user, and a motor drive that moves the contactor 11. Part 19. In addition, the detection apparatus 1 analyzes the detection data output from the AD conversion unit 16 and outputs the tactile information on the detection target surface, and displays the output result from the analysis unit 20 on a display or speaker. And a notification unit 21 for notifying the user.

  For example, as shown in FIG. 3, the contactor 11 includes a first layer 11 a having a high hardness such as an aluminum foil that allows a soft member such as sponge or silicon to efficiently transmit the force of the power point, and the first layer 11 a. And a second layer 11b such as a cloth that has unevenness on the surface and is susceptible to vibration due to friction. The contactor 11 is particularly configured as described above, so that it is possible to increase the frictional sound generated when it rubs against the surface to be detected. However, it generates the frictional sound when it rubs against the surface to be detected. If it is a member to be made, you may make it use what is comprised from another member.

  The contact 11 is connected to a motor driven by a motor drive unit 19 described later, and is moved while being pressed against the detection target surface by the driving force of the motor. Note that the contact 11 is not limited to being moved by the driving force of the motor by the motor driving unit 19, and may be moved while being pressed against the detection target surface by the user.

  The frictional sound detection microphone 12 functions as a frictional sound detection unit that detects a frictional sound, and is provided inside the contactor 11 so as to be as close as possible to the frictional sound source, for example, as shown in FIG. The frictional sound detecting microphone 12 collects a frictional sound generated when the surface to be detected and the contact 11 rub against each other, and supplies the collected acoustic signal to the amplifying unit 15.

  Note that the frictional sound detecting microphones 12 are not limited to being provided inside the contactor 11, and a plurality of microphones may be provided around the contactor 11.

  The noise detection microphone 13 collects the friction sound generated by the contact between the surface to be detected and the contactor 11 in the same manner as the above-described friction sound detection microphone 12 in order to detect a noise component included in the friction sound. The collected acoustic signal is supplied to the amplifying unit 15. That is, the noise detecting microphone 13 functions as a noise generating unit that generates noise information included in the frictional sound. The acoustic signal collected by the noise detection microphone 13 is used for noise reduction processing by the noise reduction unit 202 described later.

  For example, as shown in FIG. 3, the pressure sensor 14 is provided on the contact surface of the contact 11 in order to detect the pressure at the contact point between the detection target surface and the contact 11. The pressure sensor 14 includes a piezoelectric element, a strain sensor, and the like, and supplies a detection signal obtained by converting pressure into an electric signal to the amplification unit 15.

  The amplification unit 15 amplifies the detection signals output from the frictional sound detection microphone 12, the noise detection microphone 13, and the pressure sensor 14, and supplies the amplified detection signal to the AD conversion unit 16.

  The AD conversion unit 16 performs analog / digital conversion processing on each detection signal amplified by the amplification unit 15 and supplies detection data corresponding to each detection signal to the analysis unit 20.

  The information storage unit 17 associates the friction sound information generated when the contactor 11 is moved with at least one kind of contacted surface in contact with the tactile information of the contacted surface. It is remembered. Here, the frictional sound information may be time waveform information of the acoustic signal of the frictional sound or may be spectral information obtained by frequency-converting the acoustic signal, but in the present embodiment, the information storage unit 17 uses the spectral information as the frictional sound information. Is remembered. The tactile information is perceptual information indicating a touch when a person strokes the detection target surface, and specifically represents information on the surface state of the object such as “smooth” or “rough”. Information stored in the information storage unit 17 is accessed by the analysis unit 20.

  The voice input unit 18 receives a phrase representing the surface state of an object such as “smooth” or “rough” by the user as a voice signal, and supplies the input voice signal to the analysis unit 20.

  The motor drive unit 19 functions as a contact moving unit that moves the contact 11 in contact with the surface to be detected. The motor drive unit 19 is controlled by the analysis unit 20 as will be described later, and the contact 11 is placed on the surface to be detected. Move in the pressed state. Further, the motor drive unit 19 is a noise source for frictional noise because sound is generated by the rotation of the motor when the contactor 11 is moved. Therefore, the motor drive unit 19 supplies information related to the motor indicating when the motor is driven with a certain degree of torque to the analysis unit 20 as noise information. That is, the motor drive unit 19 functions as a noise generation unit that generates noise information included in the frictional sound.

  The analysis unit 20 is a computer including a general-purpose arithmetic processing processor for analyzing the detection data output from the AD conversion unit 16 and outputting the tactile information on the detection target surface with high accuracy using the analysis result. It has the following configuration.

  That is, the analysis unit 20 analyzes the detection data, reads the tactile information of the detection target surface from the information storage unit 17 and outputs the tactile information, and is included in the friction sound detected by the friction sound detection microphone 12. The noise reduction part 202 which reduces a noise component and the sound source position measurement part 203 which measures the sound source position where a frictional sound is generated in a to-be-detected target surface are provided. The analysis unit 20 also analyzes the friction sound information detected by the friction sound detection microphone 12 and recognizes the correspondence between the information learning unit 204 newly stored in the information storage unit 17 and the tactile information and the friction sound information. A recognition unit 205. Further, the analysis unit 20 includes a hardness information measurement unit 206 that measures the hardness of the surface to be detected from detection data detected by the pressure sensor 14, and a motor drive unit 19 based on the detection data detected by the pressure sensor 14. And a motor control unit 207 for controlling.

  The tactile information processing unit 201 calculates a correlation value between the frictional sound information detected by the frictional sound detection microphone 12 and the frictional sound information stored in the information storage unit 17, and the frictional sound information whose correlation value is higher than a predetermined value. Is read from the information storage unit 17 and output to the notification unit 21 as tactile information of the detection target surface. Specifically, the haptic information processing unit 201 performs spectrum conversion processing such as discrete Fourier transform (FFT) on detection data detected by the frictional sound detection microphone 12 to acquire spectrum information, and acquires the acquired spectrum information and By detecting the spectrum information that is the frictional sound information stored in the information storage unit 17 and the correlation value, the highly correlated tactile information is output to the notification unit 21 as tactile information of the detection target surface.

  The noise reduction unit 202 reduces a noise component from the friction sound detected by the friction sound detection microphone 12 based on the detection data detected by the noise detection microphone 13 and the noise information supplied from the motor drive unit 19.

  As will be described later, the sound source position measuring unit 203 is a sound source that generates frictional sound on the detection target surface based on a plurality of frictional sounds detected by these microphones when a plurality of frictional sound detecting microphones are arranged. Measure the position. Then, the sound source position measurement unit 203 outputs the measured sound source position information to the notification unit 21 in association with the tactile information output by the tactile information processing unit 201.

  The information learning unit 204 causes the information storage unit 17 to newly store the frictional sound information detected by the frictional sound detection microphone 12 in association with the tactile information analyzed and output by the tactile information processing unit 201. In addition, the information learning unit 204 causes the information storage unit 17 to store the voice information input from the voice input unit 18 as tactile information.

  The meaning recognizing unit 205 recognizes the correspondence between the audio information input from the audio input unit 18 and the tactile information based on the information stored in the information storage unit 17 and outputs a recognition result.

  The hardness information measuring unit 206 measures the hardness of the surface to be detected from the pressure information detected by the pressure sensor 14, and associates the measured hardness information with the tactile information output by the tactile information processing unit 201. To 21.

  The motor control unit 207 places the contactor 11 on the surface to be detected so that the volume of the frictional sound detected by the microphone 12 for detecting the frictional sound that changes according to the pressure detected by the pressure sensor 14 is equal to or higher than a predetermined threshold. The motor drive unit 19 is moved in the contacted state.

  In the analysis unit 20 configured as described above, the tactile information processing unit 201 generates frictional sound generated when the contactor 11 in contact with the detection target surface moves in a state of being in contact with the detection target surface, and information storage. A correlation value with the frictional sound information stored in the unit 17 is calculated, and the tactile information associated with the frictional sound information whose correlation value is higher than a predetermined value is read from the information storage unit 17 and is detected on the surface to be detected. Output as tactile information. By doing in this way, the analysis unit 20 uses the correlation characteristic between the frictional sound generated when the target surface to be detected and the contact 11 rub against each other and the contact information on the target surface to be detected. It is possible to accurately detect tactile information indicating the touch when a human is stroking with a simple method.

  Next, a specific method of noise reduction processing by the analysis unit 20 will be described with reference to the processing flow shown in FIG.

  Detection data detected by the noise detection microphone 13 is input to the analysis unit 20 (step ST11a), and noise information corresponding to the driving state of the motor is input by the motor driving unit 19 (step ST11b). The analysis unit 20 receives detection data detected by the frictional sound detection microphone 12 (step ST11c) and detection data detected by the pressure sensor 14 (step ST11d).

  When the data as described above is input, the noise reduction unit 202 performs frequency conversion processing on the detection data input in step ST11a to calculate external noise spectrum information (step ST12). Further, the noise reduction unit 202, based on the spectrum information calculated in step ST12 and the noise information input in step ST11b, a spectrum corresponding to the noise component included in the friction sound detected by the friction sound detection microphone 12. Information is estimated (step ST13). Then, the noise reduction unit 202 supplies the estimated spectrum information to the tactile information processing unit 201.

  Further, the haptic information processing unit 201 performs frequency conversion processing on the detection data input in step ST11c to calculate the spectrum information of the frictional sound (step ST14). Then, the tactile information processing unit 201 subtracts the spectrum information corresponding to the noise component calculated in step ST13 from the spectrum information calculated in step ST14, thereby acquiring the spectrum information of the frictional sound in which the noise component is reduced. (Step ST15).

  Further, in step ST16, the hardness information measuring unit 206 determines whether or not the contact 11 is in contact with the detection target surface from the pressure value based on the detection data from the pressure sensor 14, and only when it is determined that the contact is made. The tactile information processing unit 201 and the noise reduction unit 202 are notified and controlled so as to perform the processes according to steps ST12 to ST15 described above.

  In the analysis unit 20, the tactile information processing unit 201 detects the correlation information by detecting the spectrum information obtained in step ST15, the spectrum information that is the frictional sound information stored in the information storage unit 17, and the correlation value. High tactile information is output to the notification unit 21 as tactile information of the detection target surface.

  In this manner, in the analysis unit 20, the haptic information processing unit 201 calculates a correlation value between the friction sound information whose noise component has been reduced by the noise reduction unit 202 and the friction sound information stored in the information storage unit 17. Then, the tactile information associated with the friction sound information whose correlation value is higher than a predetermined value is read from the information storage unit 17 and output as tactile information of the detection target surface. In this way, the analysis unit 20 considers the noise component included in the frictional sound detected by the frictional sound detecting microphone 12 and outputs the tactile information without being affected by the operation sound of the motor or the surrounding environmental noise. It can be detected with higher accuracy.

  Next, a specific method of the frictional sound source position measurement process by the analysis unit 20 will be described. First, when measuring the frictional sound generation position X, for example, as shown in FIG. 5, the detection device 1 includes four frictional sound detection microphones 12 a to 12 d arranged in an array as a frictional sound detection unit.

  The analysis unit 20 measures the position of the sound source where the frictional sound is generated by calculating the phase difference of the detection signals between these frictional sound detection microphones. Specifically, the sound source position measurement processing using the detection signals detected from the three friction sound detection microphones will be described with reference to the processing flow shown in FIG.

  First, in step ST21, detection data detected by the three frictional sound detection microphones is input to the analysis unit 20.

  In step ST22, the sound source position measurement unit 203 performs frequency conversion processing such as FFT on the three detection data input in step ST21 to calculate the spectrum information of the frictional sound.

  In step ST23, the sound source position measuring unit 203 performs cross-correlation calculation processing on each piece of spectrum information calculated in step ST22 to calculate cross power spectrum information.

  In step ST24, the sound source position measurement unit 203 performs processing such as IFFT on each cross power spectrum information calculated in step ST23, and calculates waveform information indicating a time response obtained by inversely converting the frequency response.

  In step ST25, the sound source position measuring unit 203 searches for the maximum amplitude value of each waveform information calculated in step ST24. Here, when the positions of the respective frictional sound detection microphones with respect to the sound source position are different from each other, a phase difference is generated in the waveform information, and the maximum amplitude value of the waveform information is generated at different times.

  Then, the sound source position measurement unit 203 refers to a table indicating the sound source position associated with the combination of maximum amplitude values stored in advance in the internal memory (step ST26), and determines each maximum amplitude value searched in step ST25. A sound source position is estimated from the combination (step ST27).

  As described above, the sound source position measuring unit 203 can measure the sound source position at which the friction sound is generated on the detection target surface based on the plurality of friction sounds detected by the plurality of friction sound detection microphones. In particular, when performing a sound source position detection process by arranging a large number of pressure elements in an array, the pressure elements must be disposed over the entire detection target surface. In the detection apparatus 1, the number of microphones related to the sound source position detection process can be reduced. The analysis unit 20 outputs the sound source position information to the notification unit 21 in association with the contact information output by the tactile information processing unit 201. Further, the analysis unit 20 can improve the accuracy of the tactile information detection process using the sound source position information measured by the sound source position measurement unit 203. For example, in the analysis unit 20, the haptic information processing unit 201 uses only the friction sound information having a relatively good S / N ratio obtained from the microphone for detecting the friction sound closest to the sound source position measured by the sound source position measurement unit 203. By detecting information, it is possible to improve the accuracy related to the tactile information detection process. Further, the analysis unit 20 can detect the moving speed and direction in which the contact 11 moves on the detection target surface by causing the sound source position measurement unit 203 to measure the sound source position at predetermined time intervals. Here, if the moving speed at which the contact 11 moves on the detection target surface is too fast or too slow, the frequency component of the frictional sound also changes. Therefore, in order to maintain the accuracy related to the tactile information detection process, It is desirable that the contact moves within a predetermined range. In other words, it is desirable that the force for pressing the contact 11 against the surface to be detected is kept uniform. Accordingly, the analysis unit 20 causes the tactile information processing unit 201 to perform the tactile information detection process only when the detected moving speed is within a predetermined range, thereby improving the accuracy associated with the tactile information detection process. Can be maintained.

  Next, the movement control process of the contactor 11 according to the detection information from the pressure sensor by the analysis unit 20 will be described with reference to the process flow shown in FIG.

  Detection data detected by the noise detection microphone 13 is input to the analysis unit 20 (step ST31a), and noise information corresponding to the driving state of the motor is input by the motor driving unit 19 (step ST31b). The analysis unit 20 receives detection data detected by the frictional sound detection microphone 12 (step ST31c), and receives detection data detected by the pressure sensor 14 (step ST31d).

  When the data as described above is input, the noise reduction unit 202 performs frequency conversion processing on the detection data input in step ST31a to calculate spectrum information of external noise (step ST32). Further, the noise reduction unit 202, based on the spectrum information calculated in step ST32 and the noise information input in step ST31b, the spectrum corresponding to the noise component included in the friction sound detected by the friction sound detection microphone 12. Estimate information. Then, the noise reduction unit 202 supplies the estimated spectrum information to the tactile information processing unit 201.

  Further, the haptic information processing unit 201 performs frequency conversion processing on the detection data input in step ST31c to calculate spectrum information of the frictional sound (step ST34). Then, the tactile information processing unit 201 subtracts the spectrum information corresponding to the noise component calculated in step ST33 from the spectrum information calculated in step ST34, thereby acquiring the spectrum information of the frictional sound in which the noise component is reduced. (Step ST35).

  The hardness information measuring unit 206 determines whether or not the contact 11 is in contact with the surface to be detected from the pressure value based on the detection data from the pressure sensor 14, and only when the contact is determined to be in contact, the above-described steps ST32 to ST32- The tactile information processing unit 201 and the noise reduction unit 202 are notified and controlled to perform the processing according to step ST35 (step ST36).

  The motor control unit 207 calculates the sum of the spectrum of the spectrum information of the frictional sound acquired in step ST35, that is, the power spectrum (step ST37). Then, motor control unit 207 determines whether or not the power spectrum calculated in step ST37 is larger than a predetermined threshold (step ST38). That is, the motor control unit 207 determines whether the volume of the frictional sound detected by the frictional sound detection microphone 12 is equal to or greater than a predetermined threshold value. From this determination result, the motor control unit 207 controls the motor drive unit 19 to reduce the torque of the motor that generates a force that presses the contact 11 against the detection target surface when it is greater than the predetermined threshold. Further, the motor control unit 207 controls the motor drive unit 19 to increase the torque of the motor that generates a force for pressing the contact 11 against the detection target surface when the motor control unit 19 is smaller than the predetermined threshold value.

  As described above, the motor control unit 207 covers the contact 11 so that the volume of the frictional sound detected by the frictional sound detection microphone that changes according to the pressure detected by the pressure sensor 14 becomes a predetermined value. By controlling to move by the motor drive unit 19 in contact with the detection target surface, a uniform friction sound can be obtained, and as a result, detection accuracy of tactile information can be improved. That is, the motor control unit 207 improves the uniformity of the frictional sound detected by the frictional sound detection microphone 12 by keeping the strength with which the contactor 11 is pressed against the surface to be detected uniform. As a result, the tactile information Detection accuracy can be increased.

  The reason why the detection accuracy can be improved in this way is that the obtained frictional sound changes somewhat depending on how much force the contactor 11 is brought into contact with the detection target surface when acquiring the frictional sound information.

  Next, vocabulary meaning acquisition processing relating to tactile information will be described with reference to the processing flow shown in FIG.

  First, a plurality of pieces of friction sound information are acquired as learning data by causing the contactor 11 to touch various objects (step ST41). The information learning unit 204 classifies the learning data acquired in step ST41 into classes indicating a plurality of tactile stimuli using a learning data self-organizing clustering method such as SOM and Bayes learning (step ST42). Note that the information learning unit 204 sets frictional sound information indicating a tactile stimulus that cannot be classified into these classes as new tactile stimulus data (step ST43). Then, as shown in FIG. 9, the contactor 11 makes a class determination on the learning data acquired in step ST41 (step ST44).

  Moreover, the information learning part 204 acquires the audio | voice data input by the audio | voice input part 18 matched with each learning data (step ST45). The information learning unit 204 performs speech recognition processing (step ST46) and morpheme analysis processing (step ST47) on the speech data acquired in step ST45, and extracts haptic vocabulary associated with each learning data ( Step ST48).

  Then, the information learning unit 204 associates each class determined in step ST44 with the vocabulary extracted in step ST48 by associating the haptic class with the vocabulary by the associative memory mechanism using the information storage unit 17 (step ST49). . In this way, the information storage unit 17 uses the information obtained by connecting the tactile class and the vocabulary as tactile information, and stores a plurality of pieces of friction sound information in association with each tactile information.

  The meaning recognition unit 205 performs meaning recognition using information in which the tactile class and the vocabulary are associated in step ST49. For example, the meaning recognizing unit 205 inputs friction sound information corresponding to a new tactile stimulus detected using the contact 11 (step ST51), and receives the tactile information corresponding to the input friction sound information from the information storage unit 17. By referring to and understanding (step ST53), for example, a vocabulary indicating tactile information understood by a speaker (not shown) is pronounced. In addition, the semantic recognition unit 205 refers to and understands the haptic information corresponding to the vocabulary (step ST52) input by the voice input unit from the information storage unit 17 (step ST53), and estimates an object indicating the haptic information. By doing so, it is possible to understand tactile information related to the hand from human speech.

  In this way, the semantic recognition unit 205 incorporates the detection apparatus 1 to calculate which words and phrases can be expressed by calculating how much correlation the newly input tactile information has to which class. The robot can judge and speak.

  Thus, since the detection apparatus 1 can detect tactile information with high accuracy, for example, by using such a detection result, a correspondence relationship with a parameter of the tactile display is calculated in advance, and the touch-related information is detected. It becomes possible to reproduce tactile information.

  As described above, the detection apparatus 1 to which the present invention is applied can quantitatively evaluate tactile information that has been difficult in the past, in particular, sensory information such as touch, so the following is possible. .

  First, the detection apparatus 1 can be applied as a new human interface device using a tactile sense. That is, since the detection apparatus 1 can detect a fine touch feeling, this detection result is used as a switch, or the user's favorite touch is learned and used as a substitute for a personal identification number. Can be used as a device.

  Secondly, the detection device 1 can be applied as a device that performs object recognition processing. That is, in the detection apparatus 1, the touch of the object can be used as a clue for recognizing the object, and the accuracy of object recognition by the machine can be improved.

  Thirdly, the detection device 1 can be applied as a device that performs tactile sense and language understanding by a computer or a robot. That is, in order for the robot to understand the essential meanings such as tactile sensation and touch, and related vocabulary and words, it is necessary to detect the physical information. The process can improve human language communication.

  Fourthly, the detection device 1 can be applied as a device for researching and developing human senses. That is, since the detection apparatus 1 can detect tactile information with high accuracy, based on the detection result, the detection apparatus 1 performs a quantitative evaluation on a human tactile sense (for example, a favorite touch or an unpleasant touch). Objective objective indicators can be obtained.

  Fifth, the detection apparatus 1 can be applied as an apparatus that performs quantitative analysis of tactile information used for augmented reality (AR) and virtual reality (VR). That is, the purpose of AR and VR is to simulate a human sense. For this purpose, it is necessary to quantitatively record stimuli for generating human senses. In response to such a problem, the detection device 1 can detect tactile information with high accuracy, and based on this detection result, obtain numerical data for artificially reproducing a human sense. Any touch, that is, tactile information can be reproduced by calibration with a device that performs tactile reproduction.

  Sixth, the detection apparatus 1 can be applied to a mobile phone. That is, the detection device 1 can easily communicate (transmit) tactile information by using a microphone of a mobile phone as the microphone 12 for detecting frictional sound. The tactile information transmitted in this way can be used for various purposes such as ordering products and simple diagnosis of skin and the like.

  As described above, since the detection apparatus 1 to which the present invention is applied can be realized with a relatively simple apparatus configuration, it can be applied to the first to sixth uses described above at low cost.

  Next, a specific example in which the tactile information detection device is specifically configured, learning and identification of tactile information indicating a touch feeling is performed, and tactile information is detected using a specific object will be described.

  First, the contact 50 configured to satisfy the following conditions was used. In order to increase the frictional noise generated when the contact 50 is brought into contact with the surface to be detected and rubs against the surface to be detected, for example, the following covering process is performed. In other words, the contact 50 is covered with an aluminum foil 51 that efficiently transmits the force at the power point, and further coated with a cloth 52 that has irregularities on the outer surface of the aluminum foil 51 and is susceptible to vibration due to friction.

  Further, the contact 50 is provided with a pressure sensor 54 in order to take into account the difference in pressure generated when the object surface is rubbed.

  In the following embodiments, the contact 50 incorporating the frictional sound detecting microphone 53 and the pressure sensor 54 is configured in consideration of these points. FIG. 10 shows an example of mounting on a human finger, but it may be mounted on a robot.

  Eight kinds of objects were used as learning data for tactile information indicating the touch, and five kinds of objects were used for identification. Table 1 below shows a list of objects used.

  Next, an experiment for learning and identifying tactile information will be described with reference to the processing flow shown in FIG.

  The condition of the rubbing operation is based on the value when the human confirms the touch, and the pressure is about 30 (the strength that can smoothly rub the object surface) with the output value of the pressure sensor 54, and the rubbing speed is about 20 cm / s. The reciprocating range was about 4 cm (step ST61).

  Under this condition, the object surface was rubbed with the contact 50, and the generated sound was recorded. Specifically, the contact 50 was held by an experimenter's hand, and the object surface was rubbed while making the output value of the pressure sensor 54 constant. The sampling rate was 16 kHz, the recording time was 4 seconds, and the part to be measured was rubbed back and forth (step ST62).

  In step ST63, as shown in FIG. 12A, the frictional sound data for learning was divided into frames and converted into 13-dimensional MFCC (feature vector) data. This data was then modeled with a multidimensional mixed normal distribution. The EM algorithm was used for this modeling.

  Similarly, as shown in FIG. 12B, the frictional sound data for identification was divided into frames and converted into 13-dimensional MFCC data.

  In step ST64, the likelihood of the MFCC for identification was calculated using the learned mixed normal distribution. However, at this time, the average likelihood was calculated by averaging the likelihood of all frames. The average likelihood of all the learning objects in the identification data was calculated, and the object having the maximum likelihood (or the word corresponding to the touch sensation associated with the object, that is, the tactile sensation) was used as the recognition result. For example, in the example of the object used this time, a correct answer will be obtained if “Kurae no Plush” is recognized as a handkerchief (or “towel”).

  The mixed normal distribution used for likelihood calculation is expressed by the following equation (1).

However, x, μ _k , and Σ _k are an input data vector, an average vector for the kth learning object, and a covariance matrix for the kth learning object, respectively.

  Under the above recording conditions, a total of 13 types of frictional sound data, 8 types for learning and 5 types for identification, were recorded. Eight types of learning were learned as learning data, and five types of touch data for identification were identified. In order to confirm the accuracy, 8 kinds of learning data were also identified.

  Table 8 below shows the results when eight types of learning objects are stored and five types of identification objects are identified as described above. As shown in Table 2, the same recognition result as the correct answer given by humans was obtained for all objects. Also, when 8 types of learning data were input, all were recognized correctly.

  Thus, in the detection device 1 using the specific contact 50, correct answers can be obtained from all the identification results, and the tactile information can be detected with high accuracy using the correlation characteristics between the frictional sound and the tactile information. Can do. Further, from this embodiment, it is possible to confirm that the touch feeling, that is, the words related to the tactile information can be understood.

  In addition, this invention is not limited only to the above embodiment, Of course, a various change is possible in the range which does not deviate from the summary of this invention.

It is a figure which shows the example of application of the tactile information detection apparatus which concerns on this invention. It is a figure which shows the example of application of the tactile information detection apparatus which concerns on this invention. It is a block diagram which shows the specific structure of the detection apparatus to which this invention was applied. It is a figure which shows the processing flow with which it uses for description of a noise reduction process. It is a figure where it uses for description of a sound source position estimation process. It is a figure which shows the process flow with which it uses for description of a sound source position estimation process. It is a figure which shows the process flow with which it uses for description of the movement control process of the contactor according to the detection information from a pressure sensor. It is a figure which shows the processing flow with which it uses for description of the specific process which concerns on an information learning part and a meaning recognition part. It is a figure which shows the friction sound information classified according to tactile information. It is a figure which shows the specific structure of a contactor. It is a figure which shows the process flow with which it uses for description of the Example of the detection process of contact information. It is a figure where it uses for description about the flow of the concrete signal processing performed in the Example of the detection process of contact information.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Detection apparatus 11, 105a-105c, 200, 50, Contact, 11a 1st layer, 11b 2nd layer, 12, 12a-12d, 102, 103, 53 Friction sound detection microphone, 13 Noise detection microphone , 14, 54, 101 Pressure sensor, 15 Amplification unit, 16 AD conversion unit, 17 Information storage unit, 18 Voice input unit, 19 Motor drive unit, 20 Analysis unit, 201 Tactile information processing unit, 202 Noise reduction unit, 203 Sound source Position measurement unit, 204 Information learning unit, 205 Semantic recognition unit, 206 Hardness information measurement unit, 207 Motor control unit, 21 Notification unit, 51 Aluminum foil, 52 Cloth, 100 Robot, 100a Arm unit, 100b Tip unit, 105 Robot hand

Claims (7)

A contact that contacts the surface to be detected;
A friction sound detection unit that detects a friction sound generated by the microphone when the contactor moves in a state of contact with the detection target surface;
Information storage in which frictional sound information generated when the contact is moved in contact with at least one kind of contacted surface and the tactile information of the contacted surface are stored in association with each other And
A correlation value between the friction sound information detected by the friction sound detection unit and the friction sound information stored in the information storage unit is calculated, and tactile information associated with the friction sound information whose correlation value is higher than a predetermined value is obtained. And a tactile information detection device comprising: an analysis unit that reads out from the information storage unit and outputs the tactile information of the detection target surface. A noise generating unit that generates noise information included in the frictional sound detected by the frictional sound detecting unit;
A noise reduction unit for reducing a noise component from the frictional sound detected by the frictional sound detection unit based on the noise information generated by the noise generation unit;
The analysis unit calculates a correlation value between the friction sound information whose noise component is reduced by the noise reduction unit and the friction sound information stored in the information storage unit, and the friction value is higher than a predetermined value. The haptic information detection apparatus according to claim 1, wherein the haptic information associated with the information is read from the information storage unit and output as the haptic information of the detection target surface. The frictional sound detection unit detects a frictional sound generated when the contactor moves in contact with the surface to be detected by a plurality of microphones,
The analysis unit measures a sound source position where a frictional sound is generated on the detection target surface based on a plurality of frictional sounds detected by a plurality of microphones of the frictional sound detection unit, and detects the sound source position information on the detected target The tactile information detection device according to claim 1, further comprising a sound source position measurement unit that outputs the sound source position in association with the tactile information of the target surface.   The tactile information detection according to claim 1, further comprising: an information learning unit that newly stores the friction sound information detected by the friction sound detection unit in the information storage unit in association with the tactile information analyzed and output by the analysis unit. apparatus. A pressure detecting element for detecting a pressure at a contact point between the surface to be detected and the contact;
The analysis unit measures the hardness of the detection target surface from the pressure information detected by the pressure detection element, and outputs the hardness information in association with the tactile information of the detection target surface. The tactile information detection device according to claim 1, comprising: A pressure detecting element for detecting a pressure at a contact point between the surface to be detected and the contact;
A contact moving unit for moving the contact in a state where it is in contact with the surface to be detected;
The contact in a state where the contact is in contact with the surface to be detected so that the volume of the frictional sound detected by the frictional sound detection unit that changes according to the pressure detected by the pressure detection element becomes a predetermined value. The tactile information detection device according to claim 1, further comprising a movement control unit that is moved by the child moving unit. A friction sound detecting step of detecting, with a microphone, a friction sound generated when the contact that is in contact with the detection target surface moves in a state of being in contact with the detection target surface;
Information storage in which frictional sound information generated when the contact is moved in a state of contact with at least one kind of contacted surface and the tactile information of the contacted surface are stored in association with each other Calculating the correlation value between the frictional sound information stored in the unit and the frictional sound information detected in the frictional sound detection step, and determining the tactile information associated with the frictional sound information whose correlation value is higher than a predetermined value as the information A tactile information detection method comprising: an analysis step of reading from the storage unit and outputting the tactile information of the detection target surface.

Images