KR102039002B1 - Device and Method for detecting tactile sense - Google Patents

Abstract

As a preferred embodiment of the present invention, the tactile detection device scans the surface of a medium with a multi-array tactile sensor and receives the position coordinates of n * m tactile sensors and n * m tactile input signals corresponding thereto in real time to provide a tactile input. When the value of the signal exceeds the predetermined value, it is determined that the tactile input exists, and the tactile input is classified and output according to the threshold value for each previously learned medium.

Description

Device and method for detecting tactile sense

The present invention relates to a tactile detection device. More particularly, the present invention relates to a tactile detection device for detecting at least one medium by sensing a surface of the medium by a plurality of methods.

Haptics are tactile and force feedback techniques that utilize the user's touch feeling by applying haptic feedback effects (ie, "haptic effects") to the user, such as forces, vibrations, and motions. Devices such as mobile devices, touchscreen devices and personal computers can be configured to generate haptic effects. In general, calls to embedded hardware (eg, actuators) that can generate haptic effects may be programmed into the device's operating system (OS). These calls specify which haptic effect to play. For example, when a user interacts with the device using, for example, a button, touchscreen, lever, joystick, wheel, or some other control, the OS of the device may send a playback command to the embedded hardware through the control circuitry. Can be. The embedded hardware then creates an appropriate haptic effect that is recognized by the user.

Portable / mobile electronic devices such as mobile phones, smartphones, camera phones, cameras, personal digital assistants ("PDAs") typically output an output to alert the user of certain events occurring on the devices. Mechanisms. For example, a cell phone typically includes a speaker for notifying the user of an incoming phone call event. In some mobile devices, kinesthetic feedback (eg, active force and resistive feedback) and / or tactile feedback (eg, vibration, texture, and heat) may also be provided to the user.

KR 10-1791927 KR 10-2017-0098681

1. Zheng, Haitian, et al. "Deep learning for surface material classification using haptic and visual information." IEEE Transactions on Multimedia 18.12 (2016): 2407-2416. 2. Fishel, Jeremy A., and Gerald E. Loeb. "Bayesian exploration for intelligent identification of textures." Frontiers in neurorobotics 6 (2012). 3. Strese, Matti, et al. "Multimodal feature-based surface material classification." IEEE transactions on haptics (2017).

Conventionally, a method of distinguishing the touch input at a predetermined time using a haptic sensor or an image sensor is disclosed, but there is a problem in that there is no detection method for the touch sensed at an arbitrary time. In addition, in the case of the existing technology, the tactile division of a single medium can be effectively used to distinguish a single medium such as cloth, wood, plastic, and metal when it is implemented and operated in a robot hand. However, most of the objects encountered in daily life are composed of complex media, so the existing technology has a limitation in obtaining information of concrete objects.

In an exemplary embodiment of the present invention, a method for recognizing and distinguishing a touch input at any time and classifying the touch of a plurality of media as well as the touch of a single medium is provided.

In addition, a preferred embodiment of the present invention is to provide a method for classifying the touch of a plurality of media, and recognize the spatial information indicating how the regions of different media are divided.

As a preferred embodiment of the present invention, the tactile detection device is a multi-array tactile sensor composed of n * m tactile sensors (n, m is any natural number) to scan the surface of a medium to position coordinates of n * m tactile sensors. And a tactile input receiver for receiving n * m tactile input signals corresponding thereto in real time; A tactile signal detection unit that determines that a tactile input exists when the value of the tactile input signal exceeds a preset value, and detects at which position on the surface of the medium the tactile input signal is generated; And a tactile classification unit for classifying and outputting the tactile input according to the previously learned threshold values for the media, when the tactile input is determined. The tactile input receiving unit includes a first action and a second action. Scanning is performed through.

Preferably, the first action is a tapping operation, and the second action is a rubbing operation.

Preferably, the multi-array tactile sensor is characterized in that it is possible to receive a tactile input signal from a plurality of media at the same time. Each of the n * m tactile sensors constituting the multi-array tactile sensor transmits n * m time series data including a position on the surface of the medium and a tactile input signal detected at the corresponding position over time.

In a preferred embodiment of the present invention, the tactile signal detection unit scans the surface of the medium through the first action, and corresponds to the position coordinates of the n * m tactile sensors received from the n * m tactile sensors and corresponding thereto. n * m tactile input signals are mapped, and the surface of the medium is scanned through the second action, and position coordinates of n * m tactile sensors received by the n * m tactile sensors and n * corresponding thereto The m touch input signals are mapped, and the tactile input signals acquired through the first action and the tactile input signals obtained through the second action are mapped based on the position coordinates of the surface of the medium.

As another preferred embodiment of the present invention, the tactile detection method is a multi-array tactile sensor composed of n * m tactile sensors in the tactile input receiver to scan the surface of the medium and the position coordinates of the n * m tactile sensors and the corresponding receiving n * m tactile input signals in real time; Determining, by the tactile signal detector, that a tactile input exists when the value of the tactile input signal exceeds a preset value, and detecting at which position of the surface of the medium the tactile input signal is generated; And when the tactile classification unit determines that the tactile input exists, classifying and outputting the tactile input according to a threshold value for each previously learned medium, wherein the tactile input receiving unit outputs the surface of the medium to the first action and the first. The scanning may be performed through two actions.

In a preferred embodiment of the present invention, it is possible to recognize and distinguish the touch input at any time, and to classify the touch of a plurality of media as well as the touch of a single medium. In particular, the objects encountered in daily life are mostly composed of a composite medium, the structure of the combination of media for each product is different, there is an effect that can obtain all the information.

In addition, in a preferred embodiment of the present invention by using a multi-array tactile sensor can measure the spatial signal to distinguish the two or more complex tactile sense, it is also possible to estimate what structure each tactile medium is divided into It has an effect.

1 is a view illustrating an internal configuration of a tactile detection device as a preferred embodiment of the present invention.
2 (a) and 2 (b) show an example of performing scanning of a surface of a medium in a tactile detection device as a preferred embodiment of the present invention.
3 illustrates an example of a plurality of channel signals received by the multi-array tactile sensor of the tactile detection device according to one preferred embodiment of the present invention.
4 illustrates an example in which a tactile signal is recognized and classified by the tactile detection device to output a tactile classification result according to an exemplary embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view illustrating an internal configuration of a tactile detection device as a preferred embodiment of the present invention.

The tactile detection device 100 includes a tactile input receiver 110, a tactile signal detector 120, and a tactile classification unit 130. The tactile input receiver 110 may receive a tactile input S110 obtained by scanning a surface of the medium or an unintentional tactile input S112 obtained at an arbitrary time. Examples of the unintentional tactile input S112 include an example of inadvertently contacting a medium while driving the robot arm, and in the present invention, the sensor value received from the multi-array tactile sensor 210 is used as the first action or first. 2 Classify one of the action values and map it.

The tactile input receiver 110 is a multi-array tactile sensor composed of n * m tactile sensors (n and m are arbitrary natural numbers) and scans the surface of the medium to detect position coordinates of n * m tactile sensors and corresponding n * m. Receive m tactile input signals in real time.

The tactile input receiver 110 may scan the surface of the medium through the first action and the second action. The first action may be a three-dimensional action, and the second action may be a two-dimensional action. In addition, the first action may be an action for measuring the strength of the tactile medium, and the second action may be an action for measuring the roughness of the medium. In one preferred embodiment of the present invention, the first action may be a tap and the second action may be a rub. 2 (a) to 2 (b) show an example of scanning the surface of the medium in the tactile input receiver 110.

2 (a) shows an example of scanning the surface 220 by tapping (S230) in the multi-array tactile sensor 210 attached to the arm 200 of the robot. Tapping is a measure of the strength of the surface of the medium by pressing the sensor onto the surface of the medium. At this time, in order to measure the strength of the entire surface of the medium surface, the tapping operation of the matrix type multi-array sensor may be performed several times while moving on the plane coordinates to measure the strength of the tactile medium.

Figure 2 (b) shows an example of scanning by rubbing (S240) method. The rubbing operation may measure the roughness of the medium by moving the multi-array tactile sensor 210 from one point of the medium surface to another point to scan the entire area of the medium surface.

In a preferred embodiment of the present invention, when the multi-array tactile sensor 210 performs tapping (S230) or rubbing (S240), each of the n * m sensors constituting the multi-array tactile sensor 210 is a medium It receives the position touched with the surface and the tactile input data value received at the position in real time. As a preferred embodiment of the present invention, the multi-array tactile sensor 210 can detect a plurality of media at the same time according to the position of the surface of the medium. Referring to FIG. 2B, when the medium surface 220 includes the first medium 221 and the second medium 222, the multi-array tactile sensor 210 may include the first medium 221 and the second medium ( The first medium 221 and the second medium 222 may be detected at the same time when rubbing or tapping the boundary point of the 222.

The tactile signal detector 120 determines that a tactile input exists when the value of the tactile input signal exceeds a preset value, and detects at which position on the surface of the medium (FIGS. 2 and 220) the corresponding tactile input signal exists. .

The tactile signal detection unit 120 scans the surface of the medium through a first action, and maps position coordinates of n * m tactile sensors received from n * m tactile sensors and n * m tactile input signals corresponding thereto. Also, the surface of the medium is scanned through the second action, and the position coordinates of the n * m tactile sensors received from the n * m tactile sensors and the corresponding n * m tactile input signals are mapped, and the position of the medium surface is mapped. The tactile input signal acquired through the first action and the tactile input signal obtained through the second action are mapped based on the coordinates. In addition, when an unintentional tactile input signal is detected, the detected unintentional tactile input signal may be further mapped.

The tactile signal detection unit 120 determines that the tactile input exists when the value of the tactile input signal exceeds a preset value. Referring to FIG. 3, the tactile detection device receives n * m channel signals in real time from a multi-array tactile sensor.

Referring to FIG. 3, the multi-array tactile sensor 310 scans the surface 300 of the tactile medium 1 301 and the tactile medium 2 302 by rubbing the n * m channel signals S330. ) Is generated in real time, and the tactile signal detection unit 120 receives n * m channel signals.

The tactile signal detector 120 detects a section S340 in which the received channel signal exceeds a preset threshold. The tactile signal detector 120 may prestore threshold values previously learned for various media. The pre-learned threshold may be a first threshold, a second threshold, a third threshold, and the like, and each threshold may be set within a certain range, above a certain range, above a certain value, below a certain value, or the like. Can be. When there are various thresholds, the tactile signal detection unit 120 may refer to when setting the tactile input sensing interval S340 by extracting the thresholds and the intersections 341, 342, 343, 344, 345, and 346. For example, the tactile input detection section S340 may set an intersection point 346 between the first threshold value and the intersection point 341 to the last threshold value.

The tactile signal detection unit 120 monitors in real time the time series data input from the n * m sensors in real time in order to determine whether the received channel signal exceeds a preset threshold.

In a preferred embodiment of the present invention, the tactile signal detection unit 120 may determine the presence of the tactile input through the following equation in the case of the tapping operation.

, 이 경우 k는 n보다 작은 수이며, 신호의 변화에 민감하게 반응하고, 노이즈 효과를 상쇄 할 수 있는 특정 윈도우 값을 나타낸다. 샘플링 주파수를 고려하여, 일 예로 샘플링 주파수의 반 또는 그보다 작은 수를 고려한다. 이 값은 센서의 민감도와 노이즈 정도에 따라 유동적으로 바뀔 수 있다. 이렇게 구해진 두드리기에 대한 특이값

을 특정 임계값과 비교를 하여 임계값을 초과하였을 시에 촉감이 입력되었음을 인지할 수 있다. 임계값은 촉감 입력이 없을 때의 특이값 검출값 h_b에 조절값 α를 곱한값 α*h_b을 사용한다.

In this case, k is a number smaller than n, and represents a specific window value that is sensitive to the change in the signal and can cancel the noise effect. In consideration of the sampling frequency, for example, consider half or less of the sampling frequency. This value can be changed flexibly depending on the sensitivity of the sensor and the noise level. Singular value for the tap thus obtained

By comparing with a specific threshold value it can be recognized that the touch is input when the threshold value is exceeded. The threshold value is a value α * h _b obtained by multiplying the control value α by the singular value detection value h _b when there is no tactile input.

As a preferred embodiment of the present invention, the tactile signal detection unit 120 may determine the presence of the tactile input through the following equation in the case of a rubbing operation. r _x (n) represents a singular value.

, 이 경우 임계값은 두드리기 동작과 비슷하게 문지르기 동작이 없고, 입력이 없을 때 특이값 검출값 r_b에 조절값 β를 곱한값 β*r_b을 이용한다.

In this case, the threshold value is similar to the tapping operation, and there is no rubbing operation. When there is no input, the singular value detection value r _b is multiplied by the adjustment value β and β * r _b is used.

As a preferred embodiment of the present invention, the tactile signal detection unit 120 is the starting point of the tactile input detection for tapping and rubbing is the time when at least one channel that exceeds the threshold value in all the n * m channels, May be considered as the detection start time, and the end time of the tactile input detection may be set to the time when the tactile input disappears in all n * m channels.

When the tactile classification unit 130 determines that the tactile input is present in the tactile signal detector 120, the tactile classification unit 130 classifies the tactile input according to the previously learned threshold values for the medium. 4 illustrates an example in which a tactile signal is recognized and classified by the tactile detection device to output a tactile classification result according to an exemplary embodiment of the present invention.

In a preferred embodiment of the present invention, the tactile classification unit 130 recombines data obtained by mapping the sensor data obtained by tapping or rubbing from n * m sensors for each coordinate of the surface of the medium in the tactile detection unit 120. Perform tactile classification.

The tactile classification unit 130 is implemented to operate based on the deconvolution 440 utilizing the convolutional neural network in the basic structure of deep learning. First, a next layer is configured by using a convolution filter of deep learning technology based on spatially recombined data by tapping operation S430 and rubbing operation S440.

The input size of tapping and rubbing depends on the size of the detected time series signal, but the size and structure of the convolutional layer 430 are the same size. Thereafter, each node of the convolutional layer 430 is flattened through flattening 431 and passed through a fully connected layer 431. In this case, too, the structure and size of the fully connected layer 431 of tapping and rubbing are the same. After that, the neural network of tapping and scrubbing is integrated to form a structure such as one fully connected layer 432 structure, and then to the same convolutional layer used for tapping and rubbing. (440).

The tactile classification unit 130 then classifies and outputs the tactile input according to the medium classification criteria previously learned through machine learning (450). When the tactile input is classified into a plurality of media 451 and 452, the tactile classification unit 130 outputs the regions of the media so as to be spatially separated. Referring to FIG. 4, the first medium 451 and the second medium 452 are represented in different forms and output so as to be spatially separated.

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

Claims (11)

Multi-array tactile sensor consisting of n * m tactile sensors (where n and m are arbitrary natural numbers) scans the surface of the medium to detect the position coordinates of n * m tactile sensors and the corresponding n * m tactile input signals in real time Tactile input receiving unit for receiving;
A tactile signal detection unit that determines that a tactile input exists when the value of the tactile input signal exceeds a preset value, and detects at which position on the surface of the medium the tactile input signal is generated;
And a tactile classification unit for classifying and outputting the tactile input according to a threshold value for each previously learned medium when it is determined that the tactile input exists.
The tactile input receiving unit scans the surface of the medium through a first action and a second action, and the tactile classification unit outputs the areas of each medium to be spatially separated when the tactile input is classified into a plurality of mediums. Tactile detection device, characterized in that. The method of claim 1,
Wherein the first action is a tapping operation, and the second action is a rubbing operation. The method of claim 1, wherein the multi-array tactile sensor
And a haptic input signal from a plurality of media at the same time. The method of claim 1, wherein the tactile signal detection unit
Scanning the surface of the medium through the first action and mapping position coordinates of n * m tactile sensors received from the n * m tactile sensors and n * m tactile input signals corresponding thereto;
In addition, the surface of the medium is scanned through the second action, and the position coordinates of the n * m tactile sensors received from the n * m tactile sensors and n * m tactile input signals corresponding thereto are mapped,
And a tactile input signal acquired through the first action and a tactile input signal obtained through the second action based on the position coordinates of the surface of the medium. The method of claim 1, wherein the multi-array tactile sensor
and each n * m tactile sensor transmits n * m time series data including a position on the surface of the medium and a tactile input signal detected at the corresponding position over time. The method of claim 5, wherein the tactile classification unit
And at least one medium is classified from n * m time series data outputs constituting the multi-array tactile sensor. The method of claim 1, wherein the multi-array tactile sensor
Tactile detection device, characterized in that attached to the robot finger. The method of claim 1, wherein the tactile signal detection unit
And an unintentional tactile signal detection unit that determines that a tactile input exists when the unintentional tactile input signal detected by the multi-array tactile sensor exceeds a preset value. Detection device. delete The method of claim 1,
Wherein the first action is an action of measuring the strength of the surface of the medium, and the second action is an action of measuring the roughness of the surface of the medium. Scanning the surface of the medium with a multi-array tactile sensor composed of n * m tactile sensors at the tactile input receiver to receive the position coordinates of the n * m tactile sensors and n * m tactile input signals corresponding thereto in real time;
Determining, by the tactile signal detector, that a tactile input exists when the value of the tactile input signal exceeds a preset value, and detecting at which position of the surface of the medium the tactile input signal is generated;
And classifying and outputting the tactile input according to a threshold value for each previously learned medium when the tactile classification unit determines that the tactile input exists.
The tactile input receiver performs scanning of the medium surface through a first action and a second action,
The tactile classification unit, when the tactile input is classified into a plurality of media, the tactile classification method outputs the regions of each medium to be spatially separated.

Images