KR102435965B1 - System amd method for recongnizing shape of object using neural network based on unsupervised learning - Google Patents

Abstract

The object shape recognition system using an unsupervised learning-based neural network of the embodiment includes a sensor unit for sensing tactile information applied to an object, and a recognition unit including a neural network based on unsupervised learning, wherein the recognition unit is sensed from the sensor unit The received tactile information is input to the neural network to recognize the shape of an object, and the neural network accumulates the input value extracted from the tactile information sensed by the sensor unit in the membrane potential of the input layer neurons, and the accumulated input value generates a spike heat signal based on shape can be recognized.
The embodiment has an effect of recognizing the shape of an object more accurately and precisely by using tactile information.

Description

Object shape recognition system and method using neural network based on unsupervised learning

Embodiments relate to an object type recognition system for recognizing a type of an object.

In general, Artificial Neural Networks (ANN) and Deep Neural Network (DNN) technologies, which are widely used in artificial intelligence research, are making unprecedented contributions in the field of image processing, and robotic hands based on computer vision Manipulation of the brain has made significant progress based on visual function. Manipulation of an object through a robot For example, a deep neural network enables the robot to identify an object's location, predict a gripping point, or classify an object's type based on visual information.

On the other hand, the tactile information of an object also has very important information required when a robot manipulates an object using its hands, but its technology level is far behind compared to the field of robot vision.

Tactile information processing allows robots to use their hands to grip objects more accurately and precisely, and to perform sophisticated manipulation tasks such as assembly.

Currently, most tactile information processing methods for robots have relied on machine learning methods involving deep neural networks to infer tactile information from the physical signals of pressure sensors.

However, the robot's tactile recognition using a deep neural network that mimics the structure and function of the visual cortex has its limitations, and its performance is significantly lower than that of humans. Therefore, there is a need to develop an artificial intelligence algorithm that applies the operating principle of the somatosensory nervous system that enables human tactile information processing.

Domestic Registered Patent No. 10-1991721 (2019.06.17.) Domestic Patent Publication No. 10-2018-0024006 (2018.02.27.)

In order to solve the above-mentioned problems, an object of the embodiment is to provide an object shape recognition system and method using an unsupervised learning-based neural network for effectively recognizing a type of an object.

The object shape recognition system using an unsupervised learning-based neural network of the embodiment includes a sensor unit for sensing tactile information applied to an object, and a recognition unit including a neural network based on unsupervised learning, wherein the recognition unit is sensed from the sensor unit The received tactile information is input to the neural network to recognize the shape of an object, and the neural network accumulates the input value extracted from the tactile information sensed by the sensor unit in the membrane potential of the input layer neurons, and the accumulated input value Generates a spike heat signal based on shape can be recognized.

The sensor unit may sense pressure applied to the object to sense tactile information, and the pressure range may include 10Pa to 140kPa.

The sensor unit may sense the pressure by using a current and a resistance value generated as the object approaches the object.

An input value input to the neural network may be a current value.

The neural network may generate the spike heat signal when the accumulated value is greater than or equal to a threshold value.

The spike column signal may be expressed as a binary number including 0 or 1.

The threshold may include 25mV to 35mV.

The input current may be measured using a synaptic current, an external stimulus current coming from the sensor unit, and Gaussian noise.

The recognition unit may control the strength of the first synapse by tracking the spike heat signal.

The recognition unit may reduce a weight of a neuron without the spike heat signal.

The output layer neurons may include a plurality, and the output layer neurons may be connected through a second synapse.

In addition, the embodiment provides an object shape recognition method performed in an object shape recognition system, the steps of detecting tactile information applied to an object, and inputting the sensed tactile information to an unsupervised learning-based neural network to form an object and the neural network accumulates the input value extracted from the sensed tactile information in the membrane potential of the input layer neurons, generates a spike heat signal based on the accumulated value, and the generated spike Information may be transmitted to the input layer neuron and the output layer neuron connected through a first synapse using the thermal signal, and the shape of the object may be recognized based on the information output from the output layer neuron.

The embodiment has an effect of recognizing the shape of an object more accurately and precisely by using tactile information.

In addition, the embodiment has the effect of remarkably reducing the amount of computation for learning by using a neural network based on tracking the spike heat signal.

1 is a block diagram illustrating an object shape recognition system using a neural network based on unsupervised learning according to an embodiment.
2 is a view showing a robot gripper to which a sensor unit is mounted.
3 is a diagram illustrating a neural network based on unsupervised learning according to an embodiment.
4 is a graph illustrating a signal output from a sensor unit.
5 is a graph showing a learning result of a spiking neural network through unsupervised learning.
6 is a graph illustrating object classification through a spiking neural network.
7 is a flowchart illustrating an object shape recognition method using a neural network based on unsupervised learning according to an embodiment.

Hereinafter, the embodiment will be described in detail with reference to the drawings.

1 is a block diagram illustrating an object shape recognition system using a neural network based on unsupervised learning according to an embodiment, FIG. 2 is a diagram showing a robot gripper equipped with a sensor unit, and FIG. of the neural network, Figure 4 is a graph showing the signal output from the sensor unit, Figure 5 is a graph showing the learning result of the spiking neural network through unsupervised learning, Figure 6 is the object through the spiking neural network It is a graph showing classification.

Referring to FIG. 1 , the object shape recognition system according to the embodiment may include a sensor unit 100 .

The sensor unit 100 may sense information applied to the object. The sensor unit 100 may be a tactile sensor that senses tactile information, but is not limited thereto. The sensor unit 100 may be mounted on the robot. The robot may grasp the object and perform an action of moving or assembling the object.

As shown in FIG. 2 , the sensor unit 100 may be mounted on a gripper of the robot 10 . The sensor unit 100 may be mounted on grippers facing each other. For this purpose, a plurality of sensor units 100 may be provided.

Each sensor unit 100 may be formed to have a sensor of 21 channels in a 7X3 array inside the urethane sponge. The size of the sensor pixel may be 3mm*3mm (2.36 pixels/cm ² ), and the sensor active pressure range may be a minimum of 10Pa and a maximum of 140kPa.

As the robot 10 gripper moves closer to the object 20 to grip the object 20 and the sensor unit 100 increases the distance between the sensors inside the polyurethane sponge according to the pressure applied to the object 20, By increasing the conduction in the vertical direction (increasing current, decreasing resistance), pressure can be sensed. The object 20 may include, for example, a triangular pole, a cylindrical pole, a polygonal pole, and the like, but is not limited thereto.

Returning to FIG. 1 , the object shape recognition system according to the embodiment may include the recognition unit 300 .

The recognition unit 300 may recognize the shape of an object by using the tactile information sensed by the sensor unit 100 .

The recognition unit 300 may include a neural network 200 . For example, the neural network 200 may be an unsupervised learning-based neural network. As an example, the neural network 200 may include a spiking neural network, but is not limited thereto.

The neural network 200 may simultaneously learn while collecting tactile information sensed by the sensor unit 100 .

As shown in FIG. 3 , the neural network 200 may include an input layer 210 and an output layer 220 . The input layer 210 may be formed to have two or more input layers, but is not limited thereto. Similarly, the output layer 220 may be formed to have two-phase output layers. In the embodiment, the neural network 200 having a structure including one input layer 210 and one output layer 220 will be described.

The input layer 210 may include a plurality of neurons. The output layer 220 may be connected to the input layer 210 through the first synapse 230 . The first synapse 230 may be an excitatory synapse. The output layer 220 may include a plurality of neurons. In the embodiment, the number of neurons in the output layer 220 is set to be the same as the number of types of objects.

Neurons of the output layer 220 may be connected by a second synapse 240 . The second synapse 240 may be an inhibitory synapse.

The input layer 210 neurons may be formed in a number corresponding to 42 (21 * 2) sensors 110 attached to the robot gripper. Of course, the number is not limited.

The input value extracted from the sensor unit 100 may be accumulated in the membrane potential of the neurons of the input layer 210 . The input value may be a current value. The neurons of the input layer 210 may generate a spike thermal signal by igniting a spike when the membrane potential accumulated in the membrane potential is greater than or equal to a threshold value. Here, the threshold may be set within the range of 25 mV to 35 mV, and may be 35 mV. The threshold value is not limited thereto.

The spike column signal may be set to a binary value. When a spike occurs, a signal of 1 can be generated, and when no spike occurs, a signal of 0 can be generated.

The spike heat signal generated as described above may be transmitted to the output layer neuron through the first synapse 230 .

The spike heat signal transmitted to the output layer neurons 220 exchanges the spike heat signal with each other by the second synapse 240 connecting the output layer neurons 220 , and then outputs a final output signal. Accordingly, the shape of the object is recognized based on the output signal.

The number of output layer neurons 220 may be set equal to the number of objects to be recognized, and each output layer neuron 220 has shape information. Therefore, it is possible to determine the shape of the object according to which output layer neuron 220 outputs the signal.

[Algorithm used in Examples]

In the Examples, the generation of membrane potentials of biological neurons was simulated using the Izhikevich model. Here, the synapses referred to below may be applied to both the first synapse and the second synapse.

The membrane potential (v) can be updated according to a nonlinear differential equation including the external stimulation current (I) and the recovery variable (u) as shown in Equations 1 to 3.

[Equation 1]

[Equation 2]

[Equation 3]

External stimuli or synaptic currents from previous neurons can be transmitted through I. Neurons accumulate an input current at a membrane potential, generate a spike signal when a threshold voltage of 30 mV is exceeded, and then transmit it to the next layer of neurons via a synapse.

Information transfer between neurons can be implemented through a spike heat signal including a binary value between neurons (1: spike occurs, 0: spike does not occur).

The parameters of the Izkivic model can be set using properties of the Regular Spiking Neuron.

Table 1. Parameter settings of firing neurons

[Learning of spiking neural networks]

)로 구성될 수 있다.The input current I _input is the synaptic current I _syn The external stimulus current I _sens and Gaussian noise (

) may consist of

[Equation 4]

W _ij may be a synaptic strength indicating a connection strength between a presynaptic neuron i and a postsynaptic neuron j. g+ and g- may mean an excitatory synapse (first synapse) and an inhibitory synapse (second synapse), respectively.

[Equation 5]

A signal input from the sensor signal can be used as an external stimulation current through a scale factor (k=0.01 in the embodiment) as follows.

[Equation 6]

The synaptic strength may be learned on input data through a spike-timing-dependent plasticity (STDP) algorithm. STDP can increase the strength of connections between pairs of neurons only when the post-synaptic neuron responds to the pre-synaptic neuron. STDP can play an important role in allowing each output layer neurons to encode information about a specific object.

The conventional STDP algorithm has a large amount of computation, but in the embodiment, the STDP based on the spike tracking is implemented instead of using the STDP algorithm directly.

The spike tracking equation is Equation 7.

[Equation 7]

는 시냅스 전 뉴런이 스파이크를 발생시킨 시간을 나타내고,

는 상수를 의미할 수 있다. 스파이크 추적은 시냅스 전 뉴런의 스파이크가 발생하지 않으면 지속적으로 감소할 수 있다.where y _i represents the spike trace of the presynaptic neuron,

represents the time the presynaptic neuron generated a spike,

may mean a constant. Spike tracking can be continuously reduced if presynaptic neuron spikes do not occur.

In an embodiment, synaptic strength may be learned by a pair-based STDP learning method. 8 illustrates an expression for increasing the weight of a neuron, and FIG. 9 may indicate an expression for decreasing the weight of a neuron.

[Equation 8]

[Equation 9]

λ represents the learning rate, and y _i and y _j may represent the spike traces of pre-synaptic neurons and post-synaptic neurons, respectively. (1-W _ij ) and W _ij can be modeled such that synaptic strength is maintained in the range of 0 to 1 to mimic biological properties.

) 학습 규칙에 따라 시냅스 강도는 시냅스 전 스파이크 자취(y_i)에 비례하여 증가할 수 있다. 이와 다르게, 시냅스 전 뉴런이 발화하면(

) 시냅스 후 스파이크 자취(y_i(t))에 비례하여 강도가 감소할 수 있다.When a post-synaptic neuron fires (

) according to the learning rule, the synaptic strength can increase in proportion to the presynaptic spike trace (y _i ). In contrast, when a presynaptic neuron fires (

) may decrease in intensity proportional to the post-synaptic spike trace (y _i (t)).

The pair-based STDP technique used in the embodiment can gradually reduce the weight of non-firing neurons, which prevents an indiscriminate increase in the sum of total weights and reduces the similarity between individual output layer neurons, thereby effectively learning.

[Robot grip experiment to verify spiking neural network and learning algorithm]

Robot gripping experiments were performed on three selected columnar rod stimuli of different shapes. The gripping of an object may consist of a grasp phase and a release phase.

As shown in FIG. 4 , the tactile sensor is attached to two robot gripper fingers, respectively, and a total of 42 multi-channel time series data can be collected including 21 channels in a 7X3 arrangement on each finger. The gripping experiment was repeated 13 times for each object.

The pixel size of the sensor is 3mm*3mm (2.36 pixels/cm ² ), and the pressure range of the sensor activity can be set from a minimum of 10Pa to a maximum of 140kPa.

[Experiment result]

In the embodiment, the spiking neural network learned grip data of various objects while evaluating how well the tactile information of the object transmitted from the sensor was reflected after learning.

As shown in FIGS. 5A and 5B , it can be seen that the synaptic strength was learned in the direction of the amplitude pattern of the tactile sensor signal.

As shown in Figure 5c, when the learning rate (λ) is set to 0.001 and the number of iterations of the model is set to 30, the synaptic weights converge. Here, the learning rate and the number of repetitions were experimented while changing within a certain range.

When we tested lateral inhibition and weight reduction through pair-based STDP learning, as a result, when one output layer neuron was activated, it suppressed the activity of other output layer neurons except itself, resulting in winner-takes-all (WTA). ) form of competitive learning effect. Here, the competitive learning may be a form in which only one output neuron monopolizes learning with respect to one input pattern.

As shown in FIG. 6 , it can be seen that the model speaks exclusively in response to a specific sensory input through learning through the change in the firing activity of the output neurons before and after learning.

As a result, it can be seen that the shape recognition system of the embodiment can distinguish the spiking neural network from the tactile sensor signal to the tactile shape for a separate stimulus.

7 is a flowchart illustrating an object shape recognition method using a neural network based on unsupervised learning according to an embodiment.

Referring to FIG. 7 , the method for recognizing an object shape using a neural network based on unsupervised learning according to an embodiment includes the steps of sensing tactile information applied to an object (S100), and applying the sensed tactile information to the neural network based on unsupervised learning. It may include a step (S200) of recognizing the shape of the object by inputting to the . Here, the object shape recognition method may be performed in an object shape recognition system.

Sensing the tactile information applied to the object ( S100 ) may be collected from a tactile sensor mounted on the robot gripper.

Sensing the tactile information applied to the object (S100) may be pressure information sensed by increasing the conduction in the vertical direction (increase current, decrease resistance) as the distance between the sensors increases according to the pressure applied to the object. .

The step of recognizing the shape of the object by inputting the sensed tactile information to the unsupervised learning-based neural network ( S200 ) may recognize the shape of the object using the spiking neural network.

The spiking neural network accumulates the input value extracted from the sensed tactile information in the membrane potential of the input layer neurons, generates a spike heat signal based on the accumulated value, and uses the generated spike heat signal to input the input value. Information can be transmitted to the layer neuron and the output layer neuron connected through the first synapse.

The step of recognizing the shape of an object by inputting the sensed tactile information to the neural network based on unsupervised learning (S200) is to track the spiking heat signal generated in the neural network and adjust the weight of the neurons to increase the recognition rate of the object shape. can

Although the above has been described with reference to the drawings and embodiments, those skilled in the art will understand that various modifications and changes can be made to the embodiments without departing from the spirit of the embodiments described in the claims below. will be able

100: sensor unit
200: neural network
210: input layer neurons
220: output layer neuron
230: first synapse
240: second synapse
300: recognition unit

Claims (12)

a sensor unit for sensing tactile information applied to an object; and
It includes a recognition unit including a neural network based on unsupervised learning,
The recognition unit receives the tactile information sensed from the sensor unit and inputs it to the neural network to recognize the shape of the object,
The neural network accumulates the input value extracted from the tactile information sensed from the sensor unit at the membrane potential of the input layer neurons, generates a spike heat signal based on the accumulated input value, and uses the generated spike heat signal an object shape recognition system for transmitting information to an output layer neuron connected through a first synapse with the input layer neuron, and recognizing the shape of an object based on the information output from the output layer neuron. According to claim 1,
The sensor unit senses pressure applied to the object to sense tactile information, and the pressure range is 10Pa to 140kPa. 3. The method of claim 2,
The sensor unit is an object shape recognition system that senses pressure by using a current and a resistance value generated as the object approaches the object. 4. The method of claim 3,
An object shape recognition system in which the input value input to the neural network is a current value. According to claim 1,
The spike heat signal is generated when the accumulated input value is greater than or equal to a threshold value. 6. The method of claim 5,
wherein the spike column signal is expressed as a binary number including 0 or 1 in an object shape recognition system. 6. The method of claim 5,
The threshold value is an object shape recognition system including 25mV to 35mV. 4. The method of claim 3,
The input value is an object shape recognition system that is measured using a synaptic current, an external stimulus current coming from the sensor unit, and Gaussian noise. According to claim 1,
The recognition unit is an object shape recognition system for controlling the strength of the first synapse through the tracking of the spike heat signal. According to claim 1,
The recognition unit reduces the weight of neurons without the spike heat signal. According to claim 1,
The output layer neurons include a plurality, and the output layer neurons are connected through a second synapse. A method for recognizing an object shape performed in an object shape recognition system, the method comprising:
detecting tactile information applied to the object; and
Recognizing the shape of an object by inputting the sensed tactile information to an unsupervised learning-based neural network,
The neural network accumulates the input value extracted from the sensed tactile information in the membrane potential of the input layer neurons, generates a spike heat signal based on the accumulated value, and uses the generated spike heat signal to the input layer An object shape recognition method for transmitting information to a neuron and an output layer neuron connected through a first synapse, and recognizing a shape of an object based on information output from the output layer neuron.

Images