CN115574998A

CN115574998A - Optical fiber light spot touch sensor based on reflection type probe structure

Info

Publication number: CN115574998A
Application number: CN202210896035.2A
Authority: CN
Inventors: 刘艳; 邹小丽; 李广德; 秦齐
Original assignee: Beijing Jiaotong University
Current assignee: Beijing Jiaotong University
Priority date: 2022-07-27
Filing date: 2022-07-27
Publication date: 2023-01-06

Abstract

The invention provides an optical fiber light spot touch sensor based on a reflection type probe structure. The method comprises the following steps: the optical fiber light guide device comprises a hemispherical contact piece, a reflector, a base and input and output optical fibers embedded into the base, wherein the light guide mode of the optical fibers is a reflection mode, the input optical fibers are single-mode optical fibers, light signals from a light source are guided into a probe to irradiate on the reflector, the output optical fibers are multi-mode optical fibers, light reflected by the reflector is conducted to a receiving camera, the camera collects light spot signals, the probe structure is made of silicone rubber and resin through 3D printing, the hardness of the hemispherical contact piece is higher than that of the reflector, and the force applied to the probe contact piece causes the reflector to deform, so that the output light spots are changed. The optical fiber light spot touch sensor takes the light spot pattern as a sensing signal, has high sensitivity, and has stronger bending interference resistance because the change of the light spot is mainly caused by the deformation of the reflector and the influence of the change of the bending state of the output optical fiber on the output light spot is smaller.

Description

Optical fiber light spot touch sensor based on reflection type probe structure

Technical Field

The invention relates to the technical field of touch sensors, in particular to an optical fiber light spot touch sensor based on a reflection type probe structure.

Background

In recent years, tactile sensing has been widely used in the fields of human-computer interaction, robot pose control, and medical surgery, particularly minimally invasive surgery. To improve the tactile perception, many tactile sensing schemes have been proposed, such as resistive, capacitive, inductive, optical, piezoelectric, ultrasonic, magnetoelectric, etc. The touch sensor based on the optical type has the characteristics of high sensitivity, good reliability, electromagnetic interference resistance and the like, and has wider application prospect. The currently proposed optical fiber tactile sensing is mainly researched from the aspects of spectral shift, light intensity, light spot patterns and the like. The former two tend to suffer from high cost or relatively low sensitivity. Wherein the sensitivity achieved by tactile sensing based on light intensity changes is low; the detection of the spectral shift requires expensive measurement equipment, and if FBGs are used as sensing devices, the manufacturing process is complicated, which increases the cost. Compared with the former two, the sensor based on facula pattern simple structure only needs a laser instrument, an optic fibre and a small-size industrial camera or miniature camera head, and the facula pattern is very easily influenced by optic fibre state change again, therefore the sensing system based on facula has advantages such as portable, low cost, high sensitivity.

In the prior art, a scholarly proposes a touch sensor based on optical fiber light spot analysis. The sensor consists of 9 microbend structures, is connected to 3 multimode optical fibers and is distributed on the contact surface in a matrix arrangement. The normalized inner product coefficients of the speckle pattern are calculated and the magnitude and location of the force exerted on the haptic frame is estimated by a data fusion technique. The sensor realizes 0.5N ^-1 Sensitivity, a change in force position of 1mm can be detected with an accuracy of 86%.

Still another scholars have proposed a 2D tactile sensor composed of a single-wound multimode optical fiber embedded in an elastic substrate, and a CCD (charge coupled device) collects a light spot image output from the optical fiber when a pogo pin automatically scans on a 2-dimensional plane. And inputting the collected light spots into a convolutional neural network for training and testing. The 2D tactile sensor realizes the spatial position identification with the accuracy rate of more than 98% and the force sensing with the accuracy rate of 100%.

The two touch sensors are of a planar structure, are simple to manufacture, have limited application range and are only suitable for surface sensing. Compared with a plane type probe, the probe type probe has the advantages of compact structure, high sensitivity and wide application range. Meanwhile, the touch sensor is composed of a plurality of elements in an arrangement mode, the structure is complex, and a single probe element of the touch sensor has improved space in the aspects of contact force position identification and force sensing sensitivity. These problems are the core problems that the present invention is to be studied to solve.

Disclosure of Invention

The embodiment of the invention provides an optical fiber light spot touch sensor based on a reflection type probe structure, so that accurate touch perception of the optical fiber light spot touch sensor is realized.

In order to achieve the purpose, the invention adopts the following technical scheme.

A fiber optic spot tactile sensor based on a reflective probe structure, comprising: the optical fiber probe comprises a hemispherical contact member, a reflector, a base and two optical fibers embedded in the base, wherein the two optical fibers are respectively an input optical fiber and an output optical fiber of the probe, the light guide mode of the optical fibers is a reflection mode, the input optical fiber is a single-mode optical fiber, an optical signal from a light source is guided into the probe and is irradiated on the reflector, the output optical fiber is a multi-mode optical fiber, the light reflected by the probe reflector is transmitted to a receiving camera, the camera collects a light spot signal, the light source and the camera receiving the light spot signal are positioned on the same side, the probe structure is made of silicone rubber and resin through 3D printing, the hemispherical contact member and the reflector are made of silicone rubber, the base is made of resin plastic, the Shore hardness of the silicone rubber is smaller than that of the resin plastic, the hardness of the hemispherical contact member is higher than that of the reflector, and the deformation of the reflector is caused by the force applied to the probe contact member.

Preferably, the reflector in the sensing probe is composed of two reflectors placed at an angle of 90 °, and at the bottom of the reflector, there is a cylindrical base made of resin plastic for fixing the input single mode fiber and the output multimode fiber of the probe, keeping the input single mode fiber and the output multimode fiber of the probe parallel to each other.

Preferably, light output by the light source enters the probe base through the single-mode fiber and is guided to the reflector, and the transmission direction of the light beam is changed by nearly 180 degrees through reflection of two vertical reflectors in the reflector, so that the light enters the output multimode fiber; when a contact force is applied to the sensing probe, the reflector in the probe deforms, the angle between two originally vertical reflectors in the reflector changes, so that the transmission path of light from the single-mode optical fiber to the multimode optical fiber changes, the spot pattern detected by the camera also changes, the deformation of the reflectors gradually increases along with the increase of the contact force, the change of the transmission path of the light increases, the change of the spot pattern acquired by the camera increases, and the change information of the contact force can be acquired by analyzing the change degree of the spot pattern.

Preferably, the signal processing process of the optical fiber tactile sensor with the probe-type structure comprises the following processing steps:

step 1, collecting a light spot pattern signal, cutting the light spot pattern signal along the edge of a light spot, and removing redundant information;

step 2, converting the cut light spot image into a gray light spot image;

step 3, scaling the gray level light spot image to a uniform size;

and 4, calculating a zero-mean normalized cross-correlation ZNCC value of the gray level light spot pattern.

According to the technical scheme provided by the embodiment of the invention, the optical fiber light spot touch sensor takes the light spot pattern as the sensing signal, and the light spot signal has high sensitivity. In the reflection type structure, the change of the bending state of the multimode optical fiber from the sensing probe to the camera has small influence on the output light spot characteristic, and is far smaller than the change of the light spot characteristic caused by the deformation of the reflector, so that the probe has stronger bending interference resistance.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 (a), (b) and (c) are structural diagrams of a fiber optic spot touch sensor based on a reflection type probe structure according to an embodiment of the present invention;

fig. 2 (a) is a structural diagram of an initial equivalent optical fiber according to an embodiment of the present invention, and fig. 2 (b) is a structural diagram of an equivalent optical fiber after being stressed according to an embodiment of the present invention;

fig. 3 (a) is a variation of ZNCC of an output light spot pattern of three MMF provided by the embodiment of the present invention with an inclination angle, fig. 3 (b) is a variation of a number of bright spots provided by the embodiment of the present invention with an increase of an inclination angle of an incident light of three MMF, and fig. 3 (c) is a light spot pattern output by an MMF3 provided by the embodiment of the present invention at different inclination angles;

FIG. 4 is a schematic diagram of a simulation model according to an embodiment of the present invention;

FIG. 5 (a) is a diagram of a change of ZNCC when a bending offset is changed and a speckle pattern output under different bending offsets, and FIG. 5 (b) is a schematic diagram of a change of ZNCC when a bending position is changed, according to an embodiment of the present invention;

FIG. 6 is a diagram of a contact force sensing experimental apparatus according to an embodiment of the present invention;

FIG. 7 is a schematic diagram showing the change of ZNCC of each model light spot pattern of the probe 1 with contact force according to the embodiment of the invention;

FIG. 8 is a schematic diagram illustrating the ZNCC of each model spot pattern of probe 2 as a function of contact force according to an embodiment of the present invention;

fig. 9 is a schematic diagram for comparing the sensing performance parameters obtained by the probe 1 and the probe 2 according to an embodiment of the present invention.

FIG. 10 is a diagram of a convolutional neural network architecture provided by an embodiment of the present invention;

fig. 11 (a) is a schematic diagram of a network model training process of an MMF2 optical spot pattern provided in an embodiment of the present invention, and fig. 11 (b) is a normalized confusion matrix for classification provided in an embodiment of the present invention (an inset is a typical optical spot pattern corresponding to 10 classifications);

fig. 12 (a) is a schematic diagram of a network model training process of an MMF2 speckle pattern according to an embodiment of the present invention, and fig. 12 (b) is a schematic diagram of a normalized confusion matrix for classification according to an embodiment of the present invention;

fig. 13 is a diagram of an inter-group-intra-group correlation according to an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary only for explaining the present invention and are not construed as limiting the present invention.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or coupled. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

For the convenience of understanding the embodiments of the present invention, the following description will be further explained by taking several specific embodiments as examples in conjunction with the drawings, and the embodiments are not to be construed as limiting the embodiments of the present invention.

The invention adopts the spot pattern as a signal, the camera as a detection Device, the type of the camera can be CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), and the whole system has simple structure and low cost. In the reflective structure, the transmission path of the light which is output from the single-mode fiber and reflected by the reflector and returns to the multimode fiber is easily affected by the outside, and the output light spot of the multimode fiber is correspondingly changed, so that the structure has high sensitivity.

The optical fiber touch sensor can be divided into two types of planar structures and probe structures, although the sensitive area of the planar structure is large, the sensitivity of the optical fiber touch sensor is relatively low, the application range of the optical fiber touch sensor is small, and the optical fiber touch sensor is mainly applied to bionic skin. The invention is a probe type structure, compared with a plane type structure, the invention has the advantages of high sensitivity, compact structure, wide application range and the like, and can be embedded into a plurality of structures which need to be subjected to touch force sensing, such as mechanical arms, bionic hands or scalpels, thereby having wide application in the fields of medical treatment and industrial intelligence.

The optical fiber tactile sensor with the probe type structure comprises a hemispherical contact element, a reflector and a base, wherein input and output optical fibers of the optical fiber tactile sensor are embedded in the base, a light source and a receiving camera of the sensor are positioned on the same side, and the sensing head is easy to install in a structure needing contact force sensing in a reflection type working mode.

An equivalent optical fiber structure is designed in the optical fiber facula touch sensor according to the geometrical optics principle, the angle between reflectors in a reflector is 90 degrees at the beginning, the light transmission in a sensing probe is equivalent to the light transmission in two optical fiber structures with centers aligned, when force is applied to the sensing probe at a contact point, the angle between the reflectors is increased, the equivalent optical fiber structure at the moment is an optical fiber incidence structure with a certain angle, the incident light of MMF has a certain inclination angle, and the influence of the contact force on the change of an MMF facula diagram is qualitatively analyzed by utilizing the equivalent optical fiber structure.

The probe structure of the optical fiber facula tactile sensor based on the reflection type probe structure is shown in fig. 1 (a), the light guide mode of the probe structure is a reflection mode, and the probe structure is made of silicone rubber and resin through 3D printing. The hemispherical contact and reflector in the probe are made of silicone rubber, which has a shore hardness less than the durometer of the resin plastic. The reflector consists of two mirrors placed at an angle of 90 deg., as shown in fig. 1 (b). At the bottom of the reflector, there is a cylindrical base made of resin plastic for fixing a Single-Mode Fiber (SMF) for transmitting and a Multi-Mode Fiber (MMF) for receiving to ensure that the two fibers are kept parallel. Light emitted by the semiconductor laser is incident on the SMF, reflected by a reflector in the sensing probe and coupled into the receiving MMF. When a precise contact force is applied to the sensing probe, the reflector in the sensing probe deforms, the angle between the two mirrors in the reflector changes, the transmission path of light from the SMF to the MMF changes, and the speckle pattern detected by the CCD changes accordingly. By analyzing the change of the light spot pattern, the change of the contact force can be known.

FIG. 1 (c) is a schematic view of 9 contact positions of the probe. In order to analyze the influence of the deformation of the reflector on the light spot of the optical fiber when the sensing probe is subjected to an acting force at the contact point 1, the reflector in the reflector is ideal, and an equivalent optical fiber structure is designed according to the geometrical optics principle. In this way, the light transmission in the sensing probe can be qualitatively analyzed by the simplified model in fig. 3. Initially the angle between the mirrors in the reflector is 90 deg., as shown in fig. 2 (a), the light transmission in the sensing probe is equivalent to the light transmission in two centrally aligned fiber structures. When a force is applied to the sensing probe at the contact point 1, the angle between the mirrors increases, as shown in fig. 2 (b), and the equivalent optical fiber structure is an optical fiber incident structure with a certain angle, which is equivalent to that, for the MMF, the incident light enters not perpendicular to the end face, but with a certain inclination angle. The transmission of the light path in the sensing probe is converted into an equivalent optical fiber structure, so that a simulation model is built conveniently, and the influence of contact force on the change of the MMF light spot diagram can be qualitatively analyzed by using the equivalent optical fiber structure.

The probe structure can realize touch sensing and position identification based on a single probe without the combination of a plurality of reflective element arrays, thereby reducing the complexity of the system structure, reducing the data volume of signal processing and improving the performance of touch sensing. The problem of integration of tip sensing is solved, and the robot arm and the like are favorably applied.

(2) The probe structure of the invention is combined, and the method adopting the light spot pattern as the processing signal comprises the following steps: this is because the speckle pattern is used as a processing signal for the reflective probe, which can achieve high touch force sensing sensitivity and strong resistance to bending interference.

The signal processing process of the optical fiber tactile sensor with the probe type structure comprises the following processing steps:

step 2, converting the cut light spot image into a gray image, wherein each pixel of the gray image only needs one byte for storing the gray value, so that strip distortion is avoided;

step 3, scaling the gray level image to a uniform size so as to obtain a ZNCC value

Step 4, calculating the ZNCC value of the gray level light spot pattern according to the formula (1);

in the aspect of perception performance, the invention has strong anti-bending interference capability and high touch perception precision, and has the perception capability of a single probe and multiple stress positions. The invention adopts the reflection type probe structure to weaken the influence caused by the bending of the optical fiber, and the change of the bending state of the optical fiber has very weak influence on the characteristics of output light spots.

In the aspect of light spot image processing, the correlation method is to compare output light spots of the optical fiber in different states by calculating spatial correlation among images, and measure the change of the light spots by the change of the correlation values, thereby quantifying the change of the optical fiber state. Compared with The conventional Cross-Correlation analysis, the Zero Mean Normalized Cross-Correlation (ZNCC) avoids The influence of brightness fluctuation of The facula images. As shown in equation (1):

wherein, I ₀ And I are the intensity distributions of the reference and current spot patterns respectively,

and

is the average intensity of the current spot pattern and the reference spot pattern. When the speckle pattern is not correlated with the reference map, ZNCC is attenuated.

The speckle pattern is preprocessed before its ZNCC value is calculated. The process of treatment is shown in figure 1: firstly, the light spot edge detection is carried out on the obtained original pattern, and external square cutting is carried out along the edge. Because the light spots occupy only a small area in the original image, if the original image is directly used, a large amount of redundant information in the image will increase the amount of calculation. In order to reduce redundancy and keep all spatial information of the light spots, an algorithm for automatically detecting the edges of the light spots and cutting the light spots into external squares along the edges is designed. Secondly, the cut light spot image is converted into a gray image, and each pixel of the gray image only needs one byte to store the gray value, so that the stripe distortion is avoided. And then scaling the facula pattern to be uniform in size, and calculating a ZNCC value of the facula pattern according to a formula (1).

The working principle of the optical fiber light spot touch sensor can be verified by simulation analysis of an equivalent optical fiber structure of the sensing probe:

(1) influence of tilt angle on output spot characteristics of multimode fiber

From the above analysis, when the sensing probe is acted by force at the contact point 1, the tilt angle of the MMF incident light in the equivalent optical fiber structure changes. Therefore, in the simulation process, the change of the contact force which the sensing probe receives in practice is simulated by changing the inclination angle in the equivalent optical fiber structure.

The light source wavelength in the simulation is 658nm, the MMF total length is kept at 3cm, and the air gap between the SMF and the MMF is 0.1mm. As bending inevitably occurs in experiments, small bending with the diameter of 1cm is introduced in the simulation, so that the simulation is closer to the actual situation. And simulating the optical fiber structures with different inclination angles by using a beam propagation method to obtain an optical fiber spot diagram, wherein the different inclination angles correspond to the situation when the sensing probe is stressed differently. Simulation analysis was performed on three MMFs, and the parameters of the optical fiber used for simulation and experiment were kept consistent, as shown in table 1.

TABLE 1 optical fiber parameter table for simulation and experiment

The simulation results are shown in fig. 4, and fig. 3 (a) shows the change of ZNCC of the output light spot patterns of three MMFs with the inclination angle. As can be seen from fig. 3 (a), the three groups of ZNCCs all decrease in a linear trend with the increase of the tilt angle, and the decreasing speed of ZNCCs is faster with the increase of the core diameter of the fiber core, because the MMF with the large core diameter excites more transmission modes, the generated speckle pattern is more complex, and the generated speckle pattern has a more sensitive response to the change of the tilt angle. Therefore, the number of bright spots of the simulated spot patterns of the three MMFs under different inclination angles is obtained.

Fig. 3 (b) shows the variation of the number of bright spots with increasing inclination angle of the incident light of the three MMFs. It can be seen that the number of bright spots increases as the tilt angle of the incident light increases. Since the guided mode number in MMF is generally close to the number of hot spots in the fiber optic spot pattern and inversely proportional to their average size, the variation in the number of hot spots further illustrates: the number of excited modes in the MMF increases with the tilt angle. Meanwhile, by comparing the three MMFs, the MMFs with large core diameters can be found to have more bright spots compared with the MMFs with small core diameters under the condition of the same inclination angle, which shows that the MMFs with large core diameters have more excitation modes under the same condition. As fig. 3 (c) shows the light spot diagram output by MMF3 under different tilt angles, it can be seen from the diagram that as the tilt angle increases, the average size of the light spots decreases and the number of the light spots increases, which further confirms the above theory.

The number of bright spots in the spot pattern can be calculated by the following steps. Firstly, carrying out binarization processing on a speckle pattern, and carrying out corrosion and expansion processing to extract boundary information of bright speckles; second, the boundary is smoothed by attenuating the stenotic portion and removing sharp, abrupt material. Then, indexing rows and columns of all the bright spots, and calculating the mass center of each bright spot; finally, the number of all the hot spot centroids in the hot spot image is calculated by adopting a cycle.

It should be noted that the equivalent fiber structure used in the simulation is only suitable for the case where a force is applied at the center of the sensing probe. When a force is applied to other locations of the sensing probe, the deformation of the reflector becomes complex and the transmission of light cannot be simulated with this simple equivalent structure. It is anticipated that in such a case, the effect of the large forces will still cause the reflector to deform significantly, and thus the fiber optic spot pattern will change even more.

(2) Effect of bend offset and bend position on fiber output spot characteristics

In order to study the influence of the bending of the optical fiber on the spot characteristics output by the reflective structure, the influence of the bending deviation and the bending position on the spot characteristics output by the equivalent optical fiber structure is analyzed from two angles. FIG. 4 is a simulated structural model, in which the vertical distance d between the highest point of the bent portion of the optical fiber and the straight portion of the optical fiber is the bending offset, wherein the distance between the points A and B is always kept constant, and the bending length increases with the increase of the offset.

The core diameter of MMF is 50 μm, the simulation of output light spots is carried out on the optical fiber structures with different inclination angles at 4 bending offsets and 3 bending positions respectively, and ZNCC of each group of light spots is calculated. Wherein, when the bending offset amount is changed, the bending position is kept at 2; the amount of bend deflection was maintained at 0.2mm as the bend position was changed.

When the bending offset is increased from 0.1mm to 0.4mm, the relation between ZNCC of the spot diagram and the inclination angle is as shown in fig. 5 (a), and the result shows that, in this range, the change of the bending offset does not affect the overall change trend of ZNCC, and neither the descending speed nor the linear range is affected. It is explained that within a certain bending offset range, the change of the bending offset does not have an influence on the relevant characteristics of the light spot, but under the same inclination angle, the change of the bending offset has an influence on the output light spot pattern, as shown in fig. 5 (a). Under the same bending offset, the output spot image bright spots are more and more in number and the average size of the bright spots is gradually reduced along with the increase of the inclination angle. Under the condition of the same inclination angle, the larger the bending offset is, the more complicated the output light spot pattern is, and the more the number of the light spots is. When the bending position is changed, no influence is caused on the change trend of the ZNCC, and as can be seen from FIG. 5 (b), three groups of ZNCCs keep perfect consistency when the bending position is changed. Simulation results show that the change of bending offset and bending position has very weak influence on the characteristics of light spots output by the optical fiber structures with different inclination angles, which indicates that the structure has strong bending interference resistance.

In terms of experimental analysis of the tactile sensor:

(1) contact force sensing experiment

After the simulation analysis, the contact force sensing experiment is performed on the touch sensors with two different probe hardnesses, and the experimental apparatus is shown in fig. 6. After light output by the laser enters the SMF, the light is reflected by a reflector of the sensing probe to enter the MMF, and light beams output by the MMF are collimated by the objective lens and then received by the CCD camera (DFK 33G 445). The camera provides an image of 1280 × 960 pixels, with a pixel size of 3.75 × 3.75 μm. Three MMFs and two lasers with the wavelengths of 658nm and 532nm are respectively used as light sources for experiments, so that the influence of the structural parameters of the sensor on the performance of the sensor is analyzed.

1) The probe 1: hemispherical contact-reflector shore hardness 60-80

At two operating wavelengths, three MMFs of different lengths were named models 1-5, respectively, as shown in table 2, which also summarizes the sensitivity and linearity measurement range of the different models.

TABLE 2 results performance of different models of Probe 1

Overall, MMF3 achieved higher sensitivity than the other two MMFs, and 0.2N was obtained over a 0-4N linear measurement range at a wavelength of 532nm ^-1 The highest sensitivity of (c). This is because the MMF3 has a large diameter and produces many transmission modes, resulting in a sensitive speckle pattern. For MMF1 and MMF3, it can be seen from a comparison of

models

1 and 2 or a comparison of

models

4 and 5 that MMF length does not have much influence on sensitivity. Comparing the drop velocity of ZNCC with increasing contact force at two wavelengths in each model, the smaller wavelength of 532nm is a beneficial factor in improving sensitivity. The reason is that at smaller operating wavelengths, there are many modes excited in MMF.

The change in ZNCC for each model spot pattern with contact force is shown in fig. 7. The statistics of triplicate measurements for each model are shown in the figure. Filled squares or circles represent the mean of ZNCC and bars represent standard deviation. A small standard deviation of the measurements in all cases indicates good reproducibility of the experiment. From fig. 7 it can be concluded that as the contact force increases linearly, both ZNCC of the fiber spot pattern decrease, indicating that the contact force can be uniquely determined by the ZNCC of the fiber spot pattern.

2) And (3) probe 2: hemispherical contact-reflector shore hardness of 80-60

The probe reflector hardness was adjusted down to 60 and the hemisphere hardness increased to 80, and the contact force sensing experiment described above was repeated, with the sensitivity and linear range for each model shown in table 3. As can be seen from the table, as the core diameter of the optical fiber increases, the sensitivity of the sensor also increases; when the wavelength of the light source is shorter, the sensitivity of the sensor is higher; the fiber length does not have much effect on the sensitivity, but when the fiber length is shorter, the linear measurement range is slightly increased, probably because the fiber length is short and the optical loss is small. The experimental results of the probe are consistent with the experimental conclusions of the probe 1, which also confirms the reliability of the sensor.

TABLE 3 results performance of different models of Probe 2

The change of ZNCC with contact force for each model light spot pattern is shown in FIG. 9, and ZNCC is linearly related to contact force. As can be seen from each graph, when the wavelength is shorter, the speed of ZNCC of the light spot is faster along with the increase of the contact force, which shows that the short wavelength is favorable for improving the sensitivity of the sensor; from FIG. 8, it can be seen that when the length of the optical fiber is changed, the ZNCC descent speed is not greatly influenced, and the linear range is increased as the length of the optical fiber is reduced; from fig. 8, it can be seen that the larger the core diameter of the optical fiber is, the faster the ZNCC of the optical spot pattern decreases with the contact force is, wherein the ZNCC of the MMF3 output optical spot with the core diameter of 200 μm decreases at the fastest speed, and the sensor has the highest sensitivity, and as can be seen from the optical spot subgraph, the number of bright spots in the MMF3 output pattern is large, and the optical spot pattern is complex. This is consistent with the results obtained from the simulation.

The experimental results of the probe 1 and the probe 2 show that the ZNCC of the light spot pattern linearly decreases along with the increase of the contact force, and the probe structure can realize the contact force sensing. The sensing performance achieved when the structural parameters of the probe are different is also different. When the wavelength of the light source is 532nm and the length of the optical fiber is 32.5cm, the sensing performance parameters obtained by the probe 1 and the probe 2 are compared, and the comparison result is shown in fig. 10. Wherein, fig. 9 (a) is a comparison of sensitivity, the black column is the probe 1, and the gray column is the probe 2, it can be found through the comparison that the probe 2 realizes higher sensitivity than the probe 1, because the hardness of the reflector of the probe 2 is relatively soft, when the probe is acted by a force, the deformation amount of the reflector is relatively large, which causes the change of the spot diagram to be obvious, and the change of the ZNCC of the spot diagram to be large, so that higher sensitivity can be provided. As can also be seen from fig. 9 (a), the larger the fiber core diameter, the greater the sensitivity achieved by the sensor.

Fig. 9 (b) is a comparison of the linear dynamic range of the two probes, from which it can be seen that the linear range achieved by the probe 2 achieving the higher sensitivity is lower, and the lower sensitivity probe 1 achieves a larger linear measurement range. The comparison shows that different sensing performances can be realized by different probe structure parameters, and the sensor probe structure parameters can be adjusted according to different application requirements so as to realize sensors with different performances.

After the touch sensor is verified to be capable of realizing touch force sensing, the contact position identification capability of the sensor is verified by adopting a convolutional neural network CNN. CNN is a deep learning architecture that is unique in that it can recognize non-linear relationships in data. Therefore, the method is widely used for recognizing complex visual patterns and achieves excellent performance. Furthermore, CNN shows better performance in speckle pattern based sensing schemes compared to other models.

The invention adopts a Network architecture of VGG-Nets (VGG: visual Geometry Group Network). As shown in fig. 10, the convolutional neural network structure is mainly composed of 6 convolutional layers, 3 pooling layers, and a global average pooling layer. The experimentally collected speckle pattern was adjusted to 64 x 64 before being put into the neural network. The first convolution was performed using 64 convolution kernels of size 3 × 3, and the feature map parameters obtained after the convolution were 64 × 64 × 64. Where 64 × 64 is the size of each image after convolution, and the last '64' is the number of feature images after convolution. To avoid the problems of gradient explosion and gradient disappearance, an activation function is added after each convolution operation. The activation function used by the network is the RELU function, defined as RELU = max (0,x).

The second convolution operates the same as the first convolution. Then, the convolved feature maps are input to a pooling layer for feature extraction, and after pooling by 2 × 2, the size of the feature maps becomes 32 × 32.Conv3 and Conv4 have 128 convolution kernels, with kernel size of 3 × 3. Similarly, after convolution of Conv3 and Conv4 and 2 × 2 pooling of maxporoling 2, the size of the feature map becomes 16 × 16, the number of feature maps becomes 128, and then the feature map output by Conv4 is input to Conv5 having 64 convolution kernels and then to Conv6, and Conv6 has 10 convolution kernels, corresponding to the number of output channels, i.e., input classes. The size of the feature map after 2 × 2 pooling becomes 8 × 8. A Global averaging pool (Global averaging pool) can merge the feature map of the last layer with the average of the whole image to form a feature point, which can be used to form the final feature vector for calculation in SoftMax.

Dividing an optical fiber light spot data set acquired by an experiment into a training set and a test set according to a ratio of 4:

TABLE 4 training set and test set partitioning

To verify the ability of the sensor to identify the location of the touch, a force of 2.9N and 3.4N was applied at a light source wavelength of 532nm at five locations on the model 3 of the probe 1, shown as

points

1, 3, 5, 7, 9 in FIG. 1 (a). Once the magnitude and position of the applied force changes, the transmission path from the SMF to the MMF in the reflector will change, and then the mode coupling of the MMF will cause the speckle pattern to change. Thus, by analysis of the pattern of spots, the position of the contact force can be determined.

For each force applied at each position, 200 spot images were collected, for a total of 2000 spot patterns, divided into training and test sets on a scale of 4. As can be seen from fig. 11 (a), the accuracy and loss curves reach steady state very quickly with the existing network depth. The precision difference between the training set and the test set is negligible, which indicates that the network has good generalization capability. Fig. 11 (b) shows 100% accuracy of the touch location classification, which verifies that the sensor has good touch location recognition capability. In fig. 11 (b), a typical speckle pattern for 10 classifications obtained when two forces are applied at five locations of the sensing probe is also shown.

In order to further verify the recognition capability of the sensor on the contact position, when the light source is 532nm, model 4 of probe 2 is adopted to recognize 3 contact forces of 9 contact positions, and 27 contact conditions are recognized, wherein 250 spot images are collected in each condition, 6750 spots are collected in total, and the contact positions are recognized according to the following conditions that 4: the ratio of 1 is divided into a training set and a test set. The network training results and classification results are shown in fig. 4-12. The results of fig. 12 (a) also verify that the network has good generalization capability. Fig. 12 (b) shows that the classification accuracy reached 100% for each of the 27 cases. Further illustrating the accurate contact location of the touch sensor.

When contact force is applied to different positions of the probe, the reflectors deform differently, so that optical transmission paths are different, and the difference of output optical spot patterns is more obvious. FIG. 13 is an inter-group-intra-group correlation diagram according to an embodiment of the present invention, as shown in FIG. 13, when the same magnitude of contact force is applied at the same position, the correlation of the light spots is better, and the ZNCC value is close to 1; when the same magnitude of contact force is applied at different positions, the correlation of the faculae is poor, and the ZNCC value is close to 0. A classification accuracy of 100% can be achieved.

The feasibility of the touch sensor is verified through the verification of a contact force sensing experiment and a contact position identification experiment. The sensing scheme was verified using 2 hardness probes and different MMFs, and the sensitivity to force was tested at two operating wavelengths, from which good repeatability was seen. Through deep learning, the contact positions are successfully classified according to the light spot patterns, and the accuracy rate is 100%. The sensing structure in this solution is simple and compact, so it is easy to miniaturize and integrate it into an array. The optical-based sensing principle also provides immunity against electromagnetic interference, which is advantageous for its further application in harsh environments where strong electromagnetic interference may occur, such as power plants, power plants and magnetic resonance imaging environments.

In summary, compared with the conventional reflective probe structure, the embodiment of the invention uses the spot pattern as the sensing signal, and the spot signal has high sensitivity. In the reflection type structure, the change of the bending state of the multimode fiber from the probe to the camera has little influence on the output light spot characteristic, and is far less than the change of the light spot characteristic caused by the deformation of the reflector, so that the probe has stronger anti-bending interference capability; meanwhile, the structure of the touch sensing probe is exquisite, position identification is carried out after a plurality of elements are arranged and combined, and touch sensing (position and size of contact force) can be accurately realized by only one unit.

The probe has a delicate structure, the reflective structure is beneficial to tip integration, and the application range is wider, such as medical instruments, detection of narrow spaces, robot limbs, bionic hands and the like. Has higher application value.

Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.

From the above description of the embodiments, it is clear to those skilled in the art that the present invention can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for apparatus or system embodiments, since they are substantially similar to method embodiments, they are described in relative terms, as long as they are described in partial descriptions of method embodiments. The above-described embodiments of the apparatus and system are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A fiber optic spot touch sensor based on a reflective probe structure, comprising: the optical fiber comprises a hemispherical contact part, a reflector, a base and two optical fibers embedded in the base, wherein the two optical fibers are respectively an input optical fiber and an output optical fiber of a probe, the light guide mode of the optical fibers is a reflection mode, the input optical fiber is a single-mode optical fiber, an optical signal from a light source is guided into the probe and irradiated on the reflector, the output optical fiber is a multi-mode optical fiber, the light reflected by the probe reflector is transmitted to a receiving camera, the camera collects a light spot signal, the light source and the camera receiving the light spot signal are positioned on the same side, the probe structure is made of silicone rubber and resin through 3D printing, the hemispherical contact part and the reflector are made of silicone rubber, the base is made of resin plastic, the Shore hardness of the silicone rubber is smaller than that of the resin plastic, the hardness of the hemispherical contact part is higher than that of the reflector, and the force applied to the probe contact part can cause the deformation of the reflector.

2. The fiber spot touch sensor according to claim 1, wherein the reflector of the sensing probe comprises two reflectors disposed at an angle of 90 °, and at the bottom of the reflector, there is a cylindrical base made of resin plastic for fixing the input single mode fiber and the output multimode fiber of the probe, keeping the input single mode fiber and the output multimode fiber of the probe parallel to each other.

3. The optical fiber spot touch sensor based on the reflective probe structure of claim 1, wherein the light output from the light source is guided to the reflector after entering the probe base through the single mode optical fiber, and the transmission direction of the light beam is changed by approximately 180 degrees by reflection of two vertical reflectors in the reflector, so as to enter the output multimode optical fiber; when a contact force is applied to the sensing probe, the reflector in the probe deforms, the angle between two originally vertical reflectors in the reflector changes, so that the transmission path of light from the single-mode optical fiber to the multimode optical fiber changes, the spot pattern detected by the camera also changes, the deformation of the reflectors gradually increases along with the increase of the contact force, the change of the transmission path of the light increases, the change of the spot pattern collected by the camera increases, and the change information of the contact force can be acquired by analyzing the change degree of the spot pattern.

4. A method according to any one of claims 1 to 3, characterized in that the signal processing of the optical fiber tactile sensor of probe-type structure comprises the following processing steps:

step 2, converting the cut light spot image into a gray light spot image;

step 3, scaling the gray level light spot image to a uniform size;