CN111721451B - Image-based touch sensing method, miniaturized device and device manufacturing method - Google Patents
Image-based touch sensing method, miniaturized device and device manufacturing method Download PDFInfo
- Publication number
- CN111721451B CN111721451B CN202010568984.9A CN202010568984A CN111721451B CN 111721451 B CN111721451 B CN 111721451B CN 202010568984 A CN202010568984 A CN 202010568984A CN 111721451 B CN111721451 B CN 111721451B
- Authority
- CN
- China
- Prior art keywords
- micro
- flexible transparent
- light
- image
- column
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000004519 manufacturing process Methods 0.000 title claims description 16
- 239000000758 substrate Substances 0.000 claims abstract description 24
- 238000002360 preparation method Methods 0.000 claims abstract description 15
- 230000009471 action Effects 0.000 claims abstract description 11
- 238000012549 training Methods 0.000 claims abstract description 11
- 239000000463 material Substances 0.000 claims description 89
- 230000000903 blocking effect Effects 0.000 claims description 55
- 238000011161 development Methods 0.000 claims description 35
- 239000011248 coating agent Substances 0.000 claims description 34
- 238000000576 coating method Methods 0.000 claims description 34
- 239000007788 liquid Substances 0.000 claims description 34
- 239000011521 glass Substances 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 13
- 238000010438 heat treatment Methods 0.000 claims description 12
- 238000002156 mixing Methods 0.000 claims description 9
- 239000000203 mixture Substances 0.000 claims description 9
- 238000004528 spin coating Methods 0.000 claims description 9
- 238000007664 blowing Methods 0.000 claims description 8
- 229920002120 photoresistant polymer Polymers 0.000 claims description 6
- 229920002545 silicone oil Polymers 0.000 claims description 6
- 238000000465 moulding Methods 0.000 claims description 4
- 230000002441 reversible effect Effects 0.000 claims description 4
- 238000005452 bending Methods 0.000 abstract description 13
- 230000010354 integration Effects 0.000 abstract description 11
- 230000008859 change Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 13
- 238000005259 measurement Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 8
- 238000001514 detection method Methods 0.000 description 7
- 239000004973 liquid crystal related substance Substances 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 239000004205 dimethyl polysiloxane Substances 0.000 description 5
- 235000013870 dimethyl polysiloxane Nutrition 0.000 description 5
- CXQXSVUQTKDNFP-UHFFFAOYSA-N octamethyltrisiloxane Chemical compound C[Si](C)(C)O[Si](C)(C)O[Si](C)(C)C CXQXSVUQTKDNFP-UHFFFAOYSA-N 0.000 description 5
- 238000004987 plasma desorption mass spectroscopy Methods 0.000 description 5
- 229920000435 poly(dimethylsiloxane) Polymers 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000035945 sensitivity Effects 0.000 description 4
- 239000003086 colorant Substances 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 238000009529 body temperature measurement Methods 0.000 description 2
- 239000003990 capacitor Substances 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 239000012528 membrane Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000036961 partial effect Effects 0.000 description 2
- 230000000149 penetrating effect Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000013329 compounding Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003071 parasitic effect Effects 0.000 description 1
- 239000000049 pigment Substances 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000003756 stirring Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01L—MEASURING FORCE, STRESS, TORQUE, WORK, MECHANICAL POWER, MECHANICAL EFFICIENCY, OR FLUID PRESSURE
- G01L1/00—Measuring force or stress, in general
- G01L1/24—Measuring force or stress, in general by measuring variations of optical properties of material when it is stressed, e.g. by photoelastic stress analysis using infrared, visible light, ultraviolet
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/16—Measuring arrangements characterised by the use of optical techniques for measuring the deformation in a solid, e.g. optical strain gauge
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01L—MEASURING FORCE, STRESS, TORQUE, WORK, MECHANICAL POWER, MECHANICAL EFFICIENCY, OR FLUID PRESSURE
- G01L5/00—Apparatus for, or methods of, measuring force, work, mechanical power, or torque, specially adapted for specific purposes
- G01L5/16—Apparatus for, or methods of, measuring force, work, mechanical power, or torque, specially adapted for specific purposes for measuring several components of force
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
Abstract
The embodiment of the invention provides a touch sensing method based on an image, a miniaturized device and a device preparation method, wherein the method comprises the following steps: projecting light rays to the top direction of the microcolumn by using a light emitting source, wherein the light rays are emitted through the microcolumn and the substrate; the microcolumns are positioned on the substrate to form a flexible transparent microcolumn array; receiving the emitted light by using an image sensor arranged at the bottom of the flexible transparent micro-column array to generate a micro-column deformation image; inputting the microcolumn deformation image into a decoupling network model, and outputting three-dimensional sensing data of an applied three-dimensional force; the decoupling network model is obtained after training based on a sample light spot image of the flexible transparent micro-column array under the action of standard pressure and a corresponding identification label. According to the embodiment of the invention, the light source is integrated in the pressure sensor, so that the bending and deformation of the microcolumn can be clearly captured, the decoupling of the three-dimensional force is completed by utilizing an intelligent algorithm based on the deformation image, and the miniaturization, high integration and precision of the sensor are realized.
Description
Technical Field
The invention relates to the technical field of intelligent sensing and the field of flexible touch sensors, in particular to a touch sensing method based on an image, a miniaturization device and a device manufacturing method.
Background
Nowadays, the demand of touch perception in the field of robot application is increasing, and when the robot can accurately complete various fine and high-difficulty works, the robot needs to be capable of detecting spatial multi-dimensional force, an integrated and miniaturized touch sensing method is needed as support, and the robot can have flexibility and high integration like a real human hand while giving consideration to three-dimensional force measurement. The research on the tactile sensing method is imminent.
Currently, the touch sensing method is mainly studied based on strain gauge, piezoelectric, capacitive and piezoresistive methods. Among them, strain gauge sensors are generally poor in flexibility and not suitable for flexible skin of robots. The piezoelectric sensor works stably and responds sensitively to external force, but is only suitable for measuring limited dynamic force due to large internal resistance, and can hardly measure static force, so that certain difficulty is brought to the decomposition of three-dimensional force. The capacitance type sensor is limited by the volume, the sensing capacitance is often small, the measurement is easily interfered by the parasitic capacitance, and the accurate measurement circuit is complex, so that the application of the capacitance type sensor in practice is greatly limited. The resistive sensor has relatively difficult decoupling due to more internal cross points, and the lead of the array sensor is complex, so that the application of the resistive sensor is limited.
Compared with electronic components such as capacitors and resistors, the image-based touch sensing method can effectively reduce the volume of a measuring unit and the number of connecting lines of the sensor, can better realize the miniaturization and high integration of the sensor, has high detection sensitivity on lateral force, can accurately identify three-dimensional force, can measure temperature, and has profound significance for realizing artificial touch. However, the existing image-based touch sensing method has poor imaging effect, so that the measurement accuracy is not high, and the method is easily limited by application environment.
In view of this, it is important to provide a flexible three-dimensional force detection method and device that are compact, have high detection sensitivity, are less limited by the application environment, and are convenient to decouple, which is of far reaching significance in realizing artificial touch.
Disclosure of Invention
In order to solve the above problems, embodiments of the present invention provide an image-based haptic sensing method, a miniaturized device, and a device manufacturing method that overcome the above problems or at least partially solve the above problems.
In a first aspect, an embodiment of the present invention provides an image-based haptic sensing method, including: projecting light rays to the top direction of the microcolumn by using a light emitting source, wherein the light rays are emitted through the microcolumn and the substrate; the microcolumns are positioned on the substrate to form a flexible transparent microcolumn array; receiving the emitted light by using an image sensor arranged at the bottom of the flexible transparent micro-column array to generate a micro-column deformation image; inputting the micro-column deformation image into a decoupling network model, and outputting three-dimensional sensing data of three-dimensional force applied to the flexible transparent micro-column array; the decoupling network model is obtained after training based on a sample light spot image of the flexible transparent micro-column array under the action of standard pressure and a corresponding identification label.
Optionally, before projecting light to the tip direction of the microcolumn by using the light emitting source, the method further includes: setting the luminous source as a luminous film; spin-coating a light-emitting film on the top of the microcolumn; coating the light blocking material layer on the bottom of the flexible transparent micro-column array and other parts of the micro-columns except the top parts.
Optionally, before projecting light to the top direction of the microcolumn by using the light emitting source, the method may further include: setting the luminous source as a luminous film; coating a light blocking material layer on the top end of each micro-column; and spin-coating the light-emitting film on the light blocking material layer.
Alternatively, the microcolumn is a widened microcolumn having a diameter of 500 μm to 1 cm.
Optionally, before the projecting the light to the tip direction of the microcolumn by the light emitting source, the method may further include: arranging the light emitting source as a fluorescent coating; coating a fluorescent coating on the top end of each micro-column; coating a light blocking material layer on the other parts of the flexible transparent micro-column array except the bottom.
Optionally, before projecting light to the top direction of the microcolumn by using the light emitting source, the method may further include: coating a light blocking material layer on the top end of each micro-column; a grating layer is arranged between the light-emitting source and the light blocking material layer and is used for enabling light rays to vertically enter the top end of the micro-column; the light-emitting source is provided as a light-emitting film or a fluorescent coating.
Optionally, a variable color development layer is additionally arranged on the inner side of the light-blocking material layer close to the microcolumns.
In a second aspect, embodiments of the present invention provide an image-based miniaturized tactile sensing device, including but not limited to: the device comprises a light emitting source, a flexible transparent micro-column array, an image sensor and an image processing unit; the light source is used for projecting light rays to the top end direction of the microcolumn, and the light rays are emitted out through the microcolumn and the substrate; the micro-columns are positioned on the substrate and integrally formed to form a flexible transparent micro-column array; the image sensor is arranged at the bottom of the flexible transparent micro-column array and used for receiving the emitted light and generating a micro-column deformation image; the image processing unit is prestored with a decoupling network model and is used for inputting the microcolumn deformation image into the decoupling network model and outputting three-dimensional sensing data of three-dimensional force applied to the flexible transparent microcolumn array; the decoupling network model is obtained after training based on a sample light spot image of the flexible transparent micro-column array under the action of standard pressure and a corresponding identification label.
In a third aspect, embodiments of the present invention provide a method for manufacturing a miniaturized image-based tactile sensing device, including, but not limited to, the following steps: the method comprises the steps of preparing a flexible transparent micro-column array, coating a variable color development layer on the flexible transparent micro-column array, and coating a light blocking material layer on the variable color development layer.
Wherein, preparing the flexible transparent micro-column array comprises: spin coating photoresist on the inner surface of the preparation container; exposing and developing the preparation container; arranging a preset pattern of the flexible transparent micro-column array on the photoresist on the inner surface of the preparation container to manufacture a mould of the flexible transparent micro-column array; uniformly mixing prepolymer for manufacturing the flexible transparent micro-pillars in the flexible transparent micro-pillar array with silicone oil according to a first set proportion, pouring the mixture on a mold, heating and curing the mixture, and performing reverse molding to manufacture a flexible transparent micro-pillar array;
coating a variable color development layer on a flexible transparent microcolumn array, comprising: bonding a glass sheet on the top end of the microcolumn; uniformly mixing the liquid variable color development material and the prepolymer in a second set proportion to generate color development material mixed liquid; injecting the color development material mixed liquid into gaps among the micro-columns of the flexible transparent micro-column array, and blowing out redundant color development material mixed liquid by using gas; heating to solidify the color developing material mixed liquid, and taking down the glass sheet;
coating a light-blocking material layer on a variable color developing layer includes: bonding a glass sheet on the top end of the microcolumn again; uniformly mixing the liquid light blocking material and the prepolymer according to a third set proportion to generate light blocking material mixed liquid; injecting the light-blocking material mixed liquid into gaps among the micro-columns of the flexible transparent micro-column array, and blowing out redundant light-blocking material mixed liquid by using gas; and heating to solidify the light blocking material mixed liquid, and taking down the glass sheet.
According to the image-based touch sensing method, the image-based touch device and the device manufacturing method, the light-emitting source is integrated in the pressure sensor, so that the bending and deformation of the microcolumn can be clearly captured, the decoupling of the three-dimensional force is completed by using an intelligent algorithm based on the deformation image, and the miniaturization, high integration and precision of the sensor are realized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from these without inventive effort.
FIG. 1 is a schematic flow chart of a method for image-based haptic sensing provided by an embodiment of the present invention;
FIG. 2 is a schematic diagram of an image-based miniaturized touch sensing device and its operation according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a decoupling method for measuring three-dimensional forces in an image-based haptic sensing method according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of another miniaturized image-based touch sensing device and its operation according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of another miniaturized image-based touch sensing device and its operation according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of another miniaturized image-based touch sensing device and its operation according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a model of an image-based miniaturized touch sensing device according to an embodiment of the present invention;
FIG. 8 is a process flow diagram for the fabrication of an image-based miniaturized touch sensing device according to an embodiment of the present invention;
fig. 9 is a process flow diagram for preparing a flexible transparent micro-pillar array in an image-based miniaturized touch sensing device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some embodiments, but not all embodiments, of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
An embodiment of the present invention provides an image-based haptic sensing method, as shown in fig. 1, including, but not limited to, the following steps:
step S1, projecting light to the top direction of the microcolumn by the light source, the light being emitted through the microcolumn and the substrate; the microcolumns are positioned on the substrate to form a flexible transparent microcolumn array;
step S2, receiving the emitted light by using an image sensor arranged at the bottom of the flexible transparent micro-column array to generate a micro-column deformation image;
step S3, inputting the microcolumn deformation image into a decoupling network model, and outputting three-dimensional sensing data of three-dimensional force applied to the flexible transparent microcolumn array; the decoupling network model is obtained after training based on a sample light spot image of the flexible transparent micro-column array subjected to standard three-dimensional force and a corresponding identification label.
Specifically, according to the image-based touch sensing method provided by the embodiment of the invention, the characteristic analysis is performed on the microcolumn deformation image by using the decoupling network model trained in advance according to the microcolumn deformation image generated by the flexible transparent microcolumn array under the action of the three-dimensional force exerted on the flexible transparent microcolumn array, so that the three-dimensional sensing data of the three-dimensional force is obtained.
Because the acquisition of the microcolumn deformation image is interfered by external environment, especially in a dark working environment, the image which can clearly reflect the microcolumn deformation cannot be acquired, and even the microcolumn deformation image cannot work normally at all. Therefore, in the image-based tactile sensing method according to the embodiment of the present invention, the light source is added and used to project light from a direction perpendicular to the plane of the top of the micro-column. Under the condition that the whole light path from the top end of the micro-column to the substrate is not shielded, the light can form a complete bright spot image at the bottom of the flexible transparent micro-column array; under the condition that partial shielding exists on the whole light path from the top end of the micro-column to the substrate, a incomplete bright spot image is formed at the bottom of the flexible transparent micro-column array; and if the whole light path from the top end of the micro-column to the substrate is completely blocked, forming a shadow image at the bottom of the flexible transparent micro-column array.
Further, an image sensor (such as a CCD or a COMS) arranged at the bottom of the flexible transparent micro-column array is used to acquire image information of the transmitted light, and a micro-column deformation image after the external three-dimensional force of the flexible transparent micro-column array is applied is generated according to the acquired image.
And finally, inputting the microcolumn deformation image into a decoupling network model trained in advance to decouple the three-dimensional force, and acquiring three-dimensional sensing data of the three-dimensional force.
Alternatively, in the embodiment of the present invention, it may be determined whether the formed micro-column deformation image is implemented according to a bright spot image or a shadow image by selecting whether to add a light blocking material layer on the top end of the micro-column.
If the light blocking material layer is not arranged at the top end of the micro-column, light can penetrate through the flexible transparent micro-column and the base to generate a corresponding bright spot image, and at the moment, the bright spot images penetrating through all the micro-columns are collected at the bottom of the flexible transparent micro-column array to construct a micro-column deformation image.
Correspondingly, if the light blocking material layer is arranged at the top end of the micro-column, light cannot pass through the top end of the micro-column and cannot be scattered completely, a shadow is projected below the micro-column to form a shadow image, and at the moment, the shadow image of the light which penetrates through other parts except the top end of the micro-column is collected at the bottom of the flexible transparent micro-column array to construct a micro-column deformation image.
As shown in fig. 2, the flexible transparent micro-column array may include a plurality of flexible transparent micro-columns, and all the flexible transparent micro-columns are arranged in an array according to actual application requirements to form the flexible transparent micro-column array.
Wherein, the shape of the micro-column is not limited to a round micro-column, a triangular micro-column, a polygonal micro-column, etc.; the diameter of the microcolumn may be set from 1 μm to several cm or the like; the height of the microcolumns can be adjusted according to the diameter of the microcolumns, generally 0.1 to 10 times of the height of the microcolumns, and the spacing between the microcolumns is more than 0.1 times of the height of the microcolumns.
The light source may be a flexible film capable of emitting light, such as a noctilucent film, an LED film, or a fluorescent coating, which is not specifically limited in this embodiment.
When part of the flexible transparent micro-pillars in the flexible transparent micro-pillar array are deformed under the action of three-dimensional force, the obtained micro-pillar deformation image is correspondingly changed.
In the method of the embodiment, the flexible transparent microcolumn is deformed correspondingly under the action of pressure, specifically, when the flexible transparent microcolumn is acted by a horizontal force, the flexible transparent microcolumn is bent in the direction of the horizontal force, and the larger the horizontal force is, the larger the bending amplitude is; when the flexible transparent micro-column is acted by a vertical force, the vertical projection radius of the flexible transparent micro-column is correspondingly increased, and the vertical projection radius is larger when the vertical force is larger.
Further, within the elastic range of the flexible transparent micro-column, the magnitude of the horizontal force is linearly proportional to the bending degree of the flexible transparent micro-column; the vertical force is linearly proportional to the change of the vertical projection radius of the flexible transparent micro-column. If a horizontal force and a vertical force are simultaneously applied to the flexible transparent microcolumn, the flexible transparent microcolumn may be simultaneously bent and the vertical projection radius may be increased. The embodiment of the invention measures the three-dimensional force based on the deformation of the flexible transparent micro-column under stress.
By acquiring the micro-column deformation image of the flexible transparent micro-column array subjected to the three-dimensional force, the embodiment of the invention does not make specific limitations on how to acquire the deformation image and the device for acquiring the deformation image.
And after acquiring a micro-column deformation image of the flexible transparent micro-column array subjected to three-dimensional force, inputting the acquired deformation image into a trained decoupling network model, analyzing and processing the acquired image based on the decoupling network model, and outputting sensing data of the three-dimensional force applied to the flexible transparent micro-column array.
Optionally, the deformation image is input to the decoupling network model, and before the sensing data of the three-dimensional force is output, that is, after step S2 is executed, the deformation image of the flexible transparent micro-column array under the action of the three-dimensional force is acquired, and then the deformation image is subjected to a sharpening process by using a focusing algorithm.
Specifically, after a three-dimensional force is applied to the flexible transparent micro-column array, a deformation image of the flexible transparent micro-column array when the three-dimensional force is applied may appear blurred due to weak defocusing. According to the image-based touch sensing method provided by the embodiment of the invention, the acquired deformation image can be subjected to sharpening processing by adopting a focusing algorithm. The method specifically comprises the following steps: and (3) adopting a defocusing blurred image restoration algorithm, such as: NAS-RIF and regular constraint based methods, etc.; intelligent zooming and other intelligent algorithms can also be adopted, the adopted focusing algorithm is not limited by the embodiment of the invention,
optionally, before the deformation image is input into the decoupling network model, applying a standard three-dimensional force to the flexible transparent micro-column array, and obtaining a sample deformation image of the flexible transparent micro-column array subjected to the standard three-dimensional force; inputting the sample deformation image into a pre-training decoupling network model, extracting deformation and bending characteristics of the flexible transparent micro-column array, corresponding the deformation and bending characteristics to sensing data of standard three-dimensional force, and establishing an identification label; changing the direction and the size of the standard three-dimensional force for multiple times, and acquiring a deformation image of the flexible transparent micro-column array after each change; corresponding each different standard three-dimensional force to the corresponding deformation image one by one, completing the training of the sample deformation image and the corresponding identification label after the flexible transparent micro-column array is acted by the standard three-dimensional force, and obtaining a decoupling network model; the deformation image comprises deformation and bending characteristics of the flexible transparent micro-column array.
As shown in fig. 3, an embodiment of the present invention provides an image-based haptic sensing method, including, but not limited to, the following steps:
the method comprises the following steps of obtaining a deformation image of the flexible transparent micro-column array after being applied with three-dimensional force through shooting by an image sensor, inputting the deformation image into a computer, and executing the following processing on the deformation image by the computer: firstly, performing the sharpening processing of a defocusing fuzzy image on an input deformation image, further extracting the deformation quantity, the bending rate and other characteristics of a flexible transparent micro-column array in the sharpened deformation image, and then decoupling the applied three-dimensional force according to the extracted deformation quantity, bending rate and other characteristics of the flexible transparent micro-column array; and finally, outputting and displaying the perception data of the three-dimensional force.
Further, by applying a standard three-dimensional force to the flexible transparent micro-column array, the three-dimensional force sensing data output by the computer is obtained by the method, and the output three-dimensional force sensing data is compared with the standard three-dimensional force for analysis, so that the performance parameters of the flexible transparent micro-column array, such as sensitivity, measurement range, coupling error and the like, are finally obtained. Further, the image-based tactile sensing method provided by the embodiment of the invention can be feedback-adjusted according to the result of the comparison analysis.
According to the image-based touch sensing method provided by the embodiment of the invention, the light emitting source is integrated in the pressure sensor, so that the bending and deformation of the microcolumn can be clearly captured, and the decoupling of the three-dimensional force is completed by utilizing an intelligent algorithm based on the deformation image, so that the miniaturization, high integration and precision of the sensor are realized.
Based on the above description of the embodiments, as an alternative embodiment, before projecting the light to the top direction of the microcolumn by using the light emitting source, the method further includes: setting the luminous source as a luminous film; spin-coating a light-emitting film on the top of the microcolumn; coating the light blocking material layer on the bottom of the flexible transparent micro-column array and other parts of the micro-columns except the top parts.
Fig. 2 is a schematic view of an image-based miniaturized touch sensing device and a working principle thereof according to an embodiment of the present invention, and as shown in fig. 2, a pressure sensing device constructed by using the image-based touch sensing method according to the embodiment of the present invention generally includes three parts, including: an image sensor (COMS, CCD, etc.), a flexible transparent micro-column array and a luminous source.
Wherein, for the high integration and the compactification of effectual realization, can select the luminous source for the flexible film that can give out light such as membrane, LED membrane, fluorescent screen of night light. By this arrangement, the overall thickness of the finished pressure sensing device can be made to not exceed 1 cm.
Fig. 2 a shows the overall structure of the pressure sensing device, and a flexible transparent micro-pillar array is disposed above the image sensor. Alternatively, the image sensor can acquire the light transmitted from the bottom of the whole micro-column array. The light-emitting film is attached to the surface of the micro-column, so that the light emitted by the light-emitting film is vertically incident to the top end of the micro-column.
Further, the light blocking material layer is coated on the bottom of the flexible transparent micro-column array and other parts of the micro-columns except the top parts, so that light can only pass through the top ends of the micro-columns and is transmitted to the image sensor, and the image sensor can convert received image information into electric signals by utilizing the photoelectric conversion principle. The left image in the image B in fig. 2 is a schematic view of the optical path after the light blocking material layer is coated on the position of the micro-pillar array, as shown in the figure, the light-colored vertical light passes through the top of the micro-pillars and is transmitted to the image sensor, but other non-vertical light is isolated by the light blocking material layer, and the right image in the image B in fig. 2 is the finally obtained deformation image of the micro-pillars.
As can be known from fig. 2, since the light of the light-emitting film is divided into direct light and scattered light, when the light-emitting film is tightly attached to the micro-pillar array, the direct light is transmitted to the image sensor through the micro-pillars from top to bottom, and the scattered light can also be transmitted through the micro-pillar array from various angles, so that the image sensor cannot effectively capture the shapes of the micro-pillars.
In order to clearly capture the bending and deformation of the microcolumn, in the image-based touch sensing method provided by the embodiment of the invention, the light blocking material, such as a pig black dye (SILC pig black), is coated on all the parts except the top end of the microcolumn (and the bottom of the flexible transparent microcolumn array) in a thin manner, so that the transmission of the scattered light of the light-emitting film in the microcolumn array can be effectively blocked, and the light can only be transmitted to the image sensor from the top end of the microcolumn, thereby clearly capturing the bending and deformation of the microcolumn, realizing the three-dimensional force measurement, and adopting the image-based touch sensing method provided by the embodiment of the invention, effectively improving the imaging definition, filtering the influence of external light on the measurement result, and improving the measurement precision.
Optionally, an embodiment of the present invention provides an image-based tactile sensing method, further including adding a variable color development layer on an inner side of the light-blocking material layer near the microcolumns.
The variable color development layer can present different color changes according to the change of the physical quantity of the environment where the variable color development layer is located. Such as: the different variable color development layers can present different colors according to the different components of the variable color development layers, the ultraviolet intensity change, the illumination intensity change, the current (or voltage) intensity change, the pressure intensity change and the like. The color change information of the variable color development layer can be obtained through the image sensor, and the measurement of the environmental physical quantity in a certain range is realized by utilizing an image recognition algorithm.
It should be noted that, in order to facilitate the content of each embodiment of the present invention to be shown, in the following description, the variable color developing layer is taken as an example for description, and details will not be described. Wherein, the temperature-controlled color-changing layer can present different colors according to the temperature of the environment where the temperature-controlled color-changing layer is located.
Optionally, a temperature-controlled color-changing layer with the color capable of changing along with the temperature can be thinly coated on all the parts except the top end of the micro-column.
Further, the color change information of the temperature-controlled color changing layer is acquired by using an image sensor, and then the measurement of the temperature (or other corresponding physical quantities) within a certain range is realized by using an image recognition algorithm (such as a KNN clustering algorithm).
Further, the material that changes with temperature used in the embodiment of the present invention may be a temperature-variable liquid crystal (for example, a temperature-variable liquid crystal with a temperature detection range of 28 ℃ to 43 ℃, or a material with a different temperature detection range may be selected according to actual use conditions).
According to the image-based touch sensing method provided by the embodiment of the invention, the temperature control discoloring layer is additionally arranged at the proper position of the flexible transparent micro-column array, the three-dimensional sensing data of the three-dimensional force is acquired by acquiring the deformation image of the micro-column, and simultaneously, the color change information of the temperature control discoloring layer is acquired by using the image sensor, so that the environment temperature of the flexible transparent micro-column array is acquired, the integration of the internal measurement and the temperature measurement is realized, the miniaturization and the high integration of the sensor are increased, and the development of the sensor towards multiple functions is facilitated.
Based on the content of the above embodiment, optionally, before projecting the light to the top direction of the microcolumn by using the light emitting source, the method may further include: setting the luminous source as a luminous film; coating a light blocking material layer on the top end of each micro-column; and spin-coating the light-emitting film on the light blocking material layer.
The embodiment of the invention provides another image-based touch sensing method, as shown in fig. 4, a micro-column array is directly placed above an image sensor, and then a light blocking material layer is coated on the top end of each micro-column of the flexible transparent micro-column array; and finally, attaching the light-emitting film to the outer layer of the light blocking material layer to provide a light source for the flexible transparent micro-column array.
Because the light blocking material layer is arranged between the light emitting film and the top ends of the micro-columns, light cannot pass through the top ends of the micro-columns and cannot be completely scattered, shadows are projected below the micro-columns, and shadow images are formed at the bottoms of the flexible transparent micro-column arrays. The image sensor (such as CCD or COMS) arranged at the bottom of the flexible transparent micro-column array is used for acquiring image information of the transmitted light, and a micro-column deformation image generated after the three-dimensional force action of the flexible transparent micro-column array is generated according to the acquired image.
Alternatively, the microcolumn in the above embodiment is a widened microcolumn having a diameter of 500 μm to 1 cm.
Repeated tests show that by adopting the image-based touch sensing method of the embodiment, if the microcolumn is provided with the widened microcolumn, that is, the diameter of the microcolumn is appropriately increased, a clearer drop shadow can be obtained, so that the finally generated microcolumn deformation image can more closely reflect the stress condition of the microcolumn, and particularly when the diameter of the widened microcolumn is between 500 μm and 1cm, the detection precision can be effectively improved while the integration of devices is considered.
Alternatively, in the image-based tactile sensing method provided by the embodiment of the invention, on the basis of the above, a temperature-controlled color-changing layer is further added on the inner side of the light-blocking material layer close to the micro-pillars.
Specifically, in the embodiment of the invention, a material (or a material changing color along with other physical quantities, such as ultraviolet, light, electricity, pressure and the like) with a color capable of changing along with the temperature is thinly coated between the top end of the micro-column and the light blocking material layer, the color change information is obtained through the image sensor, and finally the temperature measurement in a certain range is realized by using an image recognition algorithm.
Based on the content of the above embodiment, optionally, before projecting the light to the top direction of the microcolumn by using the light emitting source, the method further includes: arranging the light emitting source as a fluorescent coating; coating a fluorescent coating on the top end of each micro-column; coating a light blocking material layer on the other parts of the flexible transparent micro-column array except the bottom.
Fig. 5 is a schematic view of another miniaturized image-based touch sensing device and its operation principle according to an embodiment of the present invention, as shown in fig. 5, in the pressure sensing device according to this embodiment, a transparent and tangible micro-pillar array is disposed above an image sensor, and a fluorescent coating is spin-coated on the top of each micro-pillar to provide a light source. Then, coating light blocking materials on other parts of the micro-column array except the bottom to isolate external light, so that the light can pass through the top of the micro-column and is transmitted to the image sensor, finally, converting image information into electric information through photoelectric conversion, and reconstructing a micro-column deformation image according to the electric information.
Alternatively, in the embodiment of the present invention, a method for measuring temperature is appropriately adjusted, a material whose color can change with an environmental physical quantity is coated on the whole upper surface of the micro-column array (except for the flexible transparent micro-column array and the bottom), and color change information can be obtained by an image sensor, so that a physical quantity color image is obtained. And finally, identifying the color image of the physical quantity by using an image identification algorithm to realize the measurement of the physical quantity within a certain range.
Based on the above description of the embodiments, as an alternative embodiment, before projecting the light to the top direction of the microcolumn by using the light emitting source, the method may further include: coating a light blocking material layer on the top end of each micro-column; spin coating a luminous source on the outer layer of the light blocking material layer; a grating layer is arranged between the light-emitting source and the light blocking material layer and is used for enabling light rays to vertically enter the top end of the micro-column; the light-emitting source is provided as a light-emitting film or a fluorescent coating.
Specifically, fig. 6 is a schematic view of another miniaturized image-based tactile sensing device and a working principle thereof according to an embodiment of the present invention, as shown in fig. 6, a transparent flexible micro-pillar array is disposed above an image sensor, a light blocking material layer is coated on a top end of each micro-pillar of the transparent flexible micro-pillar array, and a light emitting film is attached to a top end surface (i.e., an outer layer of the light blocking material layer) of each micro-pillar to provide a light source. And finally, in order to filter out scattered light and ensure that the light penetrating through the top end is changed into vertical light from top to bottom, a grating layer is added between the micro-column array and the light-emitting film. Through the arrangement, light emitted by the light emitting source can not pass through the top end of the micro-column, and shadow images can be projected below the micro-column due to the fact that the light is not scattered for vertical light after passing through the grating layer. And capturing shadow images by using an image sensor, and generating a microcolumn deformation image according to the shadow images below each microcolumn. And finally. And analyzing and processing the micro-column deformation image by using the decoupling network model to obtain three-dimensional sensing data of the three-dimensional force applied to the flexible transparent micro-column array.
Alternatively, a temperature-controlled color-changing layer whose color can change with temperature is thinly coated on the top of each micro-column in the pressure sensing device according to the embodiment of the present invention, and the temperature-controlled color-changing layer is disposed on the inner side of the light-blocking material layer close to the micro-column.
The embodiment of the invention provides a miniaturized tactile sensing device based on images, as shown in fig. 7, which mainly comprises: the light emitting source 11, the flexible transparent micro-column array 21, the image sensor 31 and the image processing unit 41;
the light source 11 is mainly used for projecting light to the top direction of the microcolumn so that the light can be emitted through the microcolumn and the substrate; the micro-pillars are positioned on the substrate and form a flexible transparent micro-pillar array 21;
the image sensor 31 is arranged at the bottom of the flexible transparent micro-column array and used for receiving the emitted light and generating a micro-column deformation image; the image processing unit 31 is pre-stored with a decoupling network model, and is used for inputting the microcolumn deformation image to the decoupling network model and outputting three-dimensional sensing data of three-dimensional force applied to the flexible transparent microcolumn array; the decoupling network model is obtained after training based on the sample light spot image of the flexible transparent micro-column array 21 under the action of standard pressure and the corresponding identification label.
Specifically, the flexible transparent micro-column array may include a plurality of flexible transparent micro-columns, and all the flexible transparent micro-columns are arrayed to form the flexible transparent micro-column array according to actual application requirements.
The shape of the microcolumn is not limited to a circular microcolumn, a triangular microcolumn, a polygonal microcolumn, or the like; the diameter of the microcolumn may be set from 1 μm to several cm or the like; the height of the microcolumns can be adjusted according to the diameter of the microcolumns, generally 0.1 to 10 times of the height of the microcolumns, and the spacing between the microcolumns is more than 0.1 times of the height of the microcolumns.
The light source may be a flexible film capable of emitting light, such as a noctilucent film, an LED film, or a fluorescent coating, which is not specifically limited in this embodiment.
The image-based miniaturized touch sensing device provided by the embodiment of the invention is designed according to one principle of acquiring three-dimensional sensing data of three-dimensional force by acquiring a microcolumn deformation image generated by the flexible transparent microcolumn array under the action of the three-dimensional force exerted on the flexible transparent microcolumn array and performing characteristic analysis on the microcolumn deformation image by using a decoupling network model which is trained in advance.
Because the acquisition of the microcolumn deformation image is interfered by external environment, especially in a dark working environment, the image which can clearly reflect the microcolumn deformation cannot be acquired, and even the microcolumn deformation image cannot work normally at all. Therefore, the miniaturized tactile sensing device based on images provided by the embodiment of the invention projects light from the direction perpendicular to the plane of the top end of the microcolumn by additionally arranging the light emitting source. Under the condition that the whole light path from the top end of the micro-column to the substrate is not shielded, the light can form a complete bright spot image at the bottom of the flexible transparent micro-column array; under the condition that partial shielding exists on the whole light path from the top end of the micro-column to the substrate, a incomplete bright spot image is formed at the bottom of the flexible transparent micro-column array; and if the whole light path from the top end of the micro-column to the substrate is completely blocked, forming a shadow image at the bottom of the flexible transparent micro-column array.
Further, an image sensor (such as a CCD or cmos) arranged at the bottom of the flexible transparent micro-column array is used to acquire image information of the transmitted light, and a micro-column deformation image after a three-dimensional force is applied by the flexible transparent micro-column array is generated according to the acquired image.
And finally, inputting the microcolumn deformation image into a decoupling network model trained in advance to decouple the three-dimensional force, and acquiring three-dimensional sensing data of the three-dimensional force.
According to the image-based miniaturized touch sensing device provided by the embodiment of the invention, the light-emitting source is integrated in the pressure sensor, so that the bending and deformation of the microcolumn can be clearly captured, and the decoupling of three-dimensional force is completed by utilizing an intelligent algorithm based on the deformation image, so that the miniaturization, high integration and precision of the sensor are realized.
Based on the content of the foregoing embodiment, as an option, the image-based miniaturized tactile sensing device provided in the embodiment of the present invention further includes a light blocking material layer and a temperature-controlled color changing layer; the light-emitting source is a light-emitting film and is spin-coated on the top end of the microcolumn; the light blocking material layer is arranged at the bottom of the flexible transparent micro-column array and other parts of the micro-columns except the top parts; the temperature control color changing layer is positioned on the inner side of the light blocking material layer close to the microcolumns.
Further, embodiments of the present invention provide a method for preparing the image-based miniaturized touch sensing device as shown in fig. 8, including but not limited to the following steps:
q1, preparing a flexible transparent micro-column array;
q2, coating a variable color development layer on the flexible transparent micro-column array;
q3, coating a light-blocking material on the color variable developing layer.
Wherein, the preparation of the flexible transparent micro-column array in the step Q1 mainly comprises the following steps:
spin coating photoresist on the inner surface of the preparation container; exposing and developing the preparation container; arranging a preset pattern of the flexible transparent micro-column array on the photoresist on the inner surface of the preparation container to manufacture a mould of the flexible transparent micro-column array; and uniformly mixing the prepolymer for manufacturing the flexible transparent micro-pillars in the flexible transparent micro-pillar array with silicone oil according to a first set proportion, pouring the mixture on the mold, heating and curing the mixture, and performing reverse molding to obtain the flexible transparent micro-pillar array.
Wherein, the step Q2 is that the variable color development layer is coated on the flexible transparent micro-column array, which mainly comprises: bonding a glass sheet on the top end of the micro-column; uniformly mixing the liquid variable color development material and the prepolymer in a second set proportion to generate color development material mixed liquid; injecting the color development material mixed liquid into gaps among the micro-columns of the flexible transparent micro-column array, and blowing out redundant color development material mixed liquid by using gas; heating to solidify the color developing material mixed liquid, and taking down the glass sheet.
Wherein the step Q3 of coating the light-blocking material layer on the variable color development layer mainly comprises: bonding a glass sheet on the top end of the microcolumn again; uniformly mixing the liquid light blocking material and the prepolymer according to a third set proportion to generate light blocking material mixed liquid; injecting the light-blocking material mixed liquid into gaps among the micro-columns of the flexible transparent micro-column array, and blowing out redundant light-blocking material mixed liquid by using gas; and heating to solidify the light blocking material mixed liquid, and taking down the glass sheet.
Fig. 9 is a process flow chart of the preparation of the flexible transparent micro-pillar array in the image-based miniaturized touch sensing device according to the embodiment of the present invention, and as shown in fig. 9, the steps of preparing the transparent flexible micro-pillar array may specifically be:
(1) putting the prepolymer (liquid) of the PDMS and the silicone oil into a beaker according to a required proportion, and stirring the mixture for 2 hours by using a planetary stirrer to prepare a compound (liquid) of the prepolymer of the PDMS and the silicone oil for later use.
(2) Cleaning the wafer, performing plasma cleaning, spin-coating SU-8 on the silicon wafer, exposing, developing, transferring the designed pattern to SU-8, and making into a mold.
(3) Compounding prepolymer of PDMS with silicone oil and curing agent according to 10: 1, and removing air bubbles under vacuum, and then pouring on an SU8 template.
(4) And after heating and curing, performing reverse molding to form a micro-channel structure with micro-columns.
Further, the preparation process of the variable color development layer (temperature-controlled color development layer) may specifically be:
(1) and bonding the top ends of the micro-pillars of the sensor chip with the glass sheet.
(2) The temperature-change liquid crystal and PDMS were mixed in a ratio of 2:1, and the mixture was injected into the gap between the micro-columns with a syringe and excess liquid was removed by air-blowing.
(3) Heating to solidify the temperature-variable liquid crystal material.
(4) And taking down the bonded glass sheet after the temperature change liquid crystal material is solidified to obtain the sensor sheet only with the top of the microcolumn without the temperature change liquid crystal.
Further, the preparation process of the light blocking material layer may specifically be:
(1) and bonding the top ends of the micro-pillars of the sensor chip with the glass sheet.
(2) Pig black stain (SILC pig black) and PDMS were mixed at a ratio of 2:1, and the mixture was injected into the gap between the microcolumns with a syringe and excess liquid was removed by air blowing.
(3) Heating to solidify the black colorant.
(4) And (4) taking down the bonded glass sheet after the black pigment is solidified to obtain the sensor sheet only with the top end of the microcolumn not coated with black.
The miniaturized image-based touch sensing device prepared by the process flow provided by the embodiment of the invention does not need electronic components such as capacitors and resistors, so that the volume of a measuring unit and the number of connecting lines of the sensor can be effectively reduced, and the miniaturization and high integration of the sensor can be better realized. Meanwhile, the array type flexible transparent microcolumns are used as sensitive elements, and the sensor elements are provided with light sources, so that the detection sensitivity of the horizontal force (lateral force) is high, the direction of the three-dimensional force can be accurately identified, and the influence of the external force is small. Furthermore, the three-dimensional force measuring device provided by the embodiment of the invention can be attached to substrates with different surface appearances like human skin, and meanwhile, the acquisition of image information can be accurately and rapidly realized, so that the three-dimensional force measuring device has profound significance for realizing artificial touch. The flexible three-dimensional force sensor has wide requirements in the fields of intelligent robots, mechanical assembly, automobile manufacturing, medical treatment and the like.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It is to be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
In the description of the present invention, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description. Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
The above examples are only for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (6)
1. An image-based haptic sensing method, comprising:
projecting light rays to the top direction of the microcolumns by using a light emitting source, wherein the light rays are emitted through the microcolumns and the substrate;
the light emitting source is arranged on the surface of the top end of the microcolumn;
the microcolumns are positioned on the substrate to form a flexible transparent microcolumn array, and light blocking material layers are coated on the flexible transparent microcolumn array except the bottom and the other parts of the microcolumns except the top;
receiving the emitted light by using an image sensor arranged at the bottom of the flexible transparent micro-column array to generate a micro-column deformation image;
inputting the micro-column deformation image into a decoupling network model, and outputting three-dimensional sensing data of three-dimensional force applied to the flexible transparent micro-column array;
the decoupling network model is obtained by training a sample light spot image of the flexible transparent micro-column array subjected to standard three-dimensional force and a corresponding identification label;
or, the light source is used for projecting light rays to the top direction of the microcolumn;
the microcolumns are positioned on the substrate to form a flexible transparent microcolumn array, the luminous source is set to be a fluorescent coating, the top end of the microcolumn is coated with the fluorescent coating, and other parts of the flexible transparent microcolumn array except the bottom are coated with a light blocking material layer;
receiving the emitted light by using an image sensor arranged at the bottom of the flexible transparent micro-column array to generate a micro-column deformation image;
inputting the micro-column deformation image into a decoupling network model, and outputting three-dimensional sensing data of three-dimensional force applied to the flexible transparent micro-column array;
the decoupling network model is obtained by training a sample light spot image of the flexible transparent micro-column array subjected to standard three-dimensional force and a corresponding identification label;
or, the light source is used for projecting light rays to the top direction of the microcolumn;
the microcolumns are positioned on the substrate to form a flexible transparent microcolumn array, the top ends of the microcolumns are coated with a light blocking material layer, and the outer layer surface of the light blocking material layer is provided with the light emitting source;
receiving the emitted light by using an image sensor arranged at the bottom of the flexible transparent micro-column array to generate a micro-column deformation image;
inputting the micro-column deformation image into a decoupling network model, and outputting three-dimensional sensing data of three-dimensional force applied to the flexible transparent micro-column array;
the decoupling network model is obtained after training based on a sample light spot image of the flexible transparent micro-column array subjected to standard three-dimensional force and a corresponding identification label.
2. The method of claim 1, wherein the micro-pillars are disposed on the substrate to form a flexible transparent micro-pillar array, the tips of the micro-pillars are coated with a light blocking material layer, and the light source is disposed on an outer surface of the light blocking material layer, and further comprising:
arranging a grating layer between the light-emitting source and the light blocking material layer, wherein the grating layer is used for enabling the light to vertically enter the top end of the micro-column;
the light-emitting source is provided as a light-emitting film or a fluorescent coating.
3. The image-based haptic sensing method of claim 2, wherein the microcolumns are diameter-widening microcolumns having a diameter of 500 μm-1 cm.
4. The image-based haptic sensing method of any of claims 1-3, further comprising:
and a variable color development layer is additionally arranged on the inner side of the light blocking material layer close to the microcolumns.
5. An image-based miniaturized tactile sensing device, comprising:
the device comprises a light emitting source, a flexible transparent micro-column array, an image sensor, an image processing unit, a light blocking material layer and a variable color development layer;
the light source is a light-emitting film and is attached to the top end of the microcolumn;
the light source is used for projecting light rays to the top direction of the microcolumn, and the light rays are emitted through the microcolumn and the substrate;
the micro-pillars are positioned on the substrate and integrally formed to form a flexible transparent micro-pillar array;
the light blocking material layer is arranged at the bottom of the flexible transparent micro-column array and other parts of the micro-columns except the top parts;
the variable color development layer is positioned on the inner side of the light blocking material layer close to the microcolumns;
the image sensor is arranged at the bottom of the flexible transparent micro-column array and used for receiving the emitted light and generating a micro-column deformation image;
the image processing unit is prestored with a decoupling network model and is used for inputting the micro-column deformation image into the decoupling network model and outputting three-dimensional sensing data of three-dimensional force applied to the flexible transparent micro-column array;
the decoupling network model is obtained after training based on a sample light spot image of the flexible transparent micro-column array under the action of standard pressure and a corresponding identification label.
6. A method of making an image-based miniaturized touch-sensing device according to claim 5, comprising sequentially performing the steps of:
preparing a flexible transparent micro-column array, coating a variable color development layer on the flexible transparent micro-column array, and coating a light blocking material layer on the variable color development layer;
wherein, the preparation of the flexible transparent micro-column array comprises the following steps:
spin coating photoresist on the inner surface of the preparation container;
exposing and developing the preparation container;
arranging the preset pattern of the flexible transparent micro-column array on the photoresist on the inner surface of the preparation container to manufacture a mould of the flexible transparent micro-column array;
uniformly mixing prepolymer for manufacturing the flexible transparent micro-pillars in the flexible transparent micro-pillar array with silicone oil according to a first set proportion, pouring the mixture on the mold, heating and curing the mixture, and performing reverse molding to obtain the flexible transparent micro-pillar array;
the coating of the variable color development layer on the flexible transparent micro-pillar array comprises:
bonding a glass sheet on the top end of the micro-column;
uniformly mixing the liquid variable color development material and the prepolymer in a second set proportion to generate color development material mixed liquid;
injecting the color development material mixed liquid into gaps among the micro-pillars of the flexible transparent micro-pillar array, and blowing out redundant color development material mixed liquid by using gas;
heating to solidify the color developing material mixed liquid, and then taking down the glass sheet;
the coating of the light-blocking material layer on the variable color development layer comprises:
bonding a glass sheet on the top end of the micro-column again;
uniformly mixing the liquid light blocking material and the prepolymer according to a third set proportion to generate light blocking material mixed liquid;
injecting the light blocking material mixed liquid into gaps among the micro-pillars of the flexible transparent micro-pillar array, and blowing out redundant light blocking material mixed liquid by using gas;
and heating to solidify the light blocking material mixed liquid, and then taking down the glass sheet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010568984.9A CN111721451B (en) | 2020-06-19 | 2020-06-19 | Image-based touch sensing method, miniaturized device and device manufacturing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010568984.9A CN111721451B (en) | 2020-06-19 | 2020-06-19 | Image-based touch sensing method, miniaturized device and device manufacturing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111721451A CN111721451A (en) | 2020-09-29 |
| CN111721451B true CN111721451B (en) | 2021-09-28 |
Family
ID=72568683
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010568984.9A Active CN111721451B (en) | 2020-06-19 | 2020-06-19 | Image-based touch sensing method, miniaturized device and device manufacturing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111721451B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112561898A (en) * | 2020-12-22 | 2021-03-26 | 电子科技大学中山学院 | Optical fiber sensor light spot analysis method based on convolutional neural network |
| CN113776722A (en) * | 2021-07-27 | 2021-12-10 | 西北工业大学 | Preparation process of micro-cylinder sensor measurement array |
| CN114659678A (en) * | 2022-04-12 | 2022-06-24 | 深圳市松果体机器人科技有限公司 | Surface-shaped flexible touch sensor |
| CN114993542B (en) * | 2022-05-23 | 2023-12-08 | 东南大学 | Touch and slide sense integrated sensor based on single-contact soft vision |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105938031A (en) * | 2016-04-15 | 2016-09-14 | 上海洞舟实业有限公司 | Pressure light-emitting sensor thin film |
| CN106462285A (en) * | 2014-04-10 | 2017-02-22 | Ib韩国有限责任公司 | Biometric sensor for touch-enabled device |
| CN107141666A (en) * | 2017-06-22 | 2017-09-08 | 南京工业大学 | A kind of power electric light converts the preparation method of enhanced luminous laminated film |
| CN110455454A (en) * | 2019-06-28 | 2019-11-15 | 北京化工大学 | A kind of more array point three-dimensional force measuring methods and its device of view-based access control model |
| CN110793674A (en) * | 2019-10-25 | 2020-02-14 | 北京化工大学 | Pressure sensor array based on vision and manufacturing method thereof |
-
2020
- 2020-06-19 CN CN202010568984.9A patent/CN111721451B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106462285A (en) * | 2014-04-10 | 2017-02-22 | Ib韩国有限责任公司 | Biometric sensor for touch-enabled device |
| CN105938031A (en) * | 2016-04-15 | 2016-09-14 | 上海洞舟实业有限公司 | Pressure light-emitting sensor thin film |
| CN107141666A (en) * | 2017-06-22 | 2017-09-08 | 南京工业大学 | A kind of power electric light converts the preparation method of enhanced luminous laminated film |
| CN110455454A (en) * | 2019-06-28 | 2019-11-15 | 北京化工大学 | A kind of more array point three-dimensional force measuring methods and its device of view-based access control model |
| CN110793674A (en) * | 2019-10-25 | 2020-02-14 | 北京化工大学 | Pressure sensor array based on vision and manufacturing method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111721451A (en) | 2020-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111721451B (en) | Image-based touch sensing method, miniaturized device and device manufacturing method | |
| CN111664976B (en) | Visual halo-based array pressure measurement method and device and preparation method | |
| EP3529586B1 (en) | System and method for performing automated analysis of air samples | |
| CN103842793B (en) | Sample is applied to device sensing and placement | |
| CN110455454B (en) | Multi-array point three-dimensional force measuring method and device based on vision | |
| CN108161994A (en) | A kind of multi-modal tactile sensor | |
| CN104704423A (en) | An apparatus and method for capturing images | |
| JP5078920B2 (en) | Liquid level detection apparatus and method | |
| WO2005047814A1 (en) | Displacement sensor apparatus | |
| CN106225879B (en) | High-precision non-contact transparent liquid level measuring device and measuring method | |
| Cheng et al. | A flexible pressure sensor based on silicon nanomembrane | |
| CN104034282A (en) | High-precision surface acquiring method in in-situ liquid shaping manufacturing of an optical micro lens | |
| CN110807390A (en) | Light guide component, manufacturing method thereof and fingerprint identification device | |
| CN110945344B (en) | Sensor with gradient nanostructures and method of use thereof | |
| Yao et al. | Fabrication and experimental validation of a sensitive and robust tactile sensing array with a micro-structured porous dielectric layer | |
| TWI480534B (en) | Optical apparatus and operating method thereof | |
| JPWO2016021516A1 (en) | Localized surface plasmon resonance sensing chip and localized surface plasmon resonance sensing system | |
| CN112345510A (en) | Manufacturing process and application of flexible Raman enhanced intelligent sensing device suitable for rape crops | |
| CN207756164U (en) | Biomedical detecting system based on nano impression micro flow chip | |
| KR20140030853A (en) | Input system with electronic pen | |
| CN106442521A (en) | Method, apparatus and computer program product for testing a device | |
| CN110186802A (en) | Measurement device | |
| KR100584107B1 (en) | The Optical Fingerprint Image Apparatus and Quality Test Method of the Same | |
| US20150050187A1 (en) | Chemical sensor, method of producing chemical sensor, and chemical detection apparatus | |
| CN112345483B (en) | Method for generating near infrared spectrum data based on pattern sequence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |