KR20220024447A - System and method for detecting deformation of magnetic material and method for manufacturing the same - Google Patents

Abstract

A soft magnetic sensor comprising a soft material comprising randomly distributed magnetic particles and a magnetometer capable of estimating a force and localizing contact over a continuous region. A reference magnetometer can be used to filter out motion and ambient noise. Methods for localizing the contact location and determining the force include data analysis of the magnetometer output. In some embodiments, the sensor may localize the object prior to contact.

Description

System and method for detecting deformation of magnetic material and method for manufacturing the same

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is filed on June 21, 2019 in 35 U.S.C. Provisional Application Serial No. 62/864,766. Claims under § 119, which is incorporated herein by reference.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

This invention was made with government support under N00014-16-1-2301 awarded by the Naval Research Laboratory. The government has certain rights to inventions.

field of invention

The techniques of this disclosure relate generally to sensing. More particularly, the present invention relates to soft sensing that utilizes deformation of a magnetic material to provide feedback to the environment.

The continued development of wearable technologies, soft robotics, and human-robot interactions has led to increased interest in sensors. With these techniques, inaccuracies in determining the position of objects can affect their ability to perform certain functions. In the case of robotic systems, for example, inaccuracies can prevent the robot from locating and manipulating tools or other objects. Vision-based sensing is excellent for locating objects in the workplace, but cannot provide guidance within a distance of 1 to 2 cm from a target object. Moreover, if the camera is obscured or the surface is reflective or transparent, the vision-based system will not work properly.

To overcome the limitations of vision-based systems, tactile sensors measure contact force and provide important information about the environment. Because tactile sensors are touch-based, they provide information only after an object has been touched and do not aid in access to the target object. Soft tactile sensors are a subclass of tactile sensors that use deformable yet flexible materials on an interactive surface. Soft sensors not only provide rich environmental information, but also contribute to effective mechanical properties that enable successful robot manipulation, human-robot interactions, and material sorting. Soft tactile sensors can use different conversion modes such as optical, resistive and capacitive. Although flexible tactile sensors can provide higher precision than sight-based sensors, non-scalable manufacturing techniques, lack of customization, and complex integration requirements still limit widespread implementation of flexible sensors. In the case of resistive or capacitive flexible sensors, for example, the increased density of each unit is associated with unmanageable expansion of wiring and errors at weakly flexible electrical interfaces. Moreover, like other tactile sensors, soft tactile sensors only provide information after an object has been touched.

Magnetic sensing overcomes many of these obstacles because of its limited reliance on direct electrical wiring, but provides high-resolution, high-speed sensing by measuring changes in magnetic flux or electromagnetic induction. Furthermore, in some applications magnetic sensing may provide sensor output prior to contact. Despite these improvements over other types of sensors, magnetic sensors are susceptible to environmental magnetic noise. Additionally, when implemented as a flexible sensor, material defects may occur at the junction between the rigid magnet and the flexible elastomer used in the sensor, limiting the technique to non-ductile sensing applications. For example, a typical magnetic sensor combines a Hall effect sensing chip and individual permanent magnets suspended between two layers of elastomeric. Therefore, it would be advantageous to develop a sensing system that overcomes these limitations to provide a tactile surface for single-point contact localization and to provide rapid localization and force estimation in free space.

Aspects disclosed in the detailed description include a flexible sensor, a method for sensing deformation of a magnetic material (magnetic body), and a method for manufacturing the same. Related methods and systems are also disclosed.

At least one non-limiting embodiment is a soft magnetic sensor comprising a soft material containing randomly distributed magnetic particles and a magnetometer capable of estimating force and localizing contact over a continuous region. In one example, the sensor covers a continuous area of approximately 15-40 mm 2 . In some embodiments discussed herein, the force and local contact are estimated using an integrated circuit for data analysis of the output from the magnetometer. In some embodiments, the magnetic material, or 'skin', consists of a silicone elastomer loaded with magnetic particulates. As the elastomer deforms, some of the embedded magnetic particles may change position and/or orientation with respect to the magnetometer, resulting in a change in the net measured magnetic field. In one embodiment, the magnetometer may be embedded in a magnetic material to form an integrated sensor. In a variant embodiment, the magnetic material and the magnetometer are separate. The magnetic field data received by the magnetometer is analyzed to provide useful information on force and contact localization. A classification algorithm that analyzes the output of the magnetometer is able to localize the pressure with an accuracy of >98%. In some embodiments, the regression algorithm averages approximately 3 mm2 Pressure can be localized by area. In this regard, systems and methods for sensing deformation of magnetic materials, such as sensing skins, are designed for simple, rapidly integrated, and information-rich sensors for use in fields such as robotic manipulation, flexible systems, and wearables. can address the growing need.

1A-1E illustrate various embodiments of a sensor.
Fig. 1f is an overview of data processing steps.
2A to 2F are a series of graphs illustrating the results of sensing demonstration.
3A-3G are a series of graphs illustrating the results of alternative sensing demonstrations.
4A-4C are examples of sensors according to some variant embodiments.
5 is a series of graphs showing classification and regression results.
6 shows X and Y vectors for an example implementation for a robotic arm.
7 is a visualized vector for indentation of magnetic material.
8 is a flowchart illustrating a manufacturing technique.
9 is a variant embodiment of a sensor having a magnetometer separated from a magnetic material.
10 shows another modified embodiment.

In one exemplary embodiment, the sensor 100 includes a magnetic material 101 and a magnetometer 102 capable of sensing changes in the magnetic field of the magnetic material 101 produced by deformation of the material 100 . do. 1A-1B , the sensor 100 has a tactile skin with a fixed telescoping triaxial magnetometer 102 covered with a soft elastomer 103 embedded by dispersion of magnetic particles 104 . By being implemented as, the magnetic material 101 is formed. The composite magnetic material 101 maintains the stretch and flexibility of the host elastomer 103 and is compatible with the stretchable circuitry. In alternative embodiments, multiple magnetometers 102 may be used (see FIGS. 1C-1D ). As a strain is applied to the surface of the sensor 100 , the magnetic particles 104 are displaced with respect to the relatively static position of the magnetometer 102 (see FIG. 1E ). The magnetometer 102 measures changes in the surrounding magnetic field and analyzes this data to determine the location and force of the contact. Magnetometer 102 measures the magnetic field around it in the x-, y-, and z-directions. The plurality of magnetic particles 104 distributed throughout the magnetic material 101 exhibits input data that is ultimately reduced with a triaxial magnetic field measurement in order to preserve information regarding the deformation of the material 101 . Morphological calculations may also be utilized through reduction of the inherent dimensions of the material 101 itself. For example, sensor 100 may utilize morphological computational properties to essentially reduce the dimensions of the output prior to analysis, thus eliminating the need for a dense array of basic microelectronic chips and wiring.

In contrast to discrete permanent magnets or other conventional magnets, the overall magnetic strength of the magnetic material 101 is smaller. However, the signal magnitude is sufficient to localize the contact and estimate the force on the surface of the sensor 100 . Incorporation of particulate 104 into elastomer 103 also allows for sensor 100 with few restrictions on shape, size or thickness. Unlike some magnetic sensors, it does not require a multi-layer molding process, making it easy to manufacture.

Referring again to FIGS. 1C - 1D , a sensor 100 is shown having a plurality of magnetometers 102 , one magnetometer 102 being identified as a reference magnetometer 105 . In the embodiment shown in FIGS. 1C-1D , five magnetometers 102 are positioned adjacent to the magnetic material 101 , with a reference magnetometer 105 positioned at a constant distance from the magnetic material 101 . . In this particular example, five magnetometers 102 are spaced 15 mm apart, a range before the signal of magnetic material 101 can no longer be detected by the nearest magnetometer 102 . Each 15 mm range overlaps the other magnetometer ranges by 2.5 mm to maximize the functional surface area and minimize the required number of magnetometers 102 . The reference magnetometer 105 is used as a reference for measuring a magnetic signal distinct from the magnetic material 101 (ie, ambient magnetic noise). With multiple magnetometers 102 and a separate reference magnetometer 105, sensor 100 can filter out ambient magnetic noise and motion and incorporate data analysis of the output to address the increased nonlinearity of the system. there is. In other words, the signal from the reference magnetometer 105 coupled with the main magnetometer 102 isolates the change in magnetic flux due to deformation of the magnetic material 101 . The magnetic flux signal is evaluated to provide a real-time estimate of force and position.

1C-1D , the signal may be evaluated by a trained neural network to provide an estimate of the force and position of the contact on the sensor 100 . Figure 1f is an overview of the preprocessing steps for contact localization and force estimation. In this preprocessing step, the raw magnetometer values are individually calibrated, transformed from the reference signal, filtered, and scaled for neural network input.

More detailed description, in one embodiment, signal processing combines preprocessing with calibration to minimize the amount of data collection required while maintaining neural network input limited to raw magnetometer data. Each magnetometer 102 outputs three-axis data with respect to its surrounding magnetic field. For the embodiment shown in Figures 1-2, there are six magnetometers 102, 105 for a total of 18 data points for each sample. For pre-calibrated magnetometers 102 , 105 (providing offset and scaling), these parameters can be applied to the raw data to calibrate the signal individually. The offset may be determined by an average between the maximum and minimum signals in each direction. Scaling can be determined by dividing the average chord distance of all three by the average chord length in each direction. Next, the affine transformation of the reference magnetometer 105 is applied to the five magnetometers 102 . As long as the reference magnetometer 105 remains fixed relative to the other magnetometers 102 , this transformation allows for motion and ambient noise cancellation due to positional and environmental noise. Denoising allows data acquisition to occur in one plane. Once the data has been calibrated and filtered, the data is prepared for input to the neural network by removing the mean and scaling to the unit variance determined from the training data. Although this system provides useful information, it can be susceptible to the additional noise present in many magnetometer 102, 105 systems. In this way, a multi-layer perceptron implemented while learning MLPRegressor may be used.

In contrast to the prior art using large and rigid magnets, the magnetic particles 104, which may include magnetic Ne-Fe-B particles or nanoparticles, have diameters on the order of about 200 μm or smaller. However, other sizes and shapes of particles 104 may be used as long as the composite magnetic material 101 maintains a level of stretchability or softness depending on the intended application. Depending on the intended application and the amount of magnetic particles 104 used, the composite magnetic material 101 may have the same or similar properties as the elastomer 103 used to create the composite magnetic material 101 . In addition, the use of microscale magnetic particles 104 can reduce the intensity of internal stress concentration when a mechanical load is applied to the magnetic material 101 , and also make the material 101 flexible and/or stretchable. can do. For example, if a large magnet is embedded in an elastomer, the difference in compliance between the two materials can cause delamination at the interface between the hard magnet and the soft elastomer when a mechanical load is applied. Moreover, such embodiments may allow for thin geometries and/or include sharp 3D geometries.

In one embodiment, the sensor 100 is formed with the following manufacturing process (see FIG. 8 ). First, the magnetic material 101 is functionalized by mixing the silicone elastomer 103 and the magnetic particles 104 and curing the composite material under a magnetic field. The material is cured in a magnetic field to align the magnetic particles 104 before they are fixed to the cured elastomer 103 , creating a homogeneous magnetic orientation of the magnetic particles 104 . In a variant embodiment, non-uniform magnetic orientation is used. As a further example, the prepolymer and crosslinker may be shear mixed in a 1:1 ratio for about 30 seconds. The pre-cured elastomer mixture may be hand mixed with the magnetic particulate 104 (MQP-15-12; Magnequench) in a 1:1 weight ratio to form the magnetic material 101 . The uncured magnetic material 101 can then be poured into a mold and degassed for 5 minutes. A thin plastic film may be placed on top of the mold and excess material 101 may be squeezed out. The mold can then be filled and placed upside down on the surface of permanent magnets (N48; Applied Magnets). The material 101 can then be cured at room temperature and removed from the mold within an hour. Finally, the magnetic material 101 may be attached (Silpoxy; Smooth-On) on top of a commercial magnetometer 102 substrate (eg MLX90393, Sparkfun). In a variant embodiment, urethane foam is used as elastomer 103; One of ordinary skill in the art will appreciate that several types of elastomer 103 may be used. In another alternative embodiment, a deformable material other than a polymer may be used as the substrate for the composite magnetic material 101 .

With respect to the design and manufacture of the stretchable circuit 106 , the stretchable circuit 106 may include a magnetometer 102 (MLX90393; Melexis) and five output wires ( FIG. 1B ). An additional output wire may be for a second 3.3 volt line, which may be useful due to the single layer design of the telescoping circuit 106 . A thin layer of copper and chromium is sputtered onto the surface of PDMS (Sylgard 184; Dow Corning) and patterned with a laser, leaving circuit traces. Eutectic gallium indium (EGaIn) is selectively wetted into the remaining copper traces when immersed in a bath of NaOH. The circuit components 106 can then be placed directly on top of the liquid metal traces and sealed with an additional layer of PDMS.

For example, a sensor 100 configured in this manner may collect data such as pressure data generated due to deformation of the magnetic material 101 and a change in the magnetic field emitted from the material 101 , which may include the magnetometer 102 . ) is detected by In some embodiments, due to the non-uniform distribution of particles 104 within magnetic material 101 that can create an intrinsic magnetic field, data-driven techniques are used to classify the location of deformations occurring in magnetic material 101 and determine the depth of deformation. can be estimated In particular, the positions can be classified with 98% accuracy, for example, for a 5mm radial circle with 3mm resolution 5x5 grid and 3 discrete depths. The regression algorithm can localize the contact to a 3 mm2 area. In this regard, some embodiments disclosed herein provide an approach to address the need for a continuous and smooth tactile surface with simple fabrication, rapid integration, and adaptable geometries.

As a demonstration of location sensing, for the embodiment including a 5x5 grid, force and magnetic field changes were collected for a total of 25 classes from a 3mm resolution 5x5 grid up to a depth of 3mm (see Fig. 2a). became 2750 contact samples were collected at these 25 locations using a uniform random distribution. Each class (25 in total) contains approximately 100 samples each.

Several different classification algorithms were able to accurately distinguish between the 25 locations, as shown below. In this regard, classification results using quadratic discriminant analysis (QDA) to account for various aspects of performance are discussed here. In case of misclassification, the predicted class is adjacent to the actual position (see Fig. 2b - QDA classification for position 13). The classification accuracy for all positions is shown in Fig. 2c (all QDA classification results grouped by class). 2d-2e show the mean absolute error of the linear regression grouped by position for the x-position, the y-position, respectively, and the mean absolute error of the KNN regression for force.

To estimate the positions, 25 discrete positions were transformed into their coordinate positions. For the 5×5 grid and linear regression experiments, the x-position has a mean error of 1.1 mm and the y-position has a mean error of 3.8 mm. The output estimate near the edge of the sensor 100 may have lower accuracy and higher standard deviation. Due to the magnetic signal-to-distance relationship of 1/d3, the signal quality can be expected to decrease with distance. At these points along the edges, the random distribution of particles may begin to have a greater effect on the output signal than the applied deformation. This can lead to abnormal signal changes, which may be why data driven techniques may be more useful in some non-limiting embodiments instead of a function fitting approach.

As a demonstration of position and depth sensing, for the embodiment including the circular sensor 100, the force-controlled change of the magnetic field was measured for 8 different XY positions and 3 different depths (dZ = 1, 2 or 3 mm). (see Fig. 3a). 2850 contact samples were collected for these 24 XYZ positions using a uniform random distribution. Each class (24 in total) has about 110 samples each. As shown here, Quadratic Discriminant Analysis (QDA) can be used to classify positions based on XY position and depth. If the predicted class is wrong, it can be predicted as an adjacent class (see Fig. 3b - QDA classification results for position 3 and a depth of 1 mm). Misclassifications between adjacent locations may be more common than adjacent depths. A large correlation between the z-axis magnetic field and pressure can be used to differentiate depth. Because all tested locations are closer to the magnetometer 102 than the 5x5 experiment, the same introduced noise from the particles 104 may or may not be observed. The classification accuracy for all positions is shown in Fig. 3c. In general, the smaller the applied pressure (depth = 1), the smaller the signal change and the lower the accuracy. For this sample, position 3 and depth 1 had lower classification accuracy. This may be due to a combination of misalignment leading to a smaller signal on the right, which is also the location of Fig. 3d (mean absolute error from linear regression output grouped by position relative to x-position) and Fig. 3e (y position). The larger errors in 2, 3 and 4 are also evident. Figure 3f shows the mean absolute error for the z-position and Figure 3g shows the mean absolute error of the KNN regression grouped by position for force.

Continuing with the demonstration discussed above, 24 classes were transformed into their real (x,y,z) coordinates for position estimation. For 8-point circle and linear regression, the mean absolute error of the x-position is 1.2 mm and the mean absolute error of the y-position is 3.4 mm across all classes. The difference in error between the x-coordinate and the y-coordinate may mean a small misalignment in this test, and is also shown in the error varied by position in FIGS. 3D and 3E . The z-position error is relatively small (0.03 mm), probably due to the larger signal change associated with a 1 mm depth change (Fig. 3f).

To estimate the force, time series data and k-nearest neighbors (KNN) regression can be used. The inputs may be the Bx, By, and Bw components of the magnetic field, the internal temperature of the magnetometer Bt, and the load cell output at each time step. For the 5×5 grid demonstration, the average error for force estimation was 0.44 N (Fig. 2f). For an 8-point circle, the mean error for the force estimate was about 0.25 N (Fig. 3g). The z-axis of the magnetic field had a strong correlation with the pressing force, making the force estimation relatively accurate. However, a good signal change can depend on the amount of strain. Therefore, if the elastomer 103 used in the magnetic material 101 has a higher Young's modulus, the force resolution can be larger. The range of force applied during both tests was approximately 0 to 2.5 N limited by the selected maximum depth of 3 mm.

As a demonstration of the capabilities of sensor 100 , a simple four-key directional game pad is shown in FIG. 4A . As shown in FIG. 4A , four acrylic arrows may be attached to the surface of the sensor 100 to help the user find a location to apply pressure to input a direction command. The four commands can be identified by changes in the X, Y, and Z components of the magnetic field. In this example, a simple threshold was found to be appropriate when no classifier was used and instead the buttons were sufficiently spaced apart. Positive and negative X and Y changes in Ms. It maps to the 4 arrow keys on the keyboard to play Pac-Man. Exemplary data for each direction from the game is also shown in FIG. 4A .

To demonstrate the speed and accuracy of the 5×5 grid classifier, a short game of minesweeper was played using a robot-controlled cylindrical indenter as shown in Fig. 4b. Each of the 25 grid positions is mapped to a mouse position on the screen. The length of the signal (ie, the duration of the applied pressure) indicates whether the user will display a rectangle with a left click or place a flag with a right click. Immediately after the signal returns to rest, the QDA classifier is used to predict the position, and then the appropriate action is performed. The raw data and classification results are shown in Fig. 4b.

Because the sensor 100 is stretchable and flexible, it can be integrated with existing stretchable circuit technology. Similar to the 4-key pad, the 4 keyboard commands (ctrl+left, ctrl+right, ctrl+up, ctrl+down) navigate through the music playlist to 4 positions (previous, next, volume) via a vector threshold to navigate through. large, small volume) (Fig. 4c). Without the acrylic arrows, user input could change with position, resulting in more noisy data. In addition, the user's hands and skin can be deformed along with the magnetic skin. Although these two factors contribute to additional noise, the system can still operate with approximately four default thresholds to determine the quadrant of the contact.

In additional embodiments, the range and resolution for force and contact locations may be improved by adjusting the fabrication process of the sensor 100 or magnetic material 101 , modifying the training procedure, or adding an additional magnetometer 102 . . The sensor 100 discussed herein may be used in applications including soft robotics, medical devices, manipulation and tactile surfaces. Further, in some non-limiting embodiments, the sensor 100 may be shaped to conform to the geometry of the host system and may be magnetically programmed to respond to a prescribed mechanical load or strain.

In connection with some embodiments discussed herein, time series data is represented as a set of representative features. In addition, 21 features were identified manually instead of an automated feature selection method. The 21 features include the minimum, maximum, mean, standard deviation, median, and sum of each axis for the sample (18 features) and the scalar ratio between the three axes (3 features). At the end of the contact, the features are calculated from the data collected during the contact time, and the classification and regression results are output immediately. Thus, as discussed herein, deformation of randomly distributed magnetic particles 104 can produce repeatable and separable signals.

When analyzing data received from the magnetometer 102 , the magnetic field strength is estimated to decay with distance to the magnetometer 102 using an inverse cubic relation:

Regarding methodology, the classification algorithm of the Python scikit-learn toolkit was evaluated (see Figure 5 for a full comparison of all available classification algorithms and implementation details). The results include reduced parameter tuning while being able to successfully discriminate multiple classes with a relatively small data set. A supervised learning algorithm that does not require extensive hyperparameter tuning was used and was suitable for multi-class classification.

In this regard, the following classification algorithm was used:

ㆍ LDA: Linear Discriminant Analysis is a classifier that aims to find linear decision boundaries under the assumption that each class is a multivariate Gaussian density with mean and equal covariance. We used singular value decomposition (SVD), no shrinkage, prior or dimensionality reduction.

• QDA: Similarly, Quadratic Discriminant Analysis is a classifier that aims to find quadratic decision boundaries between each class. Each class is modeled as a Gaussian density and the output prediction is the class that maximizes Bayes' rule. One of the main differences from LDA is that QDA does not assume that each class has the same covariance matrix.

ㆍ KNN: K-Nearest Neighbors use the k nearest samples by some distance metric to classify new inputs. This is a commonly used method for clustering data. Uniform weights, Manhattan distance (11 standard) and k=5 were used in some experiments discussed here.

ㆍ RF: Random Forest classifier is suitable for decision tress on subsamples of a data set. By randomly partitioning the data set, the classifier chooses the best feature among these subsets.

ㆍ DT: Decision Tree (DT) classifier creates a binary tree and splits the nodes based on the feature that holds the largest information. In some aspects discussed herein, decision trees were used using a Classification And Regression Tree (CART) algorithm.

ㆍ GB: Gradient Boosting (GB) is an ensemble classifier that fits n regression trees to the gradient of a specified loss function. In some aspects discussed herein, 100 estimators, deviance loss, and a learning rate of 0.1 were used.

Regarding the regression algorithm, the following regression algorithm was selected to estimate the XY coordinate position. With respect to some aspects discussed above, some regression algorithms were trained using the same features and samples from previous classifications (see FIG. 6 ).

ㆍ LR: Linear regression fits a line using features as coefficients while minimizing the residual sum of squares.

SVR: Support Vector Regression fits the kernel function by minimizing the specified soft margin ε. All errors within this ε boundary are considered zero and the L1 loss is calculated starting from this boundary. In some cases discussed here, ε = 0.1 and an rbf kernel was used.

ㆍ DTR: Decision Tree Regressor follows the same principle as decision tree classification by constructing a decision tree based on maximum information and dividing the nodes. However, in this regressor, the output is continuous.

ㆍ KNN regressor: KNN regression analyzer expands the KNN classification scheme to continuous output using the weighted average of continuous distance functions.

Regarding the 5×5 grid demonstration discussed here, some algorithms are able to distinguish 25 classes with 98% accuracy. In particular, QDA requires more samples than other algorithms to achieve this accuracy. This may be because the first 1000 samples do not capture the variance of the features. The accuracy of LDA classification decreases with increasing sample size. This may mean that the features cannot be separated linearly as more samples increase the noise. It is possible to increase the space of these features to make them more linearly separable.

For the 8-point circle demonstration, some algorithms can distinguish 24 classes. In general, a decision tree-based approach performs well. Similar to the 5×5 grid results, QDA requires more samples before achieving similar performance to CART, RF, and GBoost approaches. KNN performed well in these cases as well. This can be attributed to the z-direction magnetic field separating the classes [0,7], [8,15], and [16,23] in magnitude. This allows the cluster to reduce the problem very quickly to 8 options already. Since the circle radius is smaller than the 5×5 grid, the same noise is not seen from the particle distribution of the material previously discussed.

The linear regression algorithms (linear, ridge, snare, elastic) all have continuous outputs capable of estimating an X position with an average error of about 1.1 mm and a Y position with an average error of about 2.5 mm. The KNN and DT results can be subject to the quasi-discrete nature of the data input leading to a quasi-discrete output. However, continuous sampling over the entire surface may be used.

With respect to a raw vector for a 5x5 grid, magnetometer 102 may have an internal frame of coordinates that may be determined by the magnitude and direction of the output vector. For example, the x-axis of the magnetometer 102 is crossed in FIG. 6 , and the output may vary from negative to positive. Each quadrant around the signal may reflect the correct sign based on their location, and the magnitude of the signal decreases with distance from the magnetometer 102 . However, some discrepancies from this pattern may appear at the edge of the grid experiment.

In Fig. 7, the maximum Bx and Bt vectors are displayed for each position in the grid pattern for 3 mm indentation at 25 positions at a rate of 10 mm/min. Discrepancies are marked with an asterisk in FIG. 7 . It should be noted that the inner 9 position signals are quite large compared to the edges, and the full vector is not represented in the frame. However, it is clear that the nine positions of the interior follow the expected signs of the x- and y-axes of the magnetometer 102 . Specifically, the X vector follows the (pos, neg, neg) pattern from the left to write as we cross the y-axis. Similarly, the Y vector follows a top to bottom (neg, pos, pos) pattern as we cross the x-axis. These patterns are expected and follow rough theory.

The edge case is slightly different. For example, consider the X vectors at positions 15 and 20 (red). Those represented by the center 9 positions are on the negative side of the x-axis, but the signal is positive. This discrepancy appears only at edge locations. This may be due to the non-uniformity of the sample greater than the strain applied at this distance away from the magnetometer 102 . Because of the inverse cube relationship, signals can decay very quickly with distance. In other words, the displacement of the agglomerated particle under the indenter may have a greater effect on the net magnetic field change than the bulk displacement. A similar effect can be seen with the Y vector for positions 20 and 24 (green). Here the signal must be positive, but instead is relatively small and negative. These relatively unpredictable discrepancies may motivate the use of data-driven techniques rather than model-based techniques.

In another non-limiting embodiment, the magnetometer 102 may be configured to sense deformation of the composite magnetic material 101 while being moved relative to the composite magnetic material 101 during operation. For example, in at least one non-limiting embodiment, the composite magnetic material 101 may be positioned in a hand or gripper of a robotic arm, while the magnetometer 102 may be positioned on a robot such as an elbow, shoulder, base or other location. placed on different parts of the arm. During operation of this exemplary embodiment, the gripper or hand may move simultaneously with another aspect of the robotic arm, including the aspect comprising the magnetometer 102 , such that both the composite magnetic material 101 and the magnetometer 102 interact with each other. to move about In this regard, the magnetometer 102 may be configured to sense deformation of the composite magnetic material 101 during movement during operation. In yet another embodiment, the magnetometer 102 may be positioned at a different location than other devices coupled to the composite magnetic material, such as a robotic arm or an object manipulated by the robotic arm.

In the embodiment shown in Figure 9, magnetic material 101 is attached to a key, wherein magnetometer 102 is part of a robotic gripper. The magnetometer 102 in the robot gripper can be positioned on the magnetic material 101 on the key with sum-mm accuracy and enables the robot to pick up objects in the same way in the same place every time. In addition, the robot can consistently localize repeatable grabs and object poses even before making contact. The magnetometer 104 is available in a compact format (7×7×2 mm), provides a fast sampling rate (>100 Hz), and can be easily integrated into a system via serial communication.

10 shows a variant embodiment of the sensor 100 with the magnetometer 102 separated from the magnetic material 101 to enable 3D localization. By separating the magnetic material 101 from the magnetometer 102 , the robot can move freely and measure changes in ambient magnetic flux due to both motion and deformation. The sensor 100 may supplement vision-based object localization. In one exemplary embodiment of the sensor 100 with separate components, the 3-axis magnetometer 102 is mounted on a circuit board with four input wires for SDA, SCL, 3.3V and GND. These four wires allow the magnetometer 102 to communicate with a small microcontroller attached to the end effector or gripper using i ² c.

For the sensor 100 shown in Figures 9 and 10, localization and force feedback are controlled by Maxwell's equations for electromagnetics. For some applications, the process can be simplified by estimating the shape of the magnetic field over the magnetic material 101 as a 2D Gaussian. Using this basis, the z-component of the magnetic field at the surface of the magnetic material 101 can be determined and measured by the magnetometer 102 . For example, for a thickness of 2 mm, the magnetic material 101 used in this exemplary embodiment can range from 3500 to 4500 μT and serves as a reasonable boundary for least squares fit.

The position of the gripper is recorded when the robot gripper and magnetometer 102 reach a maximum magnetic field as they pass near the magnetic material 101 . Moving the gripper to this position will center the magnetic material 101 on its axis. By repeating the process in the other direction, the robotic gripper can be centered over the magnetic material 101 .

Alternatively, the complementary vision-based system may localize the robotic gripper in the proximity of the object to which the magnetic material 101 is attached but will have no information about the scan direction. In this situation, a short-range scan in any direction can be performed and a 1D Gaussian can be fitted to the data points through non-linear least squares optimization. The process can then be repeated for the second axis. By performing these steps, the robotic gripper can be positioned at the estimated peak of the Gaussian. Once the robotic gripper including the magnetometer 102 is localized to the central axis, Maxwell's equations can be used to estimate the position of the surface of the magnetic material 101 . The robotic gripper can then be moved progressively to approach the surface of the magnetic material 101 .

In a further embodiment, the system may include multiple magnetometers at any number of locations to sense a deformation or deformations of the composite magnetic material 101 . In another embodiment, the plurality of magnetic particles 104 comprises neon (Ne), iron (Fe), boron (B), neodymium (Nd), samarium (Sm), cobalt (Co), and any suitable It may include materials such as combinations. Further, in some non-limiting embodiments, the plurality of magnetic particles has a particle having a dimension in the range of 10 ^-7.5 meters to 10 ^-4.5 meters and a particle having a dimension in the range of 0.5 μm (micrometers) to 0.5 mm (millimeters). particles and/or micro-particles, including nanoparticles, including particles having dimensions of 700 nm (nanometers) or less.

The foregoing description, or the following claims, or the features disclosed in the accompanying drawings, which are expressed as appropriate in their particular form or in connection with means for performing the disclosed features, or methods or processes for achieving the disclosed results, are individually Or any combination of these features may be utilized to realize the present invention in its various forms. In particular, one or more features of any embodiment described herein may be combined with one or more features of any other embodiment described herein.

Additionally, protection may be sought for any feature disclosed in any one or more published documents incorporated by reference and/or incorporated by reference with this disclosure.

Claims (20)

As a sensor,
magnetometer; and
Provided with a composite magnetic material having a magnetic field,
composite magnetic material
deformable materials; and
A plurality of magnetic particles dispersed in a deformable material,
and the magnetometer is configured to sense a magnetic field of the composite magnetic material.
The sensor of claim 1 , wherein the magnetometer is configured to sense a change in the resultant magnetic field caused by deformation of the composite magnetic material.
The sensor of claim 1 , wherein the modifying material is an elastomer.
3. The method of claim 1 or 2,
and a reference magnetometer positioned away from the magnetic field of the magnetic material and configured to sense the background magnetic field.
The sensor of claim 1 , wherein the magnetometer is positioned in a fixed position relative to the composite magnetic material.
The sensor of claim 1 , wherein the magnetometer is configured to maintain a substantially fixed position relative to the composite magnetic material in response to deformation of the composite magnetic material.
3. A sensor according to claim 1 or 2, characterized in that the composite magnetic material is in contact with the magnetic flux.
The sensor according to claim 1, wherein the magnetometer is a three-axis magnetometer.
The sensor according to claim 1 or 2, wherein the plurality of magnetic particles is 200 μm or less.
3. The sensor of claim 1 or 2, wherein the plurality of magnetic particles have a substantially uniform magnetic field orientation.
3. The sensor of claim 1 or 2, wherein the plurality of magnetic particles have substantially different magnetic field directions.
The sensor of claim 1 , wherein the plurality of magnetic particles is substantially non-uniformly distributed throughout the deformable material.
3. The sensor of claim 1 or 2, wherein the composite magnetic material is substantially stretchable.
3. The sensor of claim 1 or 2, wherein the composite magnetic material retains the material properties of the deformable material.
3. The method of claim 1 or 2,
A sensor further comprising a telescoping magnetometer circuit connected to the magnetometer.
16. The sensor of claim 15, wherein the stretchable magnetometer circuit is formed as part of a composite magnetic material.
3. The magnetic field of the composite magnetic material of claim 1 or 2, wherein a change in the magnetic field of the composite magnetic material occurs in response to deformation, wherein the deformation causes at least one of the plurality of magnetic particles to change position with respect to the magnetometer. sensor.
A method of localizing an object made of a soft magnetic material, comprising:
passing a magnetometer within a measurable magnetic field range of the magnetic material;
determining a maximum magnetic field measurement to localize a central axis of the magnetic field;
aligning the magnetometer on the central axis;
repeating the above steps to find a secondary axis; and
and determining a distance between the magnetometer and the surface of the object based on a difference between the measured magnetic field strength and the calculated magnetic field strength at the magnetic body surface.
A method of using a soft tactile sensor to determine a position or contact force, comprising:
receiving a signal from the magnetometer comprising a measurement value of a magnetic field generated from a magnetic material comprising an elastomer and a plurality of magnetic particles dispersed within the elastomer;
calibrating the signal;
converting the signal;
filtering the signal;
scaling the signal for input to the neural network; and
A method comprising the step of obtaining a position or contact force using a neural network.
The method of claim 19, wherein filtering the signal comprises:
and removing motion and ambient noise from the signal using the background signal of the reference magnetometer.

Images