WO2016076975A1 - Corrélation d'images numériques pour la mesure de tension et de déformation de revêtement - Google Patents

Corrélation d'images numériques pour la mesure de tension et de déformation de revêtement Download PDF

Info

Publication number
WO2016076975A1
WO2016076975A1 PCT/US2015/053978 US2015053978W WO2016076975A1 WO 2016076975 A1 WO2016076975 A1 WO 2016076975A1 US 2015053978 W US2015053978 W US 2015053978W WO 2016076975 A1 WO2016076975 A1 WO 2016076975A1
Authority
WO
WIPO (PCT)
Prior art keywords
strain
lones
data
streamlines
deformation
Prior art date
Application number
PCT/US2015/053978
Other languages
English (en)
Inventor
JR. Edward W. OBROPTA
Dava J. NEWMAN
Original Assignee
Massachusetts Institute Of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US14/837,455 external-priority patent/US10028697B2/en
Application filed by Massachusetts Institute Of Technology filed Critical Massachusetts Institute Of Technology
Priority to US15/514,297 priority Critical patent/US10555697B2/en
Publication of WO2016076975A1 publication Critical patent/WO2016076975A1/fr

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64G—COSMONAUTICS; VEHICLES OR EQUIPMENT THEREFOR
    • B64G6/00—Space suits
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/16—Measuring arrangements characterised by the use of optical techniques for measuring the deformation in a solid, e.g. optical strain gauge
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G06T2207/30208—Marker matrix

Definitions

  • space suits are needed to protect astronauts from the extreme environment.
  • the human body requires pressure to be continually and evenly applied.
  • the atmosphere applies 101.325 kPa ( 1 atmosphere at sea-level) to the surface of the human body.
  • pressure must be applied using a pressurized capsule or space suit.
  • the safe minimum pressure is limited by breathing pure oxygen at a pressure of 25.33 kPa, although this low pressure requires a long duration of time to transition from an atmosphere with inert gases to avoid decompression sickness.
  • Ideal space suits would have a pressure of 101 .325 kPa to have zero transition time, but this makes traditional space suits too stiff to move.
  • One aspect provides a method for measuring surface deformation and strain using digital image correlation of a surface of a test object.
  • a data acquisition system acquires images of the surface.
  • the surface has a unique surface pattern to facilitate image acquisition.
  • the images are grouped into one or more image sets.
  • Three dimensional image correlation is performed on each of the image sets to determine deformation and strain data.
  • the deformation and strain data from the image sets are stitched into one dataset.
  • Principal strains and lines of non-extension (LoNEs) directions are determined.
  • LoNEs streamlines and lines of maximum and minimum extensions are determined. Visualizations for the strain magnitudes, LoNE streamlines, maximum and minimum extension streamlines are generated in three dimensions.
  • a pattern for one or more customized coverings for the test object is generated based on one or more of the determined principal strains, lines of non- extension (LoNEs) directions, the one or more LoNEs streamlines and the lines of maximum and minimum extensions.
  • the surface is human skin and the test object is a human, and the one or more custom coverings comprise at least one of: custom garments, custom orthotics, custom prosthetics and custom wearable electronic devices.
  • the one or more customized coverings are generated based on the generated pattern.
  • the unique surface pattern is generated having a pattern with a granularity determined to provide image tracking resolution of
  • an application tool to apply the unique surface pattern to the surface is generated.
  • stitching the data from the image sets into one dataset includes converting the image sets from into a plurality of meshes and stitching the meshes together at points of overlap between each mesh.
  • the data of the image sets is merged by re-meshing the data at the overlap point.
  • the data is merged by averaging data of the image sets at the overlap point.
  • a measurement quality index, q is determined, and data from the image sets that reach a predetermined threshold of q is kept, and data from the image sets that do not reach the predetermined threshold of q is discarded.
  • the image sets are in a curvilinear grid data format, and the meshes are triangular meshes.
  • determining principal strains and lines of non-extension (LoNEs) directions includes identifying and determining motion of, based on the unique surface pattern, pixel groups from an image of an initial position of the surface and from an image of a deformed position of the surface.
  • One or more strains of each pixel group are determined and projected onto a two-dimensional (2D) plane tangent to the surface.
  • the projected strains are rotated onto axes defined with respect to the surface.
  • a longitudinal strain, a circumferential strain, a shear strain and principal strains for each pixel group are generated based on the rotated strains.
  • determining one or more strains surrounding each marker point includes calculating at least one of: Lagrangian strains and Euler-Almansi strains.
  • the 2D tangential plane is generated by averaging normal vectors to the planes between neighboring pairs of strain vectors associated with the corresponding pixel group.
  • a neighboring pair of strain vectors includes strain vectors associated with pixel groups that are adjacent to one another.
  • determining one or more LoNEs streamlines and determining lines of maximum and minimum extensions includes selecting one or more seed points within the image sets and linearly interpolating a vector filed of LoNEs directions of the image sets by transforming to a local tangential coordinate system.
  • Streamlines are determined where a current position plus a velocity at that point multiplied by a time step equals a new position.
  • the determined streamlines are translated from a 2D coordinate system to a 3D coordinate system.
  • the image data is acquired by one or more stereoscopic camera pairs.
  • the image data is acquired by at least one of: one or more optical cameras, computed tomography (CT) and magnetic resonance imaging (MRI).
  • CT computed tomography
  • MRI magnetic resonance imaging
  • deformation data is acquired from one or more sensors in contact with the surface.
  • a system for measuring surface deformation and strain using digital image correlation of a surface of a test object.
  • the system includes a data collection system configured to acquire images of the surface, the surface having a unique surface pattern and a lines of non-extension (LoNEs) processor.
  • the LoNEs processor is configured to group the images into one or more image sets and perform three dimensional image correlation on each of the image sets to determine deformation and strain data.
  • the deformation and strain data from the image sets is stitched into one dataset. Principal strains, LoNEs directions, one or more LoNEs streamlines and lines of maximum and minimum extensions are determined. Visualizations for the strain magnitudes, LoNE streamlines, maximum and minimum extension streamlines are generated and provided in three dimensions.
  • a pattern is generated for one or more customized coverings for the test object.
  • the surface is human skin and the test object is a human, and wherein the one or more custom coverings comprise at least one of: custom garments, custom orthotics, custom prosthetics and custom wearable electronic devices.
  • the data collection system includes one or more stereoscopic camera pairs.
  • the data collection system includes at least one of: one or more optical cameras, computed tomography (CT) imager, and a magnetic resonance imager (MRI).
  • CT computed tomography
  • MRI magnetic resonance imager
  • the data collection system includes one or more sensors in contact with the surface.
  • a non-transitory machine-readable storage medium having encoded thereon program code, wherein, when the program code is executed by a machine, the machine implements a method for measuring surface deformation and strain using digital image correlation of a surface of a test object.
  • the method includes acquiring images of the surface.
  • the surface has a unique surface pattern to facilitate image acquisition.
  • the images are grouped into one or more image sets.
  • Three dimensional image correlation is performed on each of the image sets to determine deformation and strain data.
  • the deformation and strain data from the image sets are stitched into one dataset. Principal strains and lines of non-extension (LoNEs) directions are determined.
  • LoNEs Line of non-extension
  • Visualizations for the strain magnitudes, LoNE streamlines, maximum and minimum extension streamlines are generated in three dimensions. Based on one or more of the determined principal strains, the LoNEs directions, the one or more LoNEs streamlines and the lines of maximum and minimum extensions, generating a pattern for one or more customized coverings for the test object.
  • the surface is human skin and the test object is a human, and wherein the one or more custom coverings comprise at least one of: custom garments, custom orthotics, custom prosthetics and custom wearable electronic devices.
  • FIG. 1 A shows a diagram of a finite strain ellipse and FIG. I B shows a diagram of Mohr's circle as employed by described embodiments;
  • FIG. 2 shows the strain ellipse of FIG. 1A in three cases: FIG. 2 case 1 , shows a circle of material (solid lines) deformed (dashed lines) with compression and tension, FIG. 2 case 2 shows the circle deformed with tension in all directions, and FIG. 2 case 3 shows the circle deformed with compression in all directions;
  • FIG. 3 is a graphical representation of Lines of Non-Extension (LoNEs) on a finite strain ellipse;
  • FIG. 4 is a graphical representation of LoNEs as streamlines based on vector fields and associated integration seed points
  • FIG. 5 is a block diagram of an illustrative system to process digital image correlation data, measure and model skin movement, and determine strain fields, contours and three-dimensional patterns in accordance with described embodiments;
  • FIG. 6 is a diagram showing a illustrative test setup to capture digital image correlation data in accordance with described embodiments
  • FIG. 7 is an isometric view of a test movement device in accordance with described embodiments.
  • FIG. 8 is an isometric view of a three dimensional (3D) stamp to produce the unique surface texture shown in FIG. 9;
  • FIG. 9 is a diagram illustrating a unique surface texture employed to perform
  • FIG. 10 is a flow diagram showing an illustrative process for generating LoNEs based on captured image data in accordance with described embodiments
  • FIG. 1 1 is a flow diagram showing an illustrative process for generating a unique surface textures for capturing image data in accordance with described
  • FIG. 12 is a flow diagram showing an illustrative process for correlating captured image data in accordance with described embodiments
  • FIG. 13 is a flow diagram showing an illustrative process for determining strain and LoNEs in accordance with described embodiments
  • FIG. 14 is a flow diagram showing an illustrative process of generating a garment based on determined LoNEs in accordance with described embodiments
  • FIG. 1 5 is a diagram showing an illustrative seed point streamline vector field in accordance with described embodiments
  • FIG. 16 is a diagram showing an illustrative relationship between captured images of a body under test, determined strain magnitudes and directions and LoNEs;
  • FIG. 17 is a diagram illustrating lines of principal strain projected onto a deformed configuration at 90° elbow flexion;
  • FIG. 18 is a diagram illustrating the calculated LoNEs of six test subjects with the elbow joint at 90° and similar seed point locations.
  • FIG. 19 is a diagram illustrating the application of LoNEs to garment design.
  • Described embodiments employ two dimensional (2D) and three dimensional (3D) digital image correlation (DIC) to measure skin strain.
  • Digital image correlation (DIC) is a non-contact optical technique to measure shape and deformation using digital vision, for example stereoscopic vision in the case of 3D DIC.
  • DIC digital image correlation
  • Described embodiments use DIC to measure skin strain at a spatial resolution of less than 1 cm , and compute LoNEs from the collected strain data.
  • described embodiments successfully measure skin strain data at sub-pixel resolution of less than 1 mm 2 .
  • Modelling the mechanical characteristics of human skin is an important aspect of developing skin-tight garments, such as an MCP space suit or other compression garments. Assessing skin deformation can be done using the "finite strain ellipse", which will be described in regard to FIGs. 1 - 3.
  • the finite strain ellipse involves ( 1 ) placing a reference circle on the skin at an area under test in an initial static pose, and (2) moving the area under test into a deformed configuration. The initial circle deforms
  • the finite strain ellipse creates a circle and ellipse pair that can be anal yzed. If the reference circle and deformed circle intersect, they form a pair of directions of non- extension. The directions of non-extension can be connected forming a contour map representing LoNEs, which are unique contours that do not extend during human motion.
  • LoNEs there can be an infinite number of LoNEs, although only a finite number of LoNEs are shown and analyzed to generate a contour map. Although termed lines of "non-extension," LoNEs might not remain an exactly constant length throughout a deformation, but because the LoNEs are similar throughout human motion, the contours typically change length on the order of 5 percent or less. LoNEs are not unique to a particular material, such as skin, and material properties do not cause LoNEs, although material properties can change LoNE patterns.
  • Deformation is the mapping of the reference state, X, to a deformed state, x by the function ⁇ which can be expressed as:
  • the deformation gradient, F is the gradient of the mapping from a deformed frame to a reference frame, given by:
  • Green-Lagrange and Euler-Almansi strain tensors can be computed.
  • the Green-Lagrange strain, E, with respect to the reference geometry is given by:
  • the eigenvalues represent the principal strain magnitudes, E ⁇ and £ 2 , and the eigenvectors represent the direction in which the principal strain magnitudes are oriented. From the principal strain directions, the dir -extension are given by:
  • FIG. 2 shows three cases of deformation that can occur when considering the strain ellipse.
  • circle 20 is outlined in solid lines representing an initial position and ellipse 22 outlined in dashed lines represents a defomied position.
  • an original circle of material is defomied in case 1 with compression and tension, case 2 with tension in all directions and case 3 with compression in all directions. Note that the directions of non-extension only exist in case 1 when there is both tension and compression.
  • reference circle 24 represents an initial position and the deformation ellipse 26 represents a deformed position.
  • complete extension is shown, such that the important direction is the line of minimum extension.
  • reference circle 28 represents an initial position and deformation ellipse 29 represents a deformed position.
  • complete compression is shown, such that the important direction is the line of minimum compression.
  • FIG. 3 shows a graphical representation of lines of non-extension includes a reference circle 310, a deformation circle 312, a line of maximum extension 314, a line of maximum contraction 316, and two lines of non-extension portions 318a and 318b that illustrate diameters belonging to both the circle and the ellipse.
  • LoNEs arc contours along the surface of material that remains tangential to the non-extension direction. As shown in FIG. 4, if the LoNEs directions are treated as a vector field with constant magnitude, the numerical integration of this vector field produces streamlines tangential to the vector field. Since human skin has a thickness, but only surface deformation can be observed, a local coordinate system is used that is normal to the skin surface. The directions of non-extension can be considered an un-directed vector field with first and second directions of non-extension. To calculate the streamline a seed point, 404, is selected and each vector field is integrated from the seed point. The vector field is interpolated at each integration step, which allows the streamline to exist within each mesh element.
  • the interpolated vector field is used to determine streamlines where the new position is the current position plus the velocity at that point multiplied by a time step or spatial parameter, dt. Since human skin has a thickness, the vector field lies along a surface and is not completely 2D or completely 3D. Thus, the integration is performed using a local 2D coordinate system that is transformed for each position to a 3D coordinate system, for example by:
  • Numerical integration techniques applied to seed points produce numerical space curves defined by Cartesian coordinates, which is useful for computer-aided design (CAD) of garments, for example an MCP space suit.
  • CAD computer-aided design
  • surface LIC surface Line Integral Convolution
  • surface LIC allows visualization of the entire vector field instead of only streamlines that connect to seed points.
  • FIG. 5 shows a block diagram of an illustrative system for capturing and processing digital image correlation data, measuring and modeling skin movement, and determining strain fields, contours and 3D patterns, shown as system 500.
  • System 500 includes data collection system 504 to capture information about a body under test 502.
  • data collection system 504 includes image acquisition or motion capture and tracking system 506.
  • Data collection system 504 is in communication with LoNEs processing system 508.
  • LoNEs processing system 508 includes one or more processors, shown as 512 and 516, to process data from data collection system 504.
  • LoNEs processor 512 determines strain and LoNEs data, for example LoNEs streamlines, based on data communicated from data collection system 504 for body under test 502.
  • image processor 516 might process image data collected from data collection system 504 for body under test 502.
  • Data might be transferred between data collection system 504, processors 512 and 516, and input/output (I/O) interfaces 514 by one or more communication links 518 and 520 and stored in memory 510.
  • Communication links 518 and 520 might be any suitable communication interface, for example a physical transmission medium such as a backplane, optical fibers, coaxial cables, twisted pair copper wires, or a wireless transmission medium such as one or more radio frequency (RF) channels or one or more infrared (IR) channels.
  • RF radio frequency
  • IR infrared
  • communication links 518 and 520 might employ any communication protocol over the transmission medium, for example, by operating in accordance with a custom communication protocol, or operating in accordance with standard communication protocols such as a Small Computer System Interface (“SCSI”), Serial Attached SCSI (“SAS”), Serial Advanced Technology Attachment (“SATA”), Universal Serial Bus (“USB”), Ethernet, IEEE 802.1 1 (“WiFi”), IEEE 802.15, IEEE 802.
  • SCSI Small Computer System Interface
  • SAS Serial Attached SCSI
  • SATA Serial Advanced Technology Attachment
  • USB Universal Serial Bus
  • Ethernet IEEE 802.1 1
  • WiFi IEEE 802.15
  • memory 510 might include both volatile and non-volatile storage elements, for example, non- volatile elements including solid-state media, optical media, magnetic media, or hybrid solid-state and magnetic media, and volatile elements including static random-access memory (SRAM) or dynamic random-access memory (DRAM).
  • non- volatile elements including solid-state media, optical media, magnetic media, or hybrid solid-state and magnetic media
  • volatile elements including static random-access memory (SRAM) or dynamic random-access memory (DRAM).
  • FIG. 6 shows greater detail of data collection system 504 and body under test 502.
  • an illustrative embodiment of data collection system 504 for performing digital image correlation includes a plurality of cameras shown as cameras 602a - 602d. Although shown in FIG. 6 as including 4 cameras, any suitable number of cameras might be employed depending on the range of motion to be monitored. Lamp 604 illuminates region of interest 606. Similarly, although shown in FIG. 6 as including 1 lamp, any suitable number of lamps might be employed depending on the range of motion to be monitored and desired resolution of captured images. Cameras 602a - 602d are disposed to focus on a specific region of interest 606 of body under test 502, for example the human elbow as shown. In the embodiment shown in FIG. 5, cameras 602a - 602d are disposed in an arc such that an approximately 180° view of region of interest 606 is captured by data collection system 504.
  • Each of cameras 602a - 602d beneficially employ a lens having a relatively small focal length (e.g., 16mm) with low-distortion to provide a large field of view (FOV) of region of interest 606.
  • a larger focal length lens might be employed to increase the resolution of captured images of region of interest 606.
  • the field of view (FOV) is given by:
  • cameras 602a - 602d are synchronized, for example, by hardware or software triggering for data acquisition (DAQ).
  • DAQ data acquisition
  • data collection system 504 also includes test rig 608, which allows region of interest 606 of body under test 502 to move in a predetermined manner with a predetermined range of motion, thereby reducing variability between test subjects or between tests.
  • Test rig 608 might also align a test subject in a known position for data collection system 504 to perform DAQ.
  • test rig 608 allows the elbow of a test subject to move over a predetermined range of motion, indicated by angle ⁇ , and aligns the test subject at a known distance from, and in a known orientation with regard to, cameras 602a - 602d.
  • FIG. 7 shows test rig 608 in greater detail.
  • test rig 608 includes two support members, 702 and 708, that pivot via hinge 710.
  • Test rig 608 is movable over a range of motion to allow measurement of the test subject's elbow joint angle, ⁇ .
  • Test rig 608 might be positioned attached to a wall or other structure (714) by support member 712 that allows a full range of motion for the test subject while also supporting the weight of test rig 608 itself.
  • Test rig 608 includes grips 704 and 706 to support the test subject's hand and upper arm. For example, the upper arm of a test subject rests on bottom grip 704 and top cylinder 706 is used for hand placement during right arm testing.
  • Test rig 608 could also be adapted for left arm testing.
  • FIG. 8 shows a view of 3D printed pattern stamp 800 used to apply a 3D speckle pattern to region of interest 606.
  • a base color e.g., natural skin tone, a sufficiently skin-tight garment such as a motion capture suit, or a base paint color applied to the skin
  • 3D printed pattern stamp 800 might be 3D printed out of ABS plastic or machined out of other materials.
  • An example of the applied speckle pattern on skin at the elbow of a test subject is shown in FIG. 9.
  • FIG. 1 0 shows a flow diagram of 3D DIC process 1000.
  • process 1000 starts, for example by gathering data about the test subject(s) and region of interest 606 for each test subject. For example, to measure skin deformation and strain at the elbow joint of the human body for six test subjects, age and anthropometric data might first be determined, for example as shown in Table 2. Forearm length is measured from the lateral epicondyle to the styloid process of ulna, upper arm length is measured from the acromion to the lateral epicondyle, and bicep and forearm circumference is measured at the largest circumference along the limb. All dimensions are in cm.
  • data collection system 504 is calibrated using a known pattern of a known size, to determine the optical properties of cameras 602a - 602d and the stereoscopic geometry.
  • a unique surface texture or speckle pattern is generated and stamp 800 is generated according to the generated texture or pattern. Block 1004 will be described in greater detail in reference to FIG. 1 1 .
  • stamp 800 is used to apply the texture or pattern to region of interest 606 of the test subject.
  • image or motion data is captured by data collection system 504.
  • Cameras 602a - 602d remain fixed throughout the data collection period so multiple calibrations were not necessary and all data could be collected in the same coordinate system.
  • each test subject places their arm on test rig 608 and positions their arm at predetermined angle increments throughout the range of motion of their elbow.
  • a full range of elbow and flexion and extension can be captured by data collection system 504 for each test subject.
  • test subjects might position their arm in increments of 15°, starting at 0° elbow flexion to their maximum elbow flexion angle, which is typically approximately 135°. All subjects keep their palms facing upward throughout elbow flexion so that musculature is similar throughout the motion and between test subjects.
  • SLAM simultaneous localization and mapping
  • some embodiments might employ wearable sensors to measure deformation within buckles and folds.
  • optical DIC can only measure in-plane surface deformation because it can only see the surface of the object.
  • CT computed tomography
  • MRI magnetic resonance imaging
  • 3D digital image correlation is performed.
  • LoNEs processing system 508 determines the displacement of pixel subsets in stereoscopic pairs of images captured by data collection system 504. Block 1010 will be described in greater detail in reference to FIG. 12.
  • the image correlation might be performed for stereoscopic camera pairs (e.g., cameras 602a and 602b, and 602c and 602d for the example 4 camera system shown in FIG. 6).
  • stereoscopic camera pairs e.g., cameras 602a and 602b, and 602c and 602d for the example 4 camera system shown in FIG. 6.
  • Green-Lagrange strain and/or Euler-Almansi strain is calculated. Block 1012 will be described in greater detail in reference to FIG. 13.
  • the strains and LoNEs are used to render 3D streamlines onto the surface of the body under test. Described embodiments treat the LoNEs directions as a vector field and integrate the vector field as streamlines tangential to the surface. Because the vector field is not technically a velocity field, the time step used in the integration is not technically a time step, but can be considered a spatial parameter. Interpolating the vector field at each time step determines the vector field at the integration point. To linearly interpolate the vector field, the S nearest neighbors are transformed to the local tangential coordinate system by:
  • ⁇ ( ') is the interpolated vector field.
  • This interpolated vector field is used to determine streamlines where the current position plus the velocity at that point multiplied by the time step is the new position. Since the vector field lies along a surface and is not completely 2D or completely 3D, integration is performed using a local 2D coordinate system and transforming the coordinates to 3D, as given by:
  • FIG. 15 An illustrative projection of LoNEs streamlines projected onto an arm of a body under test is shown in FIG. 15.
  • the generated strains, LoNEs and streamlines are employed to generate garments or garment pieces. Block 1016 will be described in greater detail in reference to FIG. 14.
  • process 1000
  • FIG. 1 1 shows additional detail of block 1004 of FIG. 10.
  • sub- process 1004 begins.
  • the required pattern or texture granularity to meet a desired measurement resolution is determined.
  • a unique pattern or texture having the required granularity is generated.
  • a pattern application tool such as stamp 800 shown in FIG. 8, is 3D printed or machined out of an appropriate material, such as ABS plastic.
  • the granularity of the pattern application tool might be determined by the size or radius of pattern textures 802 in the X-Y plane and the varying heights of pattern textures 802 in the Z-plane.
  • the generated pattern application tool is used to place a desired color, texture and/or pattern onto the skin (or other desired surface) to be monitored.
  • sub- process 1004 completes.
  • FIG. 12 shows additional detail of block 1010 of FIG. 10.
  • data collection system 504 obtains data at various monitored positions of a body under test. As shown in FIG. 6, in one
  • the measurement system includes a motion capture system (including, for example, a set of cameras, video acquisition, processing and tracking markers such as a speckle pattern or texture).
  • a motion capture system including, for example, a set of cameras, video acquisition, processing and tracking markers such as a speckle pattern or texture.
  • other tracking systems might be employed, such as a laser scanner, to record body positions and movements and motions either while the body under test is in motion (dynamic) or in a fixed position (static or stationary).
  • motion data for example image data captured by cameras 602a - 602d
  • the data analysis process consists of optical computer vision techniques, mechanics, mesh stitching and 3D visualization to ( 1 ) determine strain using 3D DIC (e.g., block 1206), (2) stitch datasets together from multiple stereoscopic camera pairs (e.g., block 1210), and (3) calculate LoNEs (e.g., block 1212).
  • Digital image correlation calculates the full strain field of the surface of the object, which in turn can be used to calculate the LoNEs.
  • Digital image correlation does not track the motion of individual speckles or texture points, but rather performs correspondence tests between images by finding matching image subsets.
  • the speckle pattern provides a unique texture and pattern to ensure that image subsets are matched correctly.
  • Each subset corresponds to a single data point in Cartesian space.
  • a strain calculation can be performed on each point by determining the deformation gradient.
  • a stereoscopic correspondence is performed to triangulate each image subset to determine the 3D location of the subset on the actual body under test.
  • a deformation correspondence is performed between the reference frame and the deformed frame. Once a correspondence is made between all the frames a strain calculation can be performed to determine the deformation gradient of all corresponding points.
  • deformation and strain data is determined for each image dataset.
  • the deformation and strain is determined as curvilinear grid data.
  • the stereoscopic datasets are converted from the acquired data type into a mesh data type.
  • the mesh data type might be a triangular mesh including vertices and triangular faces.
  • the meshes for each of the stereoscopic datasets are stitched together at their overlap.
  • the datasets from all of cameras 602a - 602d can be analyzed as a single dataset.
  • the data from the multiple stereoscopic camera pairs is combined into a single dataset at block 1210. It is easiest to combine the camera pair datasets when the datasets are in the same coordinate system, which can be achieved by setting the global coordinate system of each stereoscopic camera pair to a coordinate system defined by the calibration board seen in the first frame of all cameras. As described herein, at block 1208, the datasets for each stereoscopic camera pair might be converted to mesh data having vertices and triangular faces. The meshes are stitched together at their overlap.
  • q the dataset matching error decreases.
  • the quality decreases closer the edge of each dataset, and the magnitude of quality from the datasets will eventually intersect such that data with the higher quality is kept.
  • the two datasets are then stitched together by discarding data from each dataset that has a lower value of q.
  • the strain fields are calculated using a local two dimensional coordinate system and the out-of-plane strain components, E 13 , E 23 , E 33 , are assumed to be zero.
  • sub-process 1010 completes.
  • FIG. 13 shows additional detail of block 1012 of FIG. 10.
  • sub- process 1012 begins.
  • 3D strain values are computed for each monitored point in the correlated images.
  • second order Green-Lagrange (or Lagrangian) strains ⁇ are also possible.
  • the deformation gradient, F is used to determined Green-Lagrange strains, E, determined by relationship (3) above, or Euler-Almansi strains, e, determined by relationship (4) above, where F is the deformation gradient given by relationship (2) above. is the gradient of the mapping from the deformed frame to a reference frame, expressed by ⁇ given by relationship ( 1 ) above.
  • both Green-Lagrange strains and Euler-Almansi strains are determined.
  • a two-dimensional (2D) plane tangent to the body skin at each monitored position is generated.
  • the two-dimensional (2D) plane is created by first averaging the normal vectors to the planes between each neighboring pair of strain vectors. This new "average" normal of, for example, eight (8) planes defines the normal vector to the tangential plane created at the monitored position. It should be appreciated that other techniques might also be used to compute this plane.
  • the 3D strains are projected onto the 2D tangent planes.
  • the directions of principal strain and non-extension are described as 2D vectors, such that the strain field can be determined with respect to a local 2D coordinate system on the surface of the object.
  • the local coordinate system has the basis vectors e and the global coordinate system has the basis vectors e t -.
  • the local coordinate system is defined with e 3 ' aligned along the surface normal vector, has no component along e 2 and is orthogonal to the surface normal, and e 2 ' is the remaining orthogonal direction, e 3 ' x e t ' .
  • the 3D strains are rotated onto the axes defined by the location of the monitored position and at block 1312, the rotated strains are averaged together to determine the longitudinal strain, the circumferential strain, and the shear strain.
  • the vector fields can be rotated into the global reference frame.
  • X ' is the local 2D vector field to be rotated, is the transformed global 3D vector field, and (e y - ⁇ e ) represents the direction cosines.
  • the rotation matrix R is given by:
  • the principal strains E ⁇ and Ei are determined. In one embodiment, this is accomplished via eigenvector analysis as described above.
  • principal strains and Ej are used to determine the angle of the LoNEs ( ⁇ ) in block 1320. In one embodiment, the LoNEs angles are given by:
  • Sub-process 1012 proceeds to block 1322.
  • the determined LoNEs angles are projected onto the body surface (e.g., leg, knee, arm, elbow, etc.).
  • sub-process 1012 proceeds to block 1326 where the angles are used to create non-extension vector directions that can be connected or numerically integrated to produce smooth continuous lines (e.g., contours).
  • Sub-process 1012 completes at block 1330.
  • block 1328 implements a loop to repeat blocks 1304 through 1328 until all image frames are processed. Once all image frames are processed, at block 1330, sub-process continues to block 1326 where the angles are used to create non-extension vector directions that can be connected or numerically integrated to produce smooth continuous lines (e.g., contours). Sub-process 1012 completes at block 1330.
  • FIG. 14 shows additional detail of block 1016 of FIG. 10.
  • sub- process 1016 begins.
  • fabrics for various areas of a garment are selected based on fabric properties and the range of principal strain values mapped at locations of the body under test. For example, areas experiencing low magnitudes of strain could employ more rigid materials, while areas experiencing higher magnitudes of strain might employ more flexible materials.
  • the fabrics are aligned based on the directionality of the determined strains.
  • garment seams might be located in relation to the locations of determined LoNEs. It should be appreciated that placing seam lines along LoNEs is one of many ways the LoNEs might be used and that other seam placements might be employed.
  • the overall garment is sized to a specific test subject, and at block 1412, a garment pattern and/or the garment are generated, for example by sizing and connecting the various fabrics.
  • sub-process 1016 completes.
  • FIG. 19 shows how LoNEs maps can be employed to create garments and textile patterns for custom garments (e.g., skin tight garments) and other custom wearables for individual users.
  • FIG. 19 shows a portion (e.g., an elbow) of a garment designed in accordance with described embodiments.
  • described embodiments might also be applied to the design of wearable technology systems that are placed on the body, exoskeletons, prosthetics, orthotics and other garments or systems that interface with skin.
  • a clothing designer might employ the skin strain information and the 3D visualizations to, for example, determine how to pattern the textile materials that will be used for a garment.
  • the suit In order to make a tight fitting suit that is intimately interacting with the surface of the human body with large pressures, the suit should be synonymous with a second skin.
  • the second skin should be similar to human skin that it will have a pretension that applies pressure and it should move with the body to not effect human motion.
  • the designer and engineer can be informed by skin deformation visualizations. As shown in FIG. 1 9, various materials could be used to match the magnitude of deformation of human skin. Areas with larger deformations should have materials that are less stiff so as to not adversely affect mobility.
  • the directionality of the strain field could be used to align and orient fabrics, seams or material properties.
  • the LoNEs map could be used to determine how the fabric should be fused or sewn together.
  • illustrative raw images captured by data collection system 504 images of principal strain magnitudes, principal strain directions and LoNEs determined by LoNEs processing system 508 are shown.
  • the lateral side of a test subject's elbow joint has the unique texture or speckle pattern applied and images are captured quasi -statically as the subject poses at each position.
  • LoNEs processing system 508 calculates the Green- Lagrangian strain tensor in a local coordinate system.
  • E] reaches 0.5 near the edge of the dataset and E2 varies from -0.4 to 0.4, with both strain values approaching zero as moving away from the elbow joint.
  • the posterior side of the elbow experiences tension in both principal directions.
  • the anterior side of the elbow experiences compression in the longitudinal direction but tension in the circumferential direction.
  • the directions of the principal strains are also shown.
  • the streamline of the first direction of non-extension is as shown.
  • FIG. 16 shows data for test subject A.
  • the elbow joint angles are 0, 30, 60 and 90°, increasing from left to right.
  • the data analysis is shown incrementally from top to bottom.
  • the first row shows the raw images taken from one of cameras 602a - 602d.
  • the second row shows the magnitude of the second principal Green-Lagrange strain with the surface represented in the non-deformed configuration.
  • the data shows that near the tip of the elbow the Green strain reaches 0.3.
  • the third row shows the principal directions of the strain.
  • the fourth row shows the LoNEs that were calculated for the direction of non-extension.
  • the areas of the surface without LoNEs lines indicate areas that do not have LoNE directions (e.g., areas where principal strains E ⁇ and E2 have the same sign).
  • Principal directions can also be visualized as lines using the same methodology that was applied to the directions of non-extension. These lines are referred to as the lines of principal strain.
  • FIG. 1 7 shows the LoNEs and the lines of principal strain projected onto the deformed configuration at 90° elbow flexion.
  • the lines of principal strain were calculated using the strain field at 90° elbow flexion using a uniform strain field where the magnitude of each principal direction is unity. This is to visualize the general directionality of the field. In traditional fluids, the magnitude of each vector would be taken into account for the streamline calculation.
  • principal strains E] and E2 near the olecranon approach 0.5 and 0.3 respectively.
  • E2 approaches a minimum of -0.4 within the cubital fossa.
  • LoNEs are assumed to remain at a consistent length throughout deformation. To check this assumption, the LoNEs calculated at 90° elbow flexion were projected to all other deformations. The Euclidean arc length was calculated for each LoNE at each deformation. For comparison, the same process was earned out for the lines of principal strain. The sample size was 54 LoNEs and 46 lines of principal strain, where 8 lines of principal strain were not considered because they could not be successfully projected. The LoNEs remain with 6% of their original length at 0° elbow flexion whereas lines along the principal directions changed 17%. Indeed LoNEs change length when examined throughout the deformation, but these lines undergo a minimum amount of extension when compared to other lines that could be drawn on the surface.
  • the LoNEs for test subjects A-F are shown in FIG. 18.
  • the LoNEs were calculated when the elbow joint was at 90° for each subject using similar locations for the seed points.
  • the LoNEs are shown in the undeformed configuration.
  • the strain field was measured near 1 mm 2 resolution using 3D DIC and the LoNEs were calculated as continuous streamlines using seed points. As shown in FIG. 18, despite varying subject anthropometrics, the LoNEs maps look similar between subjects. Further, FIG. 18 shows that one set of LoNEs consistently converge along the
  • brachioradialis muscle might, for example during sub- process 1014, predict the LoNE map of an individual test subject in order to develop custom garments.
  • described embodiments measure human skin strain using 3D DiC at a spatial resolution on the order of 1mm 2 .
  • the millimeter scale mesh size is achieved by how DIC computes the displacements where images are broken down into small subsets of pixels. Subsets overlap, increasing resolution beyond dividing the image by subset size.
  • the streamline approach is a continuous approach that is less affected by data resolution, mesh type, or mesh connectivity than other connection methods. The streamline ending location becomes more variable the further the vector field is integrated away from the seed point, so seed point selection is done to maintain short streamline segments.
  • Described embodiments might also be implemented in the form of program code embodied in tangible media, such as magnetic recording media, hard drives, floppy diskettes, magnetic tape media, optical recording media, compact discs (CDs), digital versatile discs (DVDs), solid state memory, hybrid magnetic and solid state memoiy, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the claimed invention.
  • tangible media such as magnetic recording media, hard drives, floppy diskettes, magnetic tape media, optical recording media, compact discs (CDs), digital versatile discs (DVDs), solid state memory, hybrid magnetic and solid state memoiy, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the claimed invention.
  • Described embodiments might also be implemented in the form of program code, for example, whether stored in a storage medium, loaded into and/or executed by a machine, or transmitted over some transmission medium or carrier, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the claimed invention.
  • program code segments When implemented on a processing device, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits.
  • Such processing devices might include, for example, a general purpose microprocessor, a digital signal processor (DSP), a reduced instruction set computer (RISC), a complex instruction set computer (CISC), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic array (PLA), a microcontroller, an embedded controller, a multi-core processor, and/or others, including combinations of the above.
  • DSP digital signal processor
  • RISC reduced instruction set computer
  • CISC complex instruction set computer
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • PDA programmable logic array
  • microcontroller an embedded controller
  • multi-core processor a multi-core processor
  • Coupled refers to any manner known in the art or later developed in which energy is allowed to be transferred between two or more elements, and the interposition of one or more additional elements is contemplated, although not required. Conversely, the terms “directly coupled,” “directly connected,” etc., imply the absence of such additional elements. Signals and corresponding nodes or ports may be referred to by the same name and are interchangeable for purposes here.
  • compatible means that the element communicates with other elements in a manner wholly or partially specified by the standard, and would be recognized by other elements as sufficiently capable of communicating with the other elements in the manner specified by the standard.
  • the compatible element does not need to operate internally in a manner specified by the standard.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Remote Sensing (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

Selon des modes de réalisation, la présente invention concerne un système et un procédé de mesure de déformation et de tension de surface à l'aide d'une corrélation d'images numériques d'une surface d'un objet de test. Un système d'acquisition de données acquiert des images de la surface. La surface présente un motif de surface unique en vue de faciliter l'acquisition d'images. Les images sont regroupées en un ou plusieurs ensembles d'images. Une corrélation d'images en trois dimensions est réalisée sur chacun des ensembles d'images en vue de déterminer des données de déformation et de tension. Les données de déformation et de tension provenant des ensembles d'images sont assemblées en un ensemble de données. Des directions de tensions principales et de lignes de non-extension (LoNE) sont déterminées. Une ou plusieurs lignes LoNE et lignes d'extension maximale et minimale sont déterminées. Des visualisations des intensités de tension, des lignes LoNE, des lignes d'extension minimale et maximale sont générées en trois dimensions.
PCT/US2015/053978 2011-10-17 2015-10-05 Corrélation d'images numériques pour la mesure de tension et de déformation de revêtement WO2016076975A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/514,297 US10555697B2 (en) 2011-10-17 2015-10-05 Digital image correlation for measuring skin strain and deformation

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201462079820P 2014-11-14 2014-11-14
US62/079,820 2014-11-14
US14/837,455 US10028697B2 (en) 2011-10-17 2015-08-27 System and method for measuring skin movement and strain and related techniques
US14/837,455 2015-08-27

Publications (1)

Publication Number Publication Date
WO2016076975A1 true WO2016076975A1 (fr) 2016-05-19

Family

ID=55954824

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2015/053978 WO2016076975A1 (fr) 2011-10-17 2015-10-05 Corrélation d'images numériques pour la mesure de tension et de déformation de revêtement

Country Status (1)

Country Link
WO (1) WO2016076975A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018073174A1 (fr) * 2016-10-18 2018-04-26 Koninklijke Philips N.V. Électrode segmentée

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4969106A (en) * 1989-02-27 1990-11-06 Camsys, Inc. Computerized method of determining surface strain distributions in a deformed body
US5757473A (en) * 1996-11-13 1998-05-26 Noranda, Inc. Optical strain sensor for the measurement of microdeformations of surfaces
WO2011109029A1 (fr) * 2010-03-04 2011-09-09 Vision Optimization, Llc Procédé et appareil de détermination de caractéristiques de déformation dynamique d'objet
US20140277739A1 (en) * 2013-03-15 2014-09-18 Sri International Exosuit System
US20140311187A1 (en) * 2013-03-15 2014-10-23 Ministry Of Supply Performance dress sock

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4969106A (en) * 1989-02-27 1990-11-06 Camsys, Inc. Computerized method of determining surface strain distributions in a deformed body
US5757473A (en) * 1996-11-13 1998-05-26 Noranda, Inc. Optical strain sensor for the measurement of microdeformations of surfaces
WO2011109029A1 (fr) * 2010-03-04 2011-09-09 Vision Optimization, Llc Procédé et appareil de détermination de caractéristiques de déformation dynamique d'objet
US20140277739A1 (en) * 2013-03-15 2014-09-18 Sri International Exosuit System
US20140311187A1 (en) * 2013-03-15 2014-10-23 Ministry Of Supply Performance dress sock

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
BETHKE.: "The second skin approach : skin strain field analysis and mechanical counter pressure prototyping for advanced spacesuit design;", 2005, pages 1 - 148, Retrieved from the Internet <URL:http://dspace.mit.edu/bitstream/handle/1721.1/32443/61719483-MIT.pdf?sequence=2> *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018073174A1 (fr) * 2016-10-18 2018-04-26 Koninklijke Philips N.V. Électrode segmentée
CN109152546A (zh) * 2016-10-18 2019-01-04 皇家飞利浦有限公司 分段的电极
JP2019534070A (ja) * 2016-10-18 2019-11-28 コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． セグメント化された電極
CN109152546B (zh) * 2016-10-18 2019-11-29 皇家飞利浦有限公司 分段的电极
US11116434B2 (en) 2016-10-18 2021-09-14 Koninklijke Philips N.V. Segmented elctrode

Similar Documents

Publication Publication Date Title
US10555697B2 (en) Digital image correlation for measuring skin strain and deformation
Solav et al. MultiDIC: An open-source toolbox for multi-view 3D digital image correlation
Joo et al. Total capture: A 3d deformation model for tracking faces, hands, and bodies
Von Marcard et al. Human pose estimation from video and imus
Endo et al. Dhaiba: development of virtual ergonomic assessment system with human models
Baek et al. Parametric human body shape modeling framework for human-centered product design
CN112861598B (zh) 用于人体模型估计的系统和方法
WO2017141344A1 (fr) Système de génération de modèle tridimensionnel, procédé de génération de modèle tridimensionnel et programme
Du et al. High-resolution 3-dimensional contact deformation tracking for fingervision sensor with dense random color pattern
Arora et al. Design and fabrication of 3D fingerprint targets
CN104700452B (zh) 一种面向任意姿态的三维人体姿态模型匹配方法
CN101750029A (zh) 基于三焦点张量的特征点三维重建方法
Seo et al. A comparative study of in-field motion capture approaches for body kinematics measurement in construction
US10699480B2 (en) System and method for providing reconstruction of human surfaces from orientation data
CN106164911B (zh) 映射穿戴者移动性以用于服装设计
Obropta et al. A comparison of human skin strain fields of the elbow joint for mechanical counter pressure space suit development
US20210166478A1 (en) Methods and systems for computer-based prediction of fit and function of garments on soft bodies
Kaashki et al. Deep learning-based automated extraction of anthropometric measurements from a single 3-D scan
CN106155299A (zh) 一种对智能设备进行手势控制的方法及装置
Haouchine et al. Monocular 3D reconstruction and augmentation of elastic surfaces with self-occlusion handling
Wang et al. Facial feature extraction in an infrared image by proxy with a visible face image
Chiu et al. Automated body volume acquisitions from 3D structured-light scanning
Škorvánková et al. Automatic estimation of anthropometric human body measurements
Solav et al. Duodic: 3d digital image correlation in Matlab
WO2016076975A1 (fr) Corrélation d&#39;images numériques pour la mesure de tension et de déformation de revêtement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15858886

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15858886

Country of ref document: EP

Kind code of ref document: A1