CN109477771A - Pass through the structural healthy monitoring system of the device identification of damage based on augmented reality - Google Patents
Pass through the structural healthy monitoring system of the device identification of damage based on augmented reality Download PDFInfo
- Publication number
- CN109477771A CN109477771A CN201780027796.1A CN201780027796A CN109477771A CN 109477771 A CN109477771 A CN 109477771A CN 201780027796 A CN201780027796 A CN 201780027796A CN 109477771 A CN109477771 A CN 109477771A
- Authority
- CN
- China
- Prior art keywords
- sensor
- shm
- structure platform
- augmented reality
- transponder
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M13/00—Testing of machine parts
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M99/00—Subject matter not provided for in other groups of this subclass
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
- G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
- G06K7/10009—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation sensing by radiation using wavelengths larger than 0.1 mm, e.g. radio-waves or microwaves
- G06K7/10366—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation sensing by radiation using wavelengths larger than 0.1 mm, e.g. radio-waves or microwaves the interrogation device being adapted for miscellaneous applications
- G06K7/10376—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation sensing by radiation using wavelengths larger than 0.1 mm, e.g. radio-waves or microwaves the interrogation device being adapted for miscellaneous applications the interrogation device being adapted for being moveable
- G06K7/10386—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation sensing by radiation using wavelengths larger than 0.1 mm, e.g. radio-waves or microwaves the interrogation device being adapted for miscellaneous applications the interrogation device being adapted for being moveable the interrogation device being of the portable or hand-handheld type, e.g. incorporated in ubiquitous hand-held devices such as PDA or mobile phone, or in the form of a portable dedicated RFID reader
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Toxicology (AREA)
- Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Electromagnetism (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Testing Of Devices, Machine Parts, Or Other Structures Thereof (AREA)
- Testing Or Calibration Of Command Recording Devices (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Assess for the structural damage to structure platform and visually checks that system includes the sensor for being operatively coupled to structure platform, the structural damage of sensor evaluation structure platform or the failure of structure platform.The monitoring structural health conditions processor response for being operatively coupled to sensor determines structural damage in sensor.At least one RF transponder and associated reader determine position of the structure platform relative to user's augmented reality observation device.User's augmented reality observation device includes the camera of the image of arresting structure platform.User's augmented reality observes device and shows that the user's augmented reality that depends on of the real world images of structure platform and the structural damage of determination observes device relative to the position of structure platform and the virtual instruction of orientation.
Description
Cross reference to related applications
Nothing.
Statement in relation to federal sponsored research or exploitation
Nothing.
Technical field
The technology of this paper be related to include sensing system monitoring structural health conditions (herein referred to as " SHM "), the biography
Sensor system automatically detects the damage in the including but not limited to structure of aircraft.
Background technique
Structure platform is subjected to producing the behaviour in service for being higher than those of the regulation load of load.These situations are not being supervised
In the case where survey, structural damage can be caused after having been subjected to some service times.In addition, structure platform corrosion-vulnerable, fatigue
And accidental injury, these can be by the initiation of serviced loads, environmental aspect or accidental shock.It can be based on being abided by by operator
The maintenance plan followed detects this structural damage during schedule maintenance or during operation.
Detailed description of the invention
The described in detail below of exemplary non-limiting illustrative embodiments will be read in conjunction with the figure, in the accompanying drawings:
Fig. 1 shows the example non-limiting examples with the AR SHM system integrated；
Fig. 2 shows the non-limiting structural health monitoring technology processes based on AR of example；
Fig. 2A shows the identification of the example between SHM system and AR system；
Fig. 2 B be shown as the sensor of the SHM system of actuator or recipient example action and with application server
Communication；
Fig. 2 C shows the example in baseline signature and compared with damaging the signal between signing；
Fig. 2 D shows the example results of the communication between application server and AR system；
Fig. 2 E shows the movement and sensor position and the example results of relative position therebetween that AR system defines user；
Fig. 2 F shows the non-limiting observation presentation graphics of the example in the AR system of SHM system；
Fig. 2 G is shown without the observation presentation graphics of the example in the AR system of the SHM system of structural damage；
Fig. 2 H shows the observation presentation graphics of the example in the AR system of the SHM system with structural damage；
Fig. 3 A, Fig. 3 B, Fig. 3 C, Fig. 3 D show the traditional detailed inspection and utilization SHM-AR method in the longitudinal spliced part of fuselage
Comparison (paying attention to the instruction of Fig. 3 D virtual lesion) between the equivalent inspection executed；And
Fig. 4 shows the different phase of the non-limiting operation of example.
Specific embodiment
The technology of this paper be related to include sensing system monitoring structural health conditions (herein referred to as " SHM "), these biography
Sensor system automatically detects the damage in the including but not limited to structure of aircraft.Such SHM sensor may include Fibre Optical Sensor
Device or piezoelectric transducer；Use Lamb wave or electromechanical impedance or fiber bragg grating or the sensor of sound emission；And/or
It can be embedded into and be embedded in monitored structure platform in the structure or not.
The non-limiting technology of the example of this paper obtains result by using augmented reality (herein referred to as " AR ") application
Quick response to improve the checking process in service action.It is mounted in SHM sensor network and in AR mobile device
RFID label tag can be used for while discriminating user positioning and sensor positioning and therefore position of the discriminating user relative to sensor.
This relative position (it can be detected independently of label and image identification) can be used for the hand-held device by such as plate, all
The real world structure observed such as the wear-type device of AR goggles or glasses or relative to user is in proper position
(light can be used in it for the position of head of other AR observation devices for example based on the AR observation device and/or user that detect and orientation
, based on accelerometer, based on gyrosensor, MARG, flock of birds or other conventional tracking techniques find out) show void
Quasi- object.
In order to seek the reduction of improvements in security and maintenance cost and mistake, making great efforts exploitation can be without people
Automatic SHM (monitoring structural health conditions) system of damage is checked and detected in the case where for interference in real time.Therefore, new SHM skill
Art will lead to the early detection of the damage usually only identified by examination planning in the past.
The exploitation of monitoring structural health conditions (SHM) and damage detection technology in use has attracted a large amount of academic and industry
Researcher.Final goal is the integrality for monitoring of structures during its entire working life under operating conditions.It is logical
Crossing makes that the maintenance of explicit preemptive type minimizes and being against any misfortune property failure is come to reduce maintenance cost be in demand.
Once detecting damage in structure platform, it is however generally that, just management process is executed with true by non-destructive tests
Surely operation is to continue with still to stop operation to execute structure repair.
The non-limiting wireless repeater of example
Automatic identification can be completed by wireless system.Simple examples are transponders.For example, transponder is automatically to inquiry
The transceiver apparatus that signal (such as identification signal, radar signal or radio frequency repeater) makes a response.
Wireless identification another example is RFID (radio frequency identification), near-field communication (NFC) or low-power consumption bluetooths (BLE).This
Kind of technology provided using radio wave to object, package, animal, people, product and other objects and things be easy and it is automatic
Identification to activate and read the data storage device in the electronic tag for being readily adhered to object.RFID/NFC/BLE system packet
It includes:
Reader: it is responsible for generating and sending electronic equipment of the radio wave to inquire, receive and process data.Reader
It transmits coded radio signal and is received back the identification signal or another information sent by label.
Label: it is responsible for answering the transponder that reader is inquired by sending data to process.RFID/NFC label is logical
It often include at least two parts: (1) for storing and handling information, radio frequency (RF) signal is modulated and is demodulated, from reading
The integrated circuit of device signal collection DC electric power；(2) antenna for receiving and transmitting；It and in some cases include (3) electricity
Pond.Label can be passive, active or battery auxiliary:
ο passive label is activated by reader and is converted into electromagnetic field by reader radio wave to be used to pass through transmission
ID or other signals carry out the energy of response.
ο battery assisted passive tags have small on-board battery, and send out when the presence by RFID reader is to activate
Return ID signal.
ο active label has on-board battery and periodically transmits its ID signal (can be with passive reader or such as
The active reader of BLE operates together).
According to the international standard of foundation, the communication between label and reader can be in the various frequencies for being divided into following three types
It is carried out in rate: low frequency, high frequency and hyperfrequency.Frequency determines opereating specification and data speed.In active and battery assisted tag,
Range and data speed are generally higher.
Label can be combined with other electron component to sense simultaneously monitoring temperature, strain, radiation, health and fitness information etc..It is another can
Can application be by RFID and other system integrations to activate spare unit if necessary, thus track and monitor it is sensitive wrap up,
The patient and many other possibilities that storage is for example impacted and temperature data, monitoring are stayed at home.
The non-limiting augmented reality of example
In general, augmented reality (AR) is the object being imaged to protrude or enhance that some elements are generated by computer
Manage the view of real world.
The imaging that computer generates can include other elements related with real-world scene or data in real time.Substantially
On, augmented reality takes the existing view of real world and the information superposition that generates new computer is on it.As a result,
The technology increases the perception to real world view, thus a possibility that adding peripheral information and interaction.
Augmented reality is usually captured real world images using hardware and marks and draws additional elements, and uses software,
The software recognizes scene, handles information and coherent element is added in real world images.
In order to recognize or track the object in scene, substantially there are two methods: augmented reality and nothing based on label
Mark augmented reality.AR system based on label is by the position for recognizing scene using label in the scene.In unmarked increasing
In strong reality, any real world element is used as target to explain scene and add relevant information.It can be used herein
Any one or both.
The enhancing of structural health is monitored using AR
The non-limiting technology of the example of this paper provides the monitoring of the enhancing to structural health in an integrated manner, thus with to from distribution
The efficient access for the database that sensor network in the structure of detection information for obtaining fault of construction and accidental injury is collected
It is associated.This provide RFID label tag be installed in SHM sensor network and in AR mobile device with and meanwhile identify user with
The express user of the augmented reality device of both sensor positions responds.Such arrangement is used as the work for dynamic interaction
Tool is enable to improve the checking process of structural maintenance movement.This non-restrictive example embodiment can surmount in market
The prior art of available inspection task, to use management system to such as integrated vehicle health management (IVHM) and health
(HUMS) existing system provides additional information and analysis.
System and method for the damage check for the structure platform using monitoring structural health conditions can be integrated into increasing
In strong reality system, so that the position of structure platform and positioning are recognized simultaneously by transponder (non-restrictive example: RFID label tag), from
And allows the quick response of result and therefore the energy of checking process more preferably executed is provided in the attended operation of structure platform
Power.
Using the sensing network of monitoring structural health conditions (SHM) system of being based on, system notice has maturity and robustness
Lesion assessment.
The non-limiting output of the example of SHM system is damage characterization, can be based on:
Detection；
Positioning；And/or
Quantify (being sized).
After the characterization for determining damage, independent or augmented reality system (AR) with the SHM system integration can be
For visualizing damage.Integrated AR provides the quickly and easily assessment to structural intergrity, this is provided due to following projects
Some or all of any combination and reduce the benefit of operating cost:
Accessibility is overcome to limit；
Hide the depth of damage；
Eliminate expensive and possible destructive dismounting；
Human factor is minimized using automated data analysis；
Reduce maintenance cost.
The structure member as shown in Fig. 1, in example non-restrictive example, SHM the and AR ability of identifying system
The process of transponder starting two systems.Because the presence of non-limiting SHM system sensing damage, non-limiting AR system execute
Presentation to the observation of the presentation graphics of damage and seriousness, in a non-limiting approach, these systems are independent but collect
At, to exchange information (referring to Fig. 1).
In Fig. 1, (it is hand-held plate, smart phone or other observation devices, packet in this case to observation AR system
Include camera, RFID or other transponders and gyrosensor) user can both see " real views " of structure and will virtually tie
Structure failure is superimposed upon on it to provide augmented reality view.User therefore can be with Scan Architecture to check it and find defect.?
Position of the system (using camera, gyrosensor and/or transponder) from AR observing system in the 3d space that user is looking at
It sets when being damaged with a part of oriented detection to the fixed structure of SHM system, system can be on the display of AR system
Appropriate position overlaying structure damage or failure virtual instruction.Color coding, size, semiology etc. can be used for user
The type, the seriousness of damage, damage for indicating damage are depths etc. on the surface or in structure.
The technology can be used as entire product or realize the one of identical or other expectation targets as working together
Group part is implemented as method, system hardware, embedded firmware and/or software.
Flow chart according to fig. 2, system and method support maintenance inspector is during inspection task using in structure platform
SHM system assess and make the detection of structural damage visualize in real time.The non-limiting step of example includes:
Identification and identification between the sensor network of A.SHM system and the sensor network of AR device is (referring also to figure
2A)；
B. structural intergrity assessment (referring also to Fig. 2 C)；
The position of the sensor network of the sensor network and AR device of C.SHM system and the definition of positioning are (referring also to figure
2B)；
D. result is visualized in real time (referring also to Fig. 2 D-2H).
Between the sensor network of stage A-SHM system and the sensor network of AR device identification and identification (referring also to
Fig. 4)
In the step of method, it has been assumed that the transponder and sensor of SHM system are bonded to the table of structure member
On face, which will be for the inspection object of detection structure damage.
Moreover, it has been contemplated that database is equipped with all transponders associated with the sensor network group of SHM system,
The positioning (coordinate: x, y, z) of middle transponder and each sensor is relative to original defined in the systematic structure platform of installation
Point (coordinate: x, y and z are equal to zero).In other words, it is desirable to tested AR device (and associated user) to be located in using transponder
It comes to an end in the 3D coordinate system of structure.
In addition it already allows for and transponder is integrated to or is otherwise adhered to AR device or related to AR device
Connection is in order to provide user's location information relative to examined structure.
Maintenance test person starts scanning using AR device in structure platform.
If not identifying and recognizing in being incorporated in SHM system group and between the transponder in AR device, structure
The realtime graphic on surface only occurs in the display of AR device, damages without additional virtual object instruction structure.
If there is identification and identification in being incorporated in SHM system group and between the transponder in AR device, forward
Device preferentially transports through the wireless signal that bluetooth, Wi-Fi network or other options execute.This simultaneously startup stage B: structure is complete
The definition of the position and positioning of the sensor network of the sensor network and AR device of whole property assessment and stage C:SHM system
Process.
The assessment of stage B- structural intergrity
The starting of SHM system is executed by being incorporated in the transponder near sensor network.Then process is activated and is incorporated in
The sensor network of SHM system in structure.
The sensor network for considering SHM system, any one or all the sensors as actuator are in the knot to be checked
Guided wave is generated in the surface of structure component.Any one or all the sensors as recipient pass through structure member in these guided waves
Surface propagate after reception result (B referring to fig. 2).
Can in any one sensor or in all the sensors simultaneously or in each sensor one connect
One ground or only in unique sensor execute as actuator or recipient SHM system sensor movement (referring to
Fig. 2 B).
By sensor obtain signal be then communicated to application server, the application server can with the SHM system integration,
Perhaps can be can be integrated into AR systematic difference server or can be stand-alone application server (B referring to fig. 2).
These obtain signals by it is wired or by being executed via bluetooth, Wi-Fi network or other options it is wireless
Signal is sent to application server.
Then using signal processing algorithm to compensate since some variables are (for example, the geometric form of environment temperature, structure member
The complexity etc. of shape) influence and by guided wave obtain effect of dispersion.
The database of application server is read after sensor network is bonded to the time in structure member from SHM system
The acquisition for the signal that all the sensors of system obtain, to generate signature (herein referred to as by the situation of not structure failure
" baseline ").
System considers the baseline signature with the compensation to effect of dispersion executed for signal processing algorithm later.
The difference between signature (signature in baseline signature and the signal of real-time collecting) is carried out if stem algorithm is executed
The mathematical analysis compared.Whether these algorithm evaluations have damage (C referring to fig. 2) on structure member.
If there is structural damage, then SHM system executes other algorithms, and other algorithms calculate this damage relative to elder generation
The positioning of the origin (for example, cartesian coordinate: x, y and z are equal to zero) of preceding definition and the size, serious for determining this structural damage
The estimation of property or other characteristics.
As a result by from being sent for AR systematic difference server and can be by wired or by via bluetooth, Wi-
Wireless signal that Fi network or other options execute transmits (D referring to fig. 2).
The position of the sensor network of the sensor network and AR device of stage C-SHM system and the definition of positioning
The starting of AR system is executed by being incorporated in the transponder near sensor network.
The 3D expression of structure member is pre-registered in application server database.
Process identifies that the sensor group of SHM system is.
Next process extracts the representative of coordinate (x, y, z) and structure member in relation to SHM sensing system from database
The information of property 3D model.
Some algorithms of AR system determine user location and the system position SHM (E referring to fig. 2).
The real-time visualization of stage D- result
The image procossing that AR system executes is with the SHM system indicated in structure member with sensor network and transponder
The view (F referring to fig. 2) of system.
If SHM system does not find damage, the expression that the image procossing that AR system executes has has in structure member
The view for having the SHM system of sensor network and transponder includes message: " not having non-destructive tests evidence " (G referring to fig. 2).
If SHM system finds damage, the expression that the image procossing that AR system executes has has in structure member
The view of the SHM system of sensor network and transponder has the presentation graphics of damage and seriousness, including message: " damaging
Wound identification evidence " (H referring to fig. 2).
AR system includes the movement of the performed inspection task of user's confirmation.AR system, which provides the user with, such as checks knot
The information of fruit, date, system mistake etc..This information is stored in the external application servers of the database including maintenance plan
In.
As an example, the longitudinal spliced part of the fuselage for detailed inspection aircraft, it is expected that executing a series of removals to be able to
Close to practical structures, as indicated by Fig. 3 A-3C.In the biography that the longitudinal spliced part of example to aircraft is more than 50 hours
After system detailed inspection, it may still be desirable to reapply CIC (corrosion inhibitor compound), replace some portions damaged during process
Part (acoustic damper) and reinstall all insulators, bulkhead, panel and all souvenirs.
Using SHM-AR system, the inspection of longitudinal spliced part will not need to remove.By walking through equipped with this document
The main cabin of the aircraft of the SHM system of middle proposition, the operator with AR device can check within less than one hour.
Once transponder system identifies the AR device and neighbouring SHM sensor network that operator carries, it will just be initiated
The automatic integrity of counter structure is assessed and indicates user and SHM positioning to AR system.If in the structure exist damage or
Degrade any evidence, then the result of SHM system by allow AR device generate damage and it seriousness virtual representation (referring to
Fig. 3 D).Additionally, by the message of display such as " damage evidence is identified ".Identical process will occur inside forward direction luggage compartment,
To complete the inspection to all longitudinal spliced parts.
Although being described together the present invention together with being presently considered to most practical and preferred embodiment thing,
It is it should be understood that the present invention is not limited to the disclosed embodiments, but wants on the contrary, it is intended to cover being included in appended right
Various modifications and equivalent arrangements in the spirit and scope asked.
Claims (16)
1. carrying out the situation of evaluation structure platform by using SHM system and AR device and obtaining in the service action of structure platform
The quick response method of the result of inspection task, comprising:
A. it identifies and recognizes the sensor network of the SHM system and the sensor network of AR device；
B. evaluation structure integrality；
C. positioning of the sensor network of the SHM system relative to the AR device is defined；And
D. the visualization of the result is presented in real time relative to the positioning of the SHM system in response to the AR device.
2. according to the method described in claim 1, further comprising making having been incorporated into for the SHM system that will check object
On the surface of the structure member of body or be attached to the surface or in other ways transponder associated with the surface and
The sensor and the transponder being arranged on the AR device are associated.
3. according to the method described in claim 1, storage identifies title in the memory of application server wherein before A
And coordinate data, the coordinate data define the position of the sensor of the SHM system and the position of the transponder.
4. according to the method described in claim 1, in being bonded to SHM system group and being bonded to the AR wherein in A
Between transponder in device, by the transmission of wireless signal, the sensor network and AR device in the SHM system are executed
Identification and identification between sensor network.
5. according to the method described in claim 1, wherein in B, using automatically carrying out the detection of structural damage, positioning and big
The monitoring structural health conditions of small estimation are assessed to execute structural intergrity.
6. according to the method described in claim 5, acquired signal is sent and is stored wherein before evaluation structure situation
With the SHM system integration or with the AR system integration or independently of the AR system and the SHM systematic difference server
In.
7. according to the method described in claim 5, wherein before evaluation structure situation, acquired signal by it is wired or with
Wireless mode is sent to application server.
8. according to the method described in claim 5, executing the filtering to false signal wherein before evaluation structure situation.
9. according to the method described in claim 1, considering before the step A in application server database wherein in C
Information storage, the coordinate (x, y, z) of SHM sensing system and structure member representative 3D model, use the calculation of AR technology
Method executes the position of the sensor network of the SHM system and the sensor network of AR device and the definition of positioning.
10. according to the method described in claim 1, considering in the application server database generated by B and C wherein in D
Information storage, the real-time visualization of the result is executed using the algorithm of AR technology.
11. for evaluation structure platform situation and obtain inspection task in the service action of structure platform result it is quick
The system of response, comprising:
Structural healthy monitoring system (SHM) including multiple energy converters, including multiple sensors or pairs of actuator and sensing
Device is used for the soft of non-destructive tests for inquiring at least one described sensor to generate the device of signal and for executing analysis
Part；
Transponder, the transponder include passive or active energy converter；
Mancarried device, the mancarried device include display window, with observed and edited using computing capability image, video,
Text and electrical form are to run the 3D display with augmented reality: and
Application server, the application server include computing capability and memory for storing data.
12. system according to claim 11, wherein the sensor includes fibre optical sensor or piezoelectric transducer.
13. system according to claim 11, wherein the monitoring structural health conditions can be Lamb wave or electromechanical impedance or
Fiber bragg grating or sound emission.
14. system according to claim 11, wherein the network sensor can be embedded into the structure or
It is not embedded into the structure.
15. system according to claim 11, wherein can by it is wired or by via bluetooth, Wi-Fi network or
The wireless signal that other wireless technologys execute carrys out the communication between execution system.
16. a kind of structural damage for structure platform is assessed and visually checks system, comprising:
The sensor for being operatively coupled to or being embedded in the structure platform, structure platform described in the sensor evaluation
Structural damage or the structure platform failure；
Be operatively coupled to the monitoring structural health conditions processor of the sensor, the monitoring structural health conditions processor response in
The sensor and determine structural damage；
At least one RF transponder and associated reader, the transponder and reader determine the structure platform relative to user
The position of augmented reality observation device；
User's augmented reality observation device be operatively coupled to the monitoring structural health conditions processor and/or with the knot
The associated database of structure health monitoring processor, user's augmented reality observation device includes capturing the structure platform
The camera of image, user's augmented reality observation device show the real world images of the structure platform and the knot of determination
Structure damage depends on user's augmented reality observation device relative to the position of the structure platform and the virtual finger of orientation
Show.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US15/146,711 US10222301B2 (en) | 2016-05-04 | 2016-05-04 | Structural health monitoring system with the identification of the damage through a device based in augmented reality technology |
| US15/146,711 | 2016-05-04 | ||
| PCT/BR2017/000046 WO2017190205A1 (en) | 2016-05-04 | 2017-04-27 | Structural health monitoring system with the identification of the damage through a device based in augmented reality technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109477771A true CN109477771A (en) | 2019-03-15 |
| CN109477771B CN109477771B (en) | 2021-08-27 |
Family
ID=59053844
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201780027796.1A Active CN109477771B (en) | 2016-05-04 | 2017-04-27 | Structural health monitoring system for identifying damage through device based on augmented reality technology |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US10222301B2 (en) |
| EP (1) | EP3452798A1 (en) |
| CN (1) | CN109477771B (en) |
| BR (1) | BR112018072687B1 (en) |
| WO (1) | WO2017190205A1 (en) |
Families Citing this family (42)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101354133B1 (en) * | 2013-12-12 | 2014-02-05 | 한라아이엠에스 주식회사 | Remote place management type ballast water treatment system by augmented reality |
| US10578880B2 (en) * | 2016-06-21 | 2020-03-03 | Intel Corporation | Augmenting reality via antenna and interaction profile |
| US10671767B2 (en) | 2017-02-22 | 2020-06-02 | Middle Chart, LLC | Smart construction with automated detection of adverse structure conditions and remediation |
| US10762251B2 (en) | 2017-02-22 | 2020-09-01 | Middle Chart, LLC | System for conducting a service call with orienteering |
| US12086507B2 (en) | 2017-02-22 | 2024-09-10 | Middle Chart, LLC | Method and apparatus for construction and operation of connected infrastructure |
| US11900021B2 (en) | 2017-02-22 | 2024-02-13 | Middle Chart, LLC | Provision of digital content via a wearable eye covering |
| US11900022B2 (en) | 2017-02-22 | 2024-02-13 | Middle Chart, LLC | Apparatus for determining a position relative to a reference transceiver |
| US10824774B2 (en) | 2019-01-17 | 2020-11-03 | Middle Chart, LLC | Methods and apparatus for healthcare facility optimization |
| US11481527B2 (en) | 2017-02-22 | 2022-10-25 | Middle Chart, LLC | Apparatus for displaying information about an item of equipment in a direction of interest |
| US10467353B2 (en) | 2017-02-22 | 2019-11-05 | Middle Chart, LLC | Building model with capture of as built features and experiential data |
| US10902160B2 (en) | 2017-02-22 | 2021-01-26 | Middle Chart, LLC | Cold storage environmental control and product tracking |
| US10740502B2 (en) | 2017-02-22 | 2020-08-11 | Middle Chart, LLC | Method and apparatus for position based query with augmented reality headgear |
| US10268782B1 (en) | 2017-02-22 | 2019-04-23 | Middle Chart, LLC | System for conducting a service call with orienteering |
| US10872179B2 (en) | 2017-02-22 | 2020-12-22 | Middle Chart, LLC | Method and apparatus for automated site augmentation |
| US11194938B2 (en) | 2020-01-28 | 2021-12-07 | Middle Chart, LLC | Methods and apparatus for persistent location based digital content |
| US10628617B1 (en) | 2017-02-22 | 2020-04-21 | Middle Chart, LLC | Method and apparatus for wireless determination of position and orientation of a smart device |
| US10949579B2 (en) | 2017-02-22 | 2021-03-16 | Middle Chart, LLC | Method and apparatus for enhanced position and orientation determination |
| US11625510B2 (en) | 2017-02-22 | 2023-04-11 | Middle Chart, LLC | Method and apparatus for presentation of digital content |
| US11436389B2 (en) | 2017-02-22 | 2022-09-06 | Middle Chart, LLC | Artificial intelligence based exchange of geospatial related digital content |
| US10984146B2 (en) | 2017-02-22 | 2021-04-20 | Middle Chart, LLC | Tracking safety conditions of an area |
| US10740503B1 (en) | 2019-01-17 | 2020-08-11 | Middle Chart, LLC | Spatial self-verifying array of nodes |
| US10831945B2 (en) | 2017-02-22 | 2020-11-10 | Middle Chart, LLC | Apparatus for operation of connected infrastructure |
| US10733334B2 (en) * | 2017-02-22 | 2020-08-04 | Middle Chart, LLC | Building vital conditions monitoring |
| US11475177B2 (en) | 2017-02-22 | 2022-10-18 | Middle Chart, LLC | Method and apparatus for improved position and orientation based information display |
| US11468209B2 (en) | 2017-02-22 | 2022-10-11 | Middle Chart, LLC | Method and apparatus for display of digital content associated with a location in a wireless communications area |
| US10620084B2 (en) | 2017-02-22 | 2020-04-14 | Middle Chart, LLC | System for hierarchical actions based upon monitored building conditions |
| CN108509824B (en) * | 2017-02-24 | 2020-08-18 | 亮风台（上海）信息科技有限公司 | Article feature identification method based on AR equipment and system for checking article |
| US10515486B1 (en) | 2017-05-03 | 2019-12-24 | United Services Automobile Association (Usaa) | Systems and methods for employing augmented reality in appraisal and assessment operations |
| US10832476B1 (en) * | 2018-04-30 | 2020-11-10 | State Farm Mutual Automobile Insurance Company | Method and system for remote virtual visualization of physical locations |
| CN108647712A (en) * | 2018-05-08 | 2018-10-12 | 阿里巴巴集团控股有限公司 | Processing method, processing equipment, client and the server of vehicle damage identification |
| CN108665373B (en) * | 2018-05-08 | 2020-09-18 | 阿里巴巴集团控股有限公司 | Interactive processing method and device for vehicle loss assessment, processing equipment and client |
| EP3617849A1 (en) | 2018-08-27 | 2020-03-04 | Airbus Operations, S.L.U. | A real time virtual reality (vr) system and related methods |
| CA3114093C (en) | 2018-09-26 | 2024-06-18 | Middle Chart, LLC | Method and apparatus for augmented virtual models and orienteering |
| US11151380B2 (en) | 2019-01-30 | 2021-10-19 | International Business Machines Corporation | Augmented reality risk vulnerability analysis |
| JP7193728B2 (en) * | 2019-03-15 | 2022-12-21 | 富士通株式会社 | Information processing device and stored image selection method |
| JP7156152B2 (en) * | 2019-04-17 | 2022-10-19 | 日本電信電話株式会社 | Danger prediction method and danger prediction device |
| US10764717B1 (en) * | 2019-08-07 | 2020-09-01 | Enlighted, Inc. | Location determination of mobile devices |
| US11507714B2 (en) | 2020-01-28 | 2022-11-22 | Middle Chart, LLC | Methods and apparatus for secure persistent location based digital content |
| US11640486B2 (en) | 2021-03-01 | 2023-05-02 | Middle Chart, LLC | Architectural drawing based exchange of geospatial related digital content |
| US12041517B2 (en) | 2020-04-28 | 2024-07-16 | Peter L. Rex | Single-message electronic product and service fulfillment |
| US11989867B2 (en) | 2020-06-19 | 2024-05-21 | Peter L. Rex | Image recognition of property defects |
| EP4167723A4 (en) | 2020-06-19 | 2024-06-19 | Rex, Peter, L. | Service trust chain |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050226532A1 (en) * | 2004-04-01 | 2005-10-13 | Thompson Robert L | System and method for determining the size of an object |
| CN101470102A (en) * | 2007-12-18 | 2009-07-01 | 通用电气公司 | System and method for augmented reality inspection and data visualization |
| US20100097453A1 (en) * | 2008-10-17 | 2010-04-22 | Olympus Corporation | Monitor device for use with an endoscope apparatus |
| CN104537401A (en) * | 2014-12-19 | 2015-04-22 | 南京大学 | Reality augmentation system and working method based on technologies of radio frequency identification and depth of field sensor |
| CN104769961A (en) * | 2012-06-18 | 2015-07-08 | 总锁有限责任公司 | RFID detection system |
| KR20150138958A (en) * | 2014-05-30 | 2015-12-11 | 성균관대학교산학협력단 | N f c based plant management system |
| US20160238692A1 (en) * | 2015-02-13 | 2016-08-18 | Position Imaging, Inc. | Accurate geographic tracking of mobile devices |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170307672A9 (en) * | 2013-01-29 | 2017-10-26 | Smart Drilling And Completion, Inc. | Stable grounding system to avoid catastrophic electrical failures in fiber-reinforced composite aircraft |
| JP5991423B2 (en) * | 2013-02-21 | 2016-09-14 | 富士通株式会社 | Display device, display method, display program, and position setting system |
| US20150062123A1 (en) * | 2013-08-30 | 2015-03-05 | Ngrain (Canada) Corporation | Augmented reality (ar) annotation computer system and computer-readable medium and method for creating an annotated 3d graphics model |
| US9646431B2 (en) * | 2014-10-22 | 2017-05-09 | The Boeing Company | Augmented reality system for assessing an affected area of an aircraft |
-
2016
- 2016-05-04 US US15/146,711 patent/US10222301B2/en active Active
-
2017
- 2017-04-27 WO PCT/BR2017/000046 patent/WO2017190205A1/en unknown
- 2017-04-27 EP EP17729344.6A patent/EP3452798A1/en active Pending
- 2017-04-27 CN CN201780027796.1A patent/CN109477771B/en active Active
- 2017-04-27 BR BR112018072687-7A patent/BR112018072687B1/en active IP Right Grant
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050226532A1 (en) * | 2004-04-01 | 2005-10-13 | Thompson Robert L | System and method for determining the size of an object |
| CN101470102A (en) * | 2007-12-18 | 2009-07-01 | 通用电气公司 | System and method for augmented reality inspection and data visualization |
| US20100097453A1 (en) * | 2008-10-17 | 2010-04-22 | Olympus Corporation | Monitor device for use with an endoscope apparatus |
| CN104769961A (en) * | 2012-06-18 | 2015-07-08 | 总锁有限责任公司 | RFID detection system |
| KR20150138958A (en) * | 2014-05-30 | 2015-12-11 | 성균관대학교산학협력단 | N f c based plant management system |
| CN104537401A (en) * | 2014-12-19 | 2015-04-22 | 南京大学 | Reality augmentation system and working method based on technologies of radio frequency identification and depth of field sensor |
| US20160238692A1 (en) * | 2015-02-13 | 2016-08-18 | Position Imaging, Inc. | Accurate geographic tracking of mobile devices |
Non-Patent Citations (3)
| Title |
|---|
| ANDREA BERNASCONI ET AL: "development of a monitoring system of crack growth in bonded single-lap joints based on the strain field and visualization by augmented reality", 《THE JOURNAL OF ADHESION》 * |
| BONHYUN KOO ET AL: "Implementation of a WSN-Based structural health monitoring architecture using 3D and AR mode", 《IEICE TRANSACTIONS ON COMMUNICATIONS》 * |
| LUCA DE MARCHI ET AL: "augmented reality to support on-field post-impact maintenance operations on thin structures", 《JOURNAL OF SENSORS》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| BR112018072687A2 (en) | 2019-02-19 |
| CN109477771B (en) | 2021-08-27 |
| WO2017190205A1 (en) | 2017-11-09 |
| EP3452798A1 (en) | 2019-03-13 |
| US20170322119A1 (en) | 2017-11-09 |
| US10222301B2 (en) | 2019-03-05 |
| BR112018072687B1 (en) | 2022-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109477771A (en) | Pass through the structural healthy monitoring system of the device identification of damage based on augmented reality | |
| Xiang et al. | Wavelet‐based detection of beam cracks using modal shape and frequency measurements | |
| EP3701708B1 (en) | Machine diagnosis using mobile devices and cloud computers | |
| CN106428617A (en) | Integrated system and methods for management and monitoring of vehicles | |
| Poozesh et al. | Multicamera measurement system to evaluate the dynamic response of utility‐scale wind turbine blades | |
| CN107636702A (en) | Integrated form Asset Integrity Management System | |
| US10618532B2 (en) | Train coupler structural health monitoring system | |
| US8682508B2 (en) | Methods of identifying flight profiles in aircraft maintenance operations | |
| US11495061B2 (en) | Systems and methods for data collection from maintenance-prone vehicle components | |
| EP3333043B1 (en) | Rail inspection system and method | |
| US20130013776A1 (en) | Information processing device, information processing system, information processing method, and information storage medium | |
| CN110471376A (en) | A kind of industry spot fault detection method and equipment | |
| CN108332789A (en) | A kind of structural healthy monitoring system of train pantograph | |
| JPWO2019107420A1 (en) | Equipment management system | |
| AU2020353001A1 (en) | System and method for assessing an operating condition of an asset | |
| CN103264393A (en) | Use method of household service robot | |
| JP2018076052A (en) | Mobile disposition tool for managing aircraft having lightning-caused inconsistencies | |
| Xu et al. | Pervasive sensing technologies for facility management: a critical review | |
| JP4443247B2 (en) | Status monitoring system and status monitoring method | |
| CN102840941A (en) | System and method for monitoring stress on wind turbine blade | |
| Siddiqui et al. | IoT based railway track faults detection and localization using acoustic analysis | |
| CN105101091A (en) | Detection system applied to emergency rescue | |
| CN113962404A (en) | Information processing apparatus, information processing method, and information processing system | |
| CN115004721A (en) | System and method for object localization in an indoor environment | |
| CN109307715B (en) | Active and passive acoustic fusion detection method for storage tank bottom plate |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |