CN113706721A - Elevator inspection method and system based on augmented reality technology - Google Patents
Elevator inspection method and system based on augmented reality technology Download PDFInfo
- Publication number
- CN113706721A CN113706721A CN202111044393.2A CN202111044393A CN113706721A CN 113706721 A CN113706721 A CN 113706721A CN 202111044393 A CN202111044393 A CN 202111044393A CN 113706721 A CN113706721 A CN 113706721A
- Authority
- CN
- China
- Prior art keywords
- information
- virtual
- image
- elevator
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 96
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000005516 engineering process Methods 0.000 title claims abstract description 45
- 230000003190 augmentative effect Effects 0.000 title claims abstract description 24
- 230000004927 fusion Effects 0.000 claims abstract description 34
- 230000003993 interaction Effects 0.000 claims abstract description 31
- 230000002159 abnormal effect Effects 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims description 18
- 230000010365 information processing Effects 0.000 claims description 17
- 230000002452 interceptive effect Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 description 28
- 230000006870 function Effects 0.000 description 15
- 238000011156 evaluation Methods 0.000 description 9
- 239000011521 glass Substances 0.000 description 5
- 238000012423 maintenance Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000007405 data analysis Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000009897 systematic effect Effects 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 239000003638 chemical reducing agent Substances 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012854 evaluation process Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computer Graphics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Geometry (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Hardware Design (AREA)
- Indicating And Signalling Devices For Elevators (AREA)
Abstract
The invention relates to an elevator inspection method and system based on augmented reality technology, wherein the method comprises the following steps: acquiring image information of an object to be detected in an elevator; carrying out similarity identification on the image information and a point cloud map in a virtual information storage system to determine a virtual image of the object to be detected; sending the virtual image to a virtual-real image fusion display interaction system; acquiring real information of the object to be detected fed back by an operator; and judging whether the object to be detected is abnormal or not according to the virtual image and the real information of the object to be detected. The invention can improve the accuracy of elevator inspection.
Description
Technical Field
The invention relates to the field of elevator inspection, in particular to an elevator inspection method and system based on an augmented reality technology.
Background
At present, the cardinal number of elevators in China is large, the self-protection rate of original factories is low, the shortage of workers in inspection industry is serious, in the process of traditional elevator inspection, the problems of various steps, data loss and the like exist, and potential safety hazards of the elevators, which are generated due to the error of inspectors and the fact that standardized operation is not carried out, also exist. Digitization of elevator inspection is a trend that is currently developing, and therefore, there is a need to relate to a method that can assist inspection work with AR equipment.
Disclosure of Invention
The invention aims to provide an elevator inspection method and system based on an augmented reality technology so as to improve the accuracy of elevator inspection.
In order to achieve the purpose, the invention provides the following scheme:
an elevator inspection method based on augmented reality technology comprises the following steps:
acquiring image information of an object to be detected in an elevator;
carrying out similarity identification on the image information and a point cloud map in a virtual information storage system to determine a virtual image of the object to be detected;
sending the virtual image to a virtual-real image fusion display interaction system;
acquiring real information of the object to be detected fed back by an operator;
and judging whether the object to be detected is abnormal or not according to the virtual image and the real information of the object to be detected.
Optionally, before the obtaining of the image information of the object to be inspected in the elevator, the method further includes:
scanning the elevator to obtain an elevator scanning image;
and solving and constructing a coordinate system according to the elevator scanning image to obtain a point cloud map.
Optionally, the determining, according to the virtual image and the real information of the object to be detected, whether the object to be detected is abnormal specifically includes:
when the real information is that the virtual image is the same as the object to be detected, determining that the object to be detected is abnormal;
and when the real information is that the virtual image is different from the object to be detected, determining that the object to be detected is abnormal, acquiring related information of the object to be detected in a virtual information database and a suspected fault database, and sending the related information to a virtual-real image fusion display interactive system.
Optionally, after the obtaining of the relevant information of the object to be detected in the virtual information database and the problematic fault database and the sending of the relevant information to the virtual-real image fusion display interaction system, the method further includes:
and storing the real information and the related information to the problematic fault database.
An elevator inspection system based on augmented reality technology, comprising: the system comprises an image acquisition processing system, an information processing system and a virtual and real image fusion display interaction system;
the image acquisition and processing system is used for acquiring image information of an object to be detected in the elevator; the system is also used for scanning the elevator to obtain an elevator scanning image;
the information processing system is used for carrying out similarity identification on the image information and a point cloud map in a virtual information storage system, and determining a virtual image of the object to be detected; the virtual image fusion display interaction system is also used for sending the virtual image to the virtual and real image fusion display interaction system; the virtual image is used for acquiring the real information of the object to be detected;
and the virtual and real image fusion display interactive system is used for displaying the virtual image to an operator.
Optionally, the system further comprises a tracking and positioning system, wherein the tracking and positioning system is used for solving and constructing a coordinate system according to the elevator scanning image acquired by the image acquisition and processing system to obtain a point cloud map.
Optionally, the system further comprises a virtual information database; the virtual information database is used for storing the virtual image.
Optionally, a problematic fault database is also included; and the problematic fault database is used for storing the real information and the related information.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the elevator inspection method and system based on the augmented reality technology provided by the invention have the advantages that the similarity recognition is carried out on the image information and a point cloud map in a virtual information storage system, and the virtual image of the object to be inspected is determined; sending the virtual image to a virtual-real image fusion display interaction system; acquiring real information of the object to be detected fed back by an operator; and judging whether the object to be detected is abnormal or not according to the virtual image and the real information of the object to be detected. The operator can compare the virtual image in the virtual-real image fusion display interactive system with the object to be detected, so that the accuracy of elevator inspection can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
Fig. 1 is a flow chart of an elevator inspection method based on augmented reality technology provided by the invention;
FIG. 2 is a block diagram of the system according to the present invention;
fig. 3 is a schematic diagram of the operation of the system provided by the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide an elevator inspection method and system based on an augmented reality technology so as to improve the accuracy of elevator inspection.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
As shown in fig. 1, the elevator inspection method based on augmented reality technology provided by the present invention includes:
step 101: and acquiring image information of an object to be detected in the elevator.
Step 102: and carrying out similarity identification on the image information and a point cloud map in a virtual information storage system, and determining a virtual image of the object to be detected.
Step 103: and sending the virtual image to a virtual-real image fusion display interactive system.
Step 104: and acquiring the real information of the object to be detected fed back by the operator. And the operator feeds back the object to be detected according to the virtual image displayed by the virtual-real image fusion display interactive system.
Step 105: and judging whether the object to be detected is abnormal or not according to the virtual image and the real information of the object to be detected. In practical application, the determining whether the object to be detected is abnormal according to the virtual image and the real information of the object to be detected specifically includes: when the real information is that the virtual image is the same as the object to be detected, determining that the object to be detected is abnormal; and when the real information is that the virtual image is different from the object to be detected, determining that the object to be detected is abnormal, acquiring related information of the object to be detected in a virtual information database and a suspected fault database, and sending the related information to a virtual-real image fusion display interactive system.
In practical application, before the image information of the object to be detected in the elevator is obtained, the method further comprises the following steps:
and scanning the elevator to obtain an elevator scanning image.
And solving and constructing a coordinate system according to the elevator scanning image to obtain a point cloud map.
In practical application, after the obtaining of the relevant information of the object to be detected in the virtual information database and the problematic fault database and the sending of the relevant information to the virtual-real image fusion display interaction system, the method further comprises:
and storing the real information and the related information to the problematic fault database.
The invention also provides an elevator inspection system based on augmented reality technology, comprising: the system comprises an image acquisition processing system, an information processing system and a virtual and real image fusion display interaction system.
The image acquisition and processing system is used for acquiring image information of an object to be detected in the elevator; and the system is also used for scanning the elevator to obtain an elevator scanning image.
The information processing system is used for carrying out similarity identification on the image information and a point cloud map in a virtual information storage system, and determining a virtual image of the object to be detected; the virtual image fusion display interaction system is also used for sending the virtual image to the virtual and real image fusion display interaction system; and the virtual image is used for acquiring the real information of the object to be detected and the virtual image, and the virtual image is used for acquiring the real information of the object to be detected.
And the virtual and real image fusion display interactive system is used for displaying the virtual image to an operator.
In practical application, the elevator inspection system based on the augmented reality technology further comprises a tracking and positioning system, and the tracking and positioning system is used for solving and constructing a coordinate system according to the elevator scanning image acquired by the image acquisition and processing system to obtain a point cloud map.
In practical application, the elevator inspection system based on the augmented reality technology further comprises a virtual information database; the virtual information database is used for storing the virtual image.
In practical application, the elevator inspection system based on the augmented reality technology further comprises a difficult and complicated fault database; and the problematic fault database is used for storing the real information and the related information.
As shown in fig. 2, the system is mainly divided into 7 subsystems, which are: the system comprises an image acquisition processing system, a tracking and positioning system, a virtual information storage system, namely a virtual information database, a management evaluation system, a difficult and complicated fault database, a virtual-real image fusion display interaction system and an information processing system.
(1) The image acquisition processing system comprises:
the main functions are as follows: data acquisition is carried out on a real scene through 3D Sensing (a camera module and a sensor module), the acquired information is identified and processed, and a judgment result and data are transmitted to an information processing system.
Secondly, key technology: the core of the 3D Sensing technology using AR functionality, wherein the optical radar system TOF scheme is used. A TX transmitting terminal in the AR equipment transmits an optical pulse signal to a target identification object, and an RX receiving terminal calculates the time of the optical pulse signal transmitted from the TX transmitting terminal to the identification object and then reflected from the identification object to the RX receiving terminal, so that the distance of the object is determined and depth sensing is performed. And comparing the acquired characteristic points with the characteristic points in the template database, and judging object identification. If the matching number of the acquired feature points and the feature points in the template database exceeds a threshold value, the identification is successful, and if the matching number of the acquired feature points and the feature points in the template database does not exceed the threshold value, the identification is circulated for multiple times. The TOF scheme of the optical radar system has better effect compared with a structured light and binocular imaging scheme, can work in a dark environment and has the advantage of long recognition distance.
(2) A tracking and positioning system:
the main functions are as follows: the information obtained by the tracking and positioning system enables the virtual object to be better fused with reality, and more reliable information is provided for inspection personnel. And the generated point cloud map is stored in a map database, namely a tracking and positioning system, and the accurate positioning of the surrounding environment in the movement of the AR equipment is realized by depending on the point cloud map so as to identify the position of the part, after each identification, the more accumulated point cloud information is, the more the machine can fully recognize the environment, the identification speed of the AR equipment is further improved, and the identification precision is improved. After the positions of the parts are confirmed, the virtual information storage system and the virtual-real image fusion display interaction system are combined to carry out inspection work, and relevant information is fed back.
Secondly, key technology: the camera module and the vision sensor module collect environment information and detect and extract corresponding feature points in the image collection processing, the camera pose is calculated based on 2-D or 3-D feature matching, the environment is mapped, synchronous positioning and mapping (SLAM) are carried out, the position and the pose of an inspector are positioned by means of the AR equipment, interaction is carried out with the positions of surrounding parts, the relative positions of the parts are determined, and further inspection work is carried out by combining with other information storage systems. The elevator inspection work is higher to the required precision of AR technique, combines together SLAM and laser radar, not only improves the precision of tracking location, makes the information of gathering moreover abundanter, lets the assistance system more have the feasibility.
(3) Virtual information database:
the main functions are as follows: in the development stage, the information needed in the virtual scene is sorted and stored in a virtual information database, the information category stored in the virtual information database comprises 3D objects, texts and pictures, a large amount of information is stored, after the tracking and positioning system and the image acquisition and processing system realize part identification and positioning, the database information is read, the data information such as 3D models, parameter information, inspection standards and the like of related parts is obtained, and the comprehensive guidance prompt is given to workers, so that the elevator inspection achieves the paperless effect.
Secondly, key technology: 3D modeling is carried out through SolidWorks, the elevator structure is visualized and informationized, and 3D animation demonstration is made in each inspection step, so that inspection workers can understand the elevator structure more easily. The TSG T7001-2009 elevator supervision inspection and regular inspection rules, namely traction and forced driving of the elevator, are used for constructing a thinking guide diagram through Xmind, so that inspection workers can find needed information more easily. The elevator inspection operation is performed, and prompt is performed according to steps, so that the elevator inspection is more standardized. All relevant information is stored in the virtual information database, and the AR equipment is connected with the database through the 5G/WIFI technology.
(4) The management evaluation system comprises:
the main functions are as follows: the whole process of the operation of the inspector is recorded in the operation process record database. And the background manager obtains data from the operation record database, performs performance evaluation on the inspection workers and performs systematic management.
Secondly, key technology: and storing the inspection operation process information in an operation process record database through a 5G/WIFI technology, developing a corresponding evaluation system, and evaluating the performance of inspection workers by background management personnel through the evaluation system. The specific evaluation process is as follows: and after the inspector scans the two-dimensional code, the system automatically counts time and starts to record the whole operation process of the inspector system. Stopping timing when the system operation is finished, and classifying the system into various grades according to the difference between the time used by the current inspector and the average inspection time of the previous inspector so as to evaluate the inspectors according to the completion time; the auditor can also check the previously recorded inspection flow to know the learning condition of the software by the inspector, and accordingly evaluate the inspector.
(5) A difficult and complicated fault database:
the main functions are as follows: and (4) carrying out data analysis on the data in the operation process record database to construct a difficult and complicated fault database. When the inspection worker encounters a problem, the inspection worker can acquire information from the problem fault database, and the problem is solved efficiently.
Secondly, key technology: the MySQL is adopted to establish a database management system, after the problem is solved, the staff is checked to record the type of the problem, classification is made, such as 'machine room-tractor-turbine reducer', the reason and the solution for the problem are recorded, and when the problem in the turbine reducer is encountered next time, the problem can be searched through a problem fault database.
(6) And a virtual-real image fusion display interaction system:
the main functions are as follows: the staff operates the AR equipment through voice and gestures. Through operation, an information processing system in the AR equipment integrates information required by a user, position information of parts obtained in the tracking and positioning system, related part parameters, a related part 3D model, inspection operation standards and the like in the virtual information database transmit processed data to a display screen of the AR equipment through an AR display technology, and the processed data are fed back to inspection workers. The staff operates through pronunciation and gesture, reaches interactive effect, makes virtual information and reality fuse.
Secondly, key technology: after the positioning work is finished, the AR glasses check each part according to the operation flow, after the parts are identified by applying the image identification technology, the relative positions of the parts are well positioned by combining the positioning technology and marked by the marks, and the marks move along with the positioning of the parts. And calling the established virtual information database, and displaying information such as characters, 3D models and the like of related parts at the edge of the glasses display screen for inspection personnel to check at any time. The system is developed based on unity and developed by using easy AR SDK, achieves the function of fusing and displaying interaction of virtual and real images, and has a certain visual effect. The interaction process adopts a voice recognition technology and a gesture recognition technology, collects voices such as next item, completion and opening technical standard of assistance work and the like, compares the voices with existing audio data in a database, and adopts the instruction to complete specified operation when the similarity reaches a certain standard; for gesture recognition such as 'OK', regression and classification are completed by adopting an image recognition algorithm of deep learning, for a gesture required by inspection, a part of a gesture picture is collected as a training set, a part of the gesture picture is collected as a verification set, after positive sample preprocessing and negative sample selection are performed, training and testing of a sample are expanded, and the gesture recognition is optimized and put into use. The checking personnel completes the designated operation to make a gesture, the AR glasses execute the gesture instruction after the AR glasses are identified, man-machine interaction is generated, and the purpose that the AR equipment assists the checking work is achieved.
(7) An information processing system:
the main functions are as follows: and processing the data transmitted in the image acquisition and processing system, the tracking and positioning system, the virtual information database, the difficult and complicated fault database and the virtual and real image fusion display interaction system.
Secondly, key technology: the information processing of the system is based on a CPU and a developed algorithm of AR equipment, and all parts, namely 3D models, various standards, difficult data and processing methods, information of inspection workers and elevator conditions are stored in a developed database system. When the inspection work starts, acquiring the information of an inspector according to a login account; when the inspection work is carried out, the position of a worker and the position of an environment are positioned by applying a positioning technology, the type of a part is determined according to an image recognition technology, and data such as a 3D object and a text related to the part are acquired from a virtual information storage system to assist the worker in carrying out the inspection work; after the inspection work is finished, the technology applied to the management evaluation system registers and processes the relevant information of the inspection work, and the relevant information is imported into the database for the next calling.
Integrated data processing process of the invention
(1) In the elevator inspected by the AR technology for the first time, a tracking and positioning system in the AR equipment scans an object for many times, calculates the position of any pixel under a camera coordinate system according to RGB-D images, finds out the space coordinates of all pixel points, converts point cloud coordinates corresponding to each frame of depth map under different poses into a world coordinate system, and completes construction of a point cloud map. Therefore, the identification of the inspection environment is completed, the position of the operator in the inspection environment is determined, the generated point cloud map is stored in the map database, and the environment can be quickly identified when the elevator is inspected next time.
(2) The image acquisition and processing system identifies the inspected object and transmits an identification judgment result to the information processing system; if the judgment result is yes, the information processing system identifies the point cloud map generated by the tracking and positioning system and determines the spatial position of the identification component. Specifically, the identification is carried out through the similarity, when the similarity is larger than a set threshold, the judgment result is yes, and when the similarity is smaller than the set threshold, the judgment result is no.
(3) The information processing system calls data from the virtual information storage system, obtains the relevant verification auxiliary information of the successfully identified parts, and stores the relevant verification auxiliary information in the RAM of the AR device.
(4) And transmitting the relevant data obtained by the information processing system from the virtual information storage system to a display screen of the AR equipment through the virtual and real image fusion display interaction system, and displaying the relevant auxiliary information on an operator.
(5) And the operator performs processing through a user interaction recognition module in the virtual-real image fusion display interaction system through voice and gesture feedback, and transmits a processed result to the information processing system. The information processing system searches information required by the operating personnel from the virtual information database and the difficult and complicated fault database according to the feedback requirement of the operating personnel, transmits the information to the virtual and real image fusion display interaction system again, and feeds the information back to the operating personnel through the display screen.
(6) If the operator encounters a difficult fault in the inspection process and cannot search related information, the operator solves the problem through the help of external contact. The operation process of solving the problem of difficult faults is carried out, the shooting record is carried out and stored in the difficult fault database, and the solution of the problem can be directly obtained when the same problem is encountered next time.
(7) And (4) when the next part is inspected after the current part is inspected, repeating the steps (2) to (6) until all inspection items are finished.
(8) And the whole process camera record of the inspection, the related time and other information are stored in an operation process record database, and a background manager acquires data from the operation process record database and performs performance evaluation on the inspection operator.
As shown in fig. 3, in elevator maintenance based on augmented reality technology, the operation method is as follows:
(1) wearing glasses, and checking that a worker logs in a personal account;
(2) scanning and inspecting the two-dimensional code on the elevator to obtain the relevant information and the historical maintenance record of the elevator;
(3) the AR equipment displays a standard and standard flow of the inspection operation in the form of demonstration of characters, pictures and a 3D model, and inspection workers perform standard operation according to prompts;
(4) after one step is completed and judgment is made, selecting a 'completion' option through voice, and carrying out the next operation;
(5) if a problem is encountered in the operation process, the AR equipment can interact with the AR equipment through voice, and the AR equipment provides a solution for inspection workers by searching a problem fault database;
(6) after the inspection is finished, recording the data of the whole inspection operation process in a database;
(7) and the background manager acquires data from the operation process record database, performs performance evaluation on the field operators and manages systematically.
The method and the system provided by the invention have the following functions:
(1) the management database establishment function: in each regular inspection operation, the whole operation process of an inspector is recorded, and a management database is established, so that the platform data is easy to trace, systematic management is carried out, and the inspector is helped to conveniently check the historical maintenance and inspection records of the elevator.
(2) The fault knowledge base sorting function is as follows: on the basis of establishing a management database, a fault knowledge base is gradually formed through big data analysis so as to be convenient for rapidly solving the same problem when the same problem is encountered next time.
(3) Tracking and positioning functions: and acquiring the position and the direction of the inspector to realize space positioning and tracking so as to complete natural fusion of the virtual information and the real environment. During the use of the system, the related information of the surrounding environment is collected, corresponding virtual auxiliary information is provided according to the matching of the feature points, the pose calculation and other information, so that the positioning of the inspector in the real scene is determined, more reliable information is provided conveniently, and the identification speed and the identification precision are further improved by storing the map information in the map database.
(4) User interaction function: the real world and the virtual world are registered by means of a virtual-real fusion technology, consistency of shielding, shadow and illumination is achieved, natural interaction is supported, and an inspector can interact with the AR device through a touch pad, voice and gestures. And the AR equipment displays relevant data of a result obtained after the AR equipment calculates the current environment on the AR equipment, so that the inspection staff can obtain relevant operation information.
The method and the system provided by the invention achieve the aims of high speed, high efficiency and standardization of elevator periodic inspection by the aid of AR equipment, and solve the problems of high false detection rate, high missed detection rate and the like of the traditional elevator inspection. The advantages compared with the conventional elevator are as follows:
(1) the operation of "paperless" is: an operator does not need to hold other information equipment to inquire relevant information of equipment inspection, maintenance and repair, information of an operation instruction book, equipment history inspection and maintenance records, key technical key and the like of a complete process is displayed on the AR equipment, and the operation of the operator is guided by the interaction between reality and virtual, so that the work is more convenient.
(2) Operation standardization: the standard flow of elevator inspection is displayed on the AR equipment, so that inspection personnel can operate in a standardized way, the false inspection rate and the missed inspection rate are reduced, and the potential safety hazard of the elevator is thoroughly eliminated.
(3) Recording of problematic data: when the inspector encounters a problem in the operation process, the inspector can inquire in a difficult and complicated fault database and conduct corresponding guidance. If the problem does not exist in the difficult fault database, the current problem is recorded, and a solution is found, so that the problem can be rapidly solved when the same problem is encountered next time, and the working efficiency is improved. The system has the function of constructing a difficult and complicated fault database, extracts information from the operation process record database, performs data analysis, and records difficult and complicated problems in the inspection operation process, so that the problem that similar difficult and complicated problems are encountered by inspection operators next time can be quickly solved, and the operation efficiency is improved.
(4) And (3) overall management: the problem that the traditional elevator inspection data are difficult to trace back is solved, the whole process of inspection operation is recorded, and the elevator periodic inspection is comprehensively and systematically managed. The elevator inspection operation process is recorded in a whole-course mode and stored in an operation process recording database, and a background manager acquires information and performs performance evaluation on inspection operators, so that the elevator inspection operation process has a systematic management function. The elevator inspection system is provided with a tracking and positioning system in the elevator inspection operation process, a map is constructed in an incremental mode according to the position of the elevator inspection system, and map information is stored in a map database, so that the identification speed and the identification precision are further improved.
(5) The training period is short: the regular inspection of the elevator has higher requirements on the professional quality of inspectors, and the training period of the inspectors is longer. However, the AR technology is used for assisting elevator inspectors, the training period of the elevator inspectors can be shortened, the elevator inspectors can be put into a work post more quickly, and the problem that the elevator inspectors are in short supply at present is solved.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (8)
1. An elevator inspection method based on augmented reality technology is characterized by comprising the following steps:
acquiring image information of an object to be detected in an elevator;
carrying out similarity identification on the image information and a point cloud map in a virtual information storage system to determine a virtual image of the object to be detected;
sending the virtual image to a virtual-real image fusion display interaction system;
acquiring real information of the object to be detected fed back by an operator;
and judging whether the object to be detected is abnormal or not according to the virtual image and the real information of the object to be detected.
2. The elevator inspection method based on augmented reality technology according to claim 1, further comprising, before the obtaining of the image information of the object to be inspected in the elevator:
scanning the elevator to obtain an elevator scanning image;
and solving and constructing a coordinate system according to the elevator scanning image to obtain a point cloud map.
3. The elevator inspection method based on augmented reality technology according to claim 1, wherein the determining whether the object to be inspected is abnormal or not according to the virtual image and the real information of the object to be inspected specifically comprises:
when the real information is that the virtual image is the same as the object to be detected, determining that the object to be detected is abnormal;
and when the real information is that the virtual image is different from the object to be detected, determining that the object to be detected is abnormal, acquiring related information of the object to be detected in a virtual information database and a suspected fault database, and sending the related information to a virtual-real image fusion display interactive system.
4. The elevator inspection method based on augmented reality technology of claim 3, wherein after the obtaining of the relevant information of the object to be inspected in the virtual information database and the suspected fault database and the sending of the relevant information to the virtual-real image fusion display interaction system, the method further comprises:
and storing the real information and the related information to the problematic fault database.
5. An elevator inspection system based on augmented reality technology, comprising: the system comprises an image acquisition processing system, an information processing system and a virtual and real image fusion display interaction system;
the image acquisition and processing system is used for acquiring image information of an object to be detected in the elevator; the system is also used for scanning the elevator to obtain an elevator scanning image;
the information processing system is used for carrying out similarity identification on the image information and a point cloud map in a virtual information storage system, and determining a virtual image of the object to be detected; the virtual image fusion display interaction system is also used for sending the virtual image to the virtual and real image fusion display interaction system; the virtual image is used for acquiring the real information of the object to be detected;
and the virtual and real image fusion display interactive system is used for displaying the virtual image to an operator.
6. The elevator inspection system based on augmented reality technology of claim 5, further comprising a tracking and positioning system, wherein the tracking and positioning system is used for solving and constructing a coordinate system according to the elevator scanning image acquired by the image acquisition and processing system to obtain a point cloud map.
7. The augmented reality technology-based elevator inspection system of claim 5 further comprising a virtual information database; the virtual information database is used for storing the virtual image.
8. The augmented reality technology-based elevator inspection system of claim 5 further comprising a troubled fault database; and the problematic fault database is used for storing the real information and the related information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111044393.2A CN113706721A (en) | 2021-09-07 | 2021-09-07 | Elevator inspection method and system based on augmented reality technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111044393.2A CN113706721A (en) | 2021-09-07 | 2021-09-07 | Elevator inspection method and system based on augmented reality technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113706721A true CN113706721A (en) | 2021-11-26 |
Family
ID=78660796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111044393.2A Pending CN113706721A (en) | 2021-09-07 | 2021-09-07 | Elevator inspection method and system based on augmented reality technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113706721A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114842173A (en) * | 2022-04-15 | 2022-08-02 | 北华航天工业学院 | Augmented reality system and control method thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080264732A1 (en) * | 2005-02-18 | 2008-10-30 | Sung Sub Kim | Elevator System and Method of Monitoring Elevators by Using a Virtual Elevator Group |
| CN108615427A (en) * | 2018-04-19 | 2018-10-02 | 广州特种机电设备检测研究院 | A kind of elevator laboratory teaching system and method based on virtual reality technology |
| CN110240038A (en) * | 2019-07-19 | 2019-09-17 | 湖州市特种设备检测研究院 | A kind of elevator traction sheave sliding amount detecting device and method |
| CN112148197A (en) * | 2020-09-23 | 2020-12-29 | 北京市商汤科技开发有限公司 | Augmented reality AR interaction method and device, electronic equipment and storage medium |
| CN113728362A (en) * | 2019-02-25 | 2021-11-30 | 奈安蒂克公司 | Augmented reality moving edge computing |
-
2021
- 2021-09-07 CN CN202111044393.2A patent/CN113706721A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080264732A1 (en) * | 2005-02-18 | 2008-10-30 | Sung Sub Kim | Elevator System and Method of Monitoring Elevators by Using a Virtual Elevator Group |
| CN108615427A (en) * | 2018-04-19 | 2018-10-02 | 广州特种机电设备检测研究院 | A kind of elevator laboratory teaching system and method based on virtual reality technology |
| CN113728362A (en) * | 2019-02-25 | 2021-11-30 | 奈安蒂克公司 | Augmented reality moving edge computing |
| CN110240038A (en) * | 2019-07-19 | 2019-09-17 | 湖州市特种设备检测研究院 | A kind of elevator traction sheave sliding amount detecting device and method |
| CN112148197A (en) * | 2020-09-23 | 2020-12-29 | 北京市商汤科技开发有限公司 | Augmented reality AR interaction method and device, electronic equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| 殷勤等: "电梯三维虚拟仿真系统的研究与开发", 《机械设计与制造》 * |
| 赵艺: "住宅电梯AR检修系统研究与开发", 《中国优秀博硕士学位论文全文数据库（硕士）工程科技Ⅱ辑》 * |
| 鲍劲松 等: "《基于VR/AR的智能制造技术》", 31 August 2020 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114842173A (en) * | 2022-04-15 | 2022-08-02 | 北华航天工业学院 | Augmented reality system and control method thereof |
| CN114842173B (en) * | 2022-04-15 | 2023-08-29 | 北华航天工业学院 | Augmented reality system and control method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111722714B (en) | AR technology-based digital substation metering operation and detection auxiliary method | |
| CN109638959B (en) | Power equipment remote signaling function debugging method and system based on AR and deep learning | |
| CN112149512A (en) | Helmet wearing identification method based on two-stage deep learning | |
| CN113222184A (en) | Equipment inspection system and method based on augmented reality AR | |
| CN108711148A (en) | A kind of wheel tyre defect intelligent detecting method based on deep learning | |
| US9613328B2 (en) | Workflow monitoring and analysis system and method thereof | |
| CN112528979B (en) | Transformer substation inspection robot obstacle distinguishing method and system | |
| CN110186375A (en) | Intelligent high-speed rail white body assemble welding feature detection device and detection method | |
| CN111044149A (en) | Method and device for detecting temperature abnormal point of voltage transformer and readable storage medium | |
| CN115331002A (en) | Method for realizing remote processing of heating power station fault based on AR glasses | |
| KR20230133831A (en) | Device, method and program that automatically designs equipment lines within BIM design data | |
| CN112949457A (en) | Maintenance method, device and system based on augmented reality technology | |
| CN104316530A (en) | Part detection method and application | |
| CN113706721A (en) | Elevator inspection method and system based on augmented reality technology | |
| CN111091104A (en) | Target object protection detection method, device, equipment and storage medium | |
| CN103383240A (en) | Machine vision two-dimension detecting platform device | |
| CN113240798B (en) | Intelligent material integrity detection and configuration method based on digital twinning and AR | |
| CN114359865A (en) | Obstacle detection method and related device | |
| KR102198028B1 (en) | Position Verification Method for Equipment Layout at 3D Design of Smart Factory | |
| CN116908186A (en) | Product detection method, device, equipment and storage medium | |
| KR102603276B1 (en) | Device, method and program for assisting supevision based on XR | |
| CN116385868A (en) | Building monitoring system based on image recognition and patrol robot thereof | |
| CN114972500A (en) | Checking method, marking method, system, device, terminal, equipment and medium | |
| CN113359738A (en) | Mobile robot path planning method based on deep learning | |
| CN117726239B (en) | Engineering quality acceptance actual measurement method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20211126 |