CN112947752B - Collaborative human-computer interaction control method based on intelligent equipment - Google Patents
Collaborative human-computer interaction control method based on intelligent equipment Download PDFInfo
- Publication number
- CN112947752B CN112947752B CN202110175968.8A CN202110175968A CN112947752B CN 112947752 B CN112947752 B CN 112947752B CN 202110175968 A CN202110175968 A CN 202110175968A CN 112947752 B CN112947752 B CN 112947752B
- Authority
- CN
- China
- Prior art keywords
- maintenance
- virtual
- information
- server
- personnel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000003993 interaction Effects 0.000 title claims abstract description 45
- 238000012423 maintenance Methods 0.000 claims abstract description 225
- 238000003745 diagnosis Methods 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 8
- 238000001514 detection method Methods 0.000 claims abstract description 5
- 230000002452 interceptive effect Effects 0.000 claims description 11
- 238000005516 engineering process Methods 0.000 claims description 8
- 239000000463 material Substances 0.000 claims description 5
- 239000011521 glass Substances 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 210000001145 finger joint Anatomy 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 238000012549 training Methods 0.000 description 16
- 230000006399 behavior Effects 0.000 description 7
- 230000000875 corresponding effect Effects 0.000 description 6
- 238000004088 simulation Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000001737 promoting effect Effects 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 210000000697 sensory organ Anatomy 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Human Computer Interaction (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a collaborative human-computer interaction method based on intelligent equipment, which comprises the following steps: the detection system detects that the equipment fails and then performs fault diagnosis, diagnosis information is sent to the server for processing, and the server generates a maintenance scheme according to the diagnosis information and sends the maintenance scheme to the client virtual maintenance platform for sharing; a maintenance person sends registration login or login information to a server through a client, after successful login, maintenance request information is selected to enter the virtual maintenance platform, and a virtual scene and maintenance objects in the virtual scene are selected; the maintenance personnel wear the wearable equipment to carry out maintenance operation, the wearable equipment acquires the maintenance actions of the maintenance personnel in real time and uploads the maintenance actions to the server, and the server analyzes and processes the real-time operation information of the maintenance personnel and converts the real-time operation information into virtual maintenance operation information of a virtual hand model in a virtual scene; and judging whether the maintenance operation is finished, and exiting the virtual maintenance platform after the maintenance operation is finished to finish the maintenance interaction process.
Description
Technical Field
The invention relates to the technical field of man-machine interaction, in particular to a collaborative man-machine interaction method based on intelligent equipment.
Background
Virtual reality (VR for short) is a research direction of multidisciplinary intersection with frontier, integrates various scientific technologies such as computer graphics technology, computer simulation technology, sensor technology, display technology and the like, creates a virtual information environment on a multidimensional information space, can enable a user to have immersive sense in the body, has perfect interaction capability with the environment, and is helpful for inspiring conception. The complete virtual reality system mainly comprises a plurality of subsystems such as a simulation environment system, a perception system, a man-machine interaction system, sensing equipment and the like. The man-machine interaction system refers to data of all behavior actions generated by a person, automatically analyzes and processes the data through the system, responds to the generated content in real time and feeds back the generated content to various sensory organs of the person; the sensing devices are mainly various interactive devices and sensors used in virtual reality applications, such as location, speed, temperature, etc. Man-machine interaction, man-machine interaction (HCI or HMI for short), is a study of the interactive relationship between a research system and a user. The system may be a variety of machines, or may be computer software.
With the increasing complexity of modern electromechanical products and equipment, warranty and maintenance costs have grown substantially, and maintenance work has played a very important role in maintaining the integrity and usability of electromechanical products and equipment. Therefore, maintenance problems of complex products and equipment have attracted attention in the industry. Maintenance training, implementation, evaluation and verification work of conventional equipment is usually performed on actual equipment.
The maintenance operation is carried out based on virtual reality, and an effective way is provided for solving the maintenance training problem of large equipment.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a collaborative human-computer interaction control method based on intelligent equipment.
In order to achieve the aim of the invention, the invention adopts the following technical scheme:
The disclosure provides a collaborative human-computer interaction method based on intelligent equipment, which comprises the following steps:
The detection system detects that at least one device fails, then performs fault diagnosis and sends diagnosis information to a server for processing, and the server generates a maintenance scheme according to the diagnosis information and sends the maintenance scheme to a client virtual maintenance platform for sharing;
At least one maintenance personnel sends registration login or login information to the server through the client, after successful login, the maintenance request information is selected to enter the virtual maintenance platform, and a virtual scene and maintenance objects in the virtual scene are selected;
the maintenance personnel wear wearable equipment to carry out maintenance operation, the wearable equipment acquires maintenance actions of the maintenance personnel in real time and uploads the maintenance actions to the server, and the server analyzes and processes the real-time operation information of the maintenance personnel and then converts the real-time operation information into virtual maintenance operation information of a virtual hand model in a virtual scene;
When a plurality of maintenance personnel log in a virtual maintenance platform to carry out collaborative maintenance on the same fault, the virtual maintenance platform acquires interactive behavior information among the plurality of maintenance personnel, maintenance objects and maintenance resources, shares the interactive behavior information among the plurality of maintenance personnel and controls and manages collaborative coordination among the plurality of maintenance personnel;
Judging whether the maintenance operation is completed, and when the maintenance operation is completed, exiting the current virtual scene to finish the maintenance interaction process;
The virtual maintenance platform is built through the server, the client, the wearable device and the virtual reality device; generating the virtual scene and storing the virtual scene to a server by performing gridding reconstruction on the environment information of the equipment, wherein the virtual scene comprises at least one maintenance object and at least one maintenance resource;
the wearable device includes: data glove, spatial position tracking device, stereoscopic glasses.
In an embodiment of the disclosure, further comprising:
And after finishing the maintenance interaction process, analyzing and evaluating the maintenance operation process quality of the maintenance personnel and the cooperative coordination degree of a plurality of maintenance personnel.
In an embodiment of the disclosure, further comprising:
after finishing the maintenance interaction process, uploading the maintenance interaction process to the server to be used as historical maintenance data for storage, and enabling maintenance personnel to call and learn the historical maintenance data in the historical maintenance record.
In an embodiment of the disclosure, further comprising:
at least one visitor sends registration login or login information to the server through the client, after successful login, the visitor selects viewing request information to enter the virtual maintenance platform, and virtual scenes and maintenance objects in the virtual scenes are selected to visit virtual maintenance operation of maintenance personnel.
In one embodiment of the present disclosure, the visitor may interact with the service personnel through a voice device.
In an embodiment of the disclosure, the maintenance scheme includes a maintenance processing step and a size, a shape, a material, a weight, and a corresponding carrying capacity and a corresponding disassembling capacity of each part in the maintenance object.
In one embodiment of the present disclosure, the model of the virtual hand is built by 3DS MAX software, programming software and graphic interface software, and the data glove performs information interaction.
In an embodiment of the disclosure, the data glove is electrically connected with the spatial position tracking device, the data glove acquires motion data of each finger joint part of the maintainer in real time through a sensor, the spatial position tracking device acquires hand spatial movement information of the maintainer, and the hand gesture of the maintainer is transmitted to the virtual hand in the virtual scene in real time through the motion data and the spatial movement information.
In an embodiment of the disclosure, the data glove is further configured to receive contact information of the virtual hand with the maintenance object.
In an embodiment of the present disclosure, consistency when maintenance personnel interact with virtual scenes is improved through heterogeneous data information mapping conversion, time synchronization and information compensation techniques.
The beneficial effects of the present disclosure are:
Maintenance personnel can carry out virtual maintenance operation experience to the trouble of equipment on virtual maintenance platform through wearing wearable equipment to can simulate realize with other maintenance personnel's mutual, do not consume real resource and energy, shorten maintenance training time, reduce maintenance training cost, strengthen virtual operation training's effect, have important meaning to promoting the development of digital maintenance and maintenance training technique.
Drawings
Fig. 1 is a flow chart of steps of a collaborative human-computer interaction method based on intelligent equipment.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
The embodiment of the invention provides a collaborative human-computer interaction method based on intelligent equipment, which comprises the following steps:
Step S101: the detection system detects that at least one device fails, then performs fault diagnosis and sends diagnosis information to a server for processing, and the server generates a maintenance scheme according to the diagnosis information and sends the maintenance scheme to a client virtual maintenance platform for sharing;
Step S102: at least one maintenance personnel sends registration login or login information to the server through the client, after successful login, the maintenance request information is selected to enter the virtual maintenance platform, and a virtual scene and maintenance objects in the virtual scene are selected;
Step S103: the maintenance personnel wear wearable equipment to carry out maintenance operation, the wearable equipment acquires maintenance actions of the maintenance personnel in real time and uploads the maintenance actions to the server, and the server analyzes and processes the real-time operation information of the maintenance personnel and then converts the real-time operation information into virtual maintenance operation information of a virtual hand model in a virtual scene;
Step S104: when a plurality of maintenance personnel log in a virtual maintenance platform to carry out collaborative maintenance on the same fault, the virtual maintenance platform acquires interactive behavior information among the plurality of maintenance personnel, maintenance objects and maintenance resources, shares the interactive behavior information among the plurality of maintenance personnel and controls and manages collaborative coordination among the plurality of maintenance personnel;
step S105: judging whether the maintenance operation is completed, and when the maintenance operation is completed, exiting the current virtual scene to finish the maintenance interaction process;
The virtual maintenance platform is built through the server, the client, the wearable device and the virtual reality device; generating the virtual scene and storing the virtual scene to a server by performing gridding reconstruction on the environment information of the equipment, wherein the virtual scene comprises at least one maintenance object and at least one maintenance resource;
the wearable device includes: data glove, spatial position tracking device, stereoscopic glasses.
According to the method, maintenance personnel can carry out virtual maintenance operation experience on faults of equipment on the virtual maintenance platform by wearing the wearable equipment, interaction with other maintenance personnel can be simulated, real resources and energy are not consumed, maintenance training time is shortened, maintenance training cost is reduced, the effect of virtual operation training is enhanced, and the method has important significance in promoting development of digital maintenance and maintenance training technology.
Next, each portion in the present exemplary embodiment will be described in more detail with reference to fig. 1.
Specifically, in step S101, after the detection system detects that at least one device has a fault, the fault of the device is diagnosed and diagnosis information is sent to a server for processing, a fault information base is stored in the server, the fault information base includes multiple fault types and fault maintenance schemes corresponding to the fault types, the server matches the fault types in the fault information base according to the dispute information, so as to generate a maintenance scheme corresponding to the fault types, and the maintenance scheme is sent to a virtual maintenance platform of a client for sharing.
In step S102, a serviceman who wants to or needs to perform maintenance training sends registration login or login information to the server through the client, selects to request maintenance information to enter the virtual maintenance platform after successful login, and selects a required virtual scene and a maintenance object which wants to perform maintenance after entering; the virtual scene can comprise a plurality of maintenance objects, the virtual maintenance platform can also comprise a plurality of virtual scenes, when the number of maintenance personnel required by a certain maintenance object is enough, the maintenance objects can be locked and cannot be selected by other maintenance personnel, and when the plurality of maintenance objects in a certain virtual scene are selected by the maintenance personnel with enough number of people, the virtual scene is locked and cannot be selected by other maintenance personnel.
In step S103, maintenance operations are started after the maintenance personnel wear the wearable device, the wearable device collects maintenance actions of the maintenance personnel in real time and uploads the maintenance actions to the server in real time, and the server analyzes and processes the real-time operation information of the maintenance personnel and then reflects the real-time operation information on the virtual hand model in the virtual scene.
In step S104, when a plurality of maintenance personnel are in interactive coordination with one maintenance object or when a plurality of maintenance personnel are in interactive coordination with one another in the same virtual scene for maintenance, the virtual maintenance platform acquires information of interactive behaviors among the plurality of maintenance personnel, the maintenance object and the maintenance resource in real time and displays the information in the virtual scene, and a voice interaction unit can be further arranged on the virtual platform, so that the plurality of maintenance personnel can communicate with each other in a voice manner and the maintenance behaviors are respectively matched, the maintenance efficiency is improved, and the maintenance resource comprises a plurality of maintenance tools such as a spanner, a jack, a hacksaw and the like, although the invention is not limited thereto.
In step S105, whether the non-lost job is completed or not may be determined by exiting the current virtual scene to end the maintenance interaction process when the maintenance personnel ends the maintenance operation selection or when the virtual maintenance platform detects that the maintenance scheme is completed.
The virtual maintenance platform is built through the server, the client, the wearable device and the virtual reality device; generating the virtual scene and storing the virtual scene to a server by performing gridding reconstruction on the environment information of the equipment, wherein the virtual scene comprises at least one maintenance object and at least one maintenance resource;
the wearable device may include: data glove, spatial position tracking device, stereoscopic glasses, but of course other wearable devices may be included, although not limited thereto.
In an embodiment of the present disclosure, it may further include: and after finishing the maintenance interaction process, analyzing and evaluating the maintenance operation process quality of the maintenance personnel and the cooperative coordination degree of a plurality of maintenance personnel. Specifically, after the maintenance interaction process is finished, the operation process quality of maintenance personnel, such as maintenance efficiency, maintenance action standard degree and the like, can be analyzed and evaluated, the coordination proficiency of a plurality of maintenance personnel can be analyzed and evaluated when the maintenance personnel are more than the maintenance personnel, the maintenance personnel can be scored according to the evaluation result, and the maintenance personnel with the score higher than the preset value can be qualified, so that the maintenance operation of the entity equipment can be allowed.
In an embodiment of the present disclosure, it may further include: after finishing the maintenance interaction process, uploading the maintenance interaction process to the server to be used as historical maintenance data for storage, and enabling maintenance personnel to call and learn the historical maintenance data in the historical maintenance record. And storing the historical maintenance data, and calling by a maintenance personnel to review the historical maintenance process.
In one embodiment of the disclosure, the method further comprises the steps that at least one visitor sends registration login or login information to a server through a client, after the login is successful, viewing request information is selected to enter the virtual maintenance platform, a virtual scene and maintenance objects in the virtual scene are selected to visit virtual maintenance operation of maintenance personnel. Specifically, the visitor may be a maintenance person who cannot currently participate in the virtual maintenance training, or may be a maintenance guidance teacher.
In one embodiment of the present disclosure, the visitor may interact with the service personnel through a voice device. Specifically, when the visitor is a maintenance guidance teacher, the virtual maintenance process can be guided by voice.
In an embodiment of the disclosure, the maintenance scheme may further include a maintenance processing step and a size, a shape, a material, a weight, and a corresponding size of carrying capacity and dismounting capacity of each component in the maintenance object.
In one embodiment of the present disclosure, the model of the virtual hand is built by 3DS MAX software, programming software and graphic interface software, and the data glove performs information interaction. Specifically, after the hand data measured by the data glove are mapped to the model of the resume virtual hand in the virtual scene by using 3DS MAX software.
In an embodiment of the disclosure, the data glove is electrically connected with the spatial position tracking device, the data glove acquires motion data of each finger joint part of the maintainer in real time through a sensor, the spatial position tracking device acquires hand spatial movement information of the maintainer, and the hand gesture of the maintainer is transmitted to the virtual hand in the virtual scene in real time through the motion data and the spatial movement information.
In an embodiment of the disclosure, the data glove is further configured to receive contact information of the virtual hand with the maintenance object. The interactivity and immersion of the service personnel is greatly enhanced, and the data glove may be a force feedback data glove, although not limited thereto.
In an embodiment of the present disclosure, consistency when maintenance personnel interact with virtual scenes is improved through heterogeneous data information mapping conversion, time synchronization and information compensation techniques.
The intelligent construction of the simulation virtual scene solves the problem that the existing operation is difficult to dynamically reflect the maintenance scheme, creates a controllable virtual world in the virtual environment, enables a user to enter the virtual world by means of an avatar to experience the experience of the operation, realizes the connection among a plurality of scene objects through a client, realizes a virtual community, mostly interacts with air, cannot provide more real actual operation experience for an operator, and provides more real tactile experience for the user in the operation process through a sensor by the aid of data glove; the method comprises the steps that operation input data information is acquired, analyzed, processed and sent through a client, data information and state update of a fault maintenance simulation module and a three-dimensional vision simulation module are received, a request instruction input by a user is received by equipment, the instruction is responded, corresponding actions are executed, and data communication and interaction control with other scene users are achieved; by combining virtual reality interaction, real resources and energy are not consumed, maintenance training time is shortened, maintenance training cost is reduced, and the effect of virtual operation training is enhanced; has important significance for promoting the development of digital maintenance and maintenance training technology.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Further, one skilled in the art can engage and combine the different embodiments or examples described in this specification.
The above description is illustrative of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention, but is to be accorded the full scope of the claims.
Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
Claims (10)
1. The collaborative human-computer interaction method based on the intelligent equipment is characterized by comprising the following steps of:
The detection system detects that at least one device fails, then performs fault diagnosis and sends diagnosis information to a server for processing, and the server generates a maintenance scheme according to the diagnosis information and sends the maintenance scheme to a client virtual maintenance platform for sharing;
At least one maintenance personnel sends registration login or login information to the server through the client, after the registration is successful, maintenance request information is selected to enter the virtual maintenance platform, a virtual scene and maintenance objects in the virtual scene are selected, the virtual maintenance platform comprises a plurality of virtual scenes, the virtual scenes comprise a plurality of maintenance objects, when the number of maintenance personnel required by a certain maintenance object is enough, the maintenance objects can be locked and cannot be selected by other maintenance personnel any more, and when the number of maintenance objects in a certain virtual scene is selected by maintenance personnel with enough number, the virtual scenes are locked and cannot be selected by other maintenance personnel any more;
The maintenance personnel wear the wearable equipment to carry out maintenance operation, the wearable equipment acquires the maintenance actions of the maintenance personnel in real time and uploads the maintenance actions to the server, and the server analyzes and processes the real-time operation information of the maintenance personnel and then converts the real-time operation information into virtual maintenance operation information of a virtual hand model in a virtual scene;
When a plurality of maintenance personnel log in a virtual maintenance platform to carry out collaborative maintenance on the same fault, the virtual maintenance platform acquires interactive behavior information among the plurality of maintenance personnel, maintenance objects and maintenance resources, shares the interactive behavior information among the plurality of maintenance personnel and controls and manages collaborative coordination among the plurality of maintenance personnel;
after maintenance personnel performs maintenance operation finishing selection or the virtual maintenance platform detects that the maintenance scheme is finished, judging that the maintenance operation is finished, and after the maintenance operation is finished, exiting the current virtual scene to finish the maintenance interaction process;
The virtual maintenance platform is built through the server, the client, the wearable device and the virtual reality device; generating the virtual scene and storing the virtual scene to a server by performing gridding reconstruction on the environment information of the equipment, wherein the virtual scene comprises at least one maintenance object and at least one maintenance resource;
the wearable device includes: data glove, spatial position tracking device, stereoscopic glasses.
2. The intelligent device-based collaborative human-computer interaction method according to claim 1, further comprising:
and after the maintenance operation is finished, analyzing and evaluating the quality of the maintenance operation process of the maintenance personnel and the cooperative matching degree of a plurality of maintenance personnel.
3. The intelligent device-based collaborative human-computer interaction method according to claim 2, further comprising:
after finishing the maintenance interaction process, uploading the maintenance interaction process to the server to be used as historical maintenance data for storage, and enabling maintenance personnel to call and learn the historical maintenance data in the historical maintenance record.
4. The collaborative human-computer interaction method based on intelligent equipment according to claim 3, further comprising:
at least one visitor sends registration login or login information to the server through the client, after successful login, the visitor selects viewing request information to enter the virtual maintenance platform, and virtual scenes and maintenance objects in the virtual scenes are selected to visit virtual maintenance operation of maintenance personnel.
5. The intelligent device-based collaborative human-computer interaction method according to claim 4, wherein the visitor interacts with the maintenance personnel through a voice device.
6. The collaborative human-computer interaction method based on intelligent equipment according to claim 5, wherein the maintenance scheme comprises a maintenance processing step and the size, shape, material, weight and corresponding carrying capacity and dismounting force of each part in a maintenance object.
7. The collaborative human-machine interaction method based on intelligent devices of any of claims 1-6, wherein the virtual hand model is built and information interacted with a data glove via 3DS MAX software, programming software, and graphical interface software.
8. The collaborative human-computer interaction method based on intelligent equipment according to claim 7, wherein the data glove is electrically connected with the spatial position tracking equipment, the data glove acquires motion data of each finger joint part of the maintainer in real time through a sensor, the spatial position tracking equipment acquires hand spatial movement information of the maintainer, and the hand gesture of the maintainer is transmitted to the virtual hand in the virtual scene in real time through the motion data and the spatial movement information.
9. The intelligent device-based collaborative human-machine interaction method of claim 8, wherein the data glove is further configured to receive contact information of the virtual hand with the maintenance object.
10. The intelligent device-based collaborative human-computer interaction method according to claim 9, wherein consistency of interaction between maintenance personnel and virtual scenes is improved through heterogeneous data information mapping conversion, time synchronization and information compensation technology.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110175968.8A CN112947752B (en) | 2021-02-09 | 2021-02-09 | Collaborative human-computer interaction control method based on intelligent equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110175968.8A CN112947752B (en) | 2021-02-09 | 2021-02-09 | Collaborative human-computer interaction control method based on intelligent equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112947752A CN112947752A (en) | 2021-06-11 |
| CN112947752B true CN112947752B (en) | 2024-05-07 |
Family
ID=76244567
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110175968.8A Active CN112947752B (en) | 2021-02-09 | 2021-02-09 | Collaborative human-computer interaction control method based on intelligent equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112947752B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116862481B (en) * | 2023-08-31 | 2023-12-05 | 深圳风向标教育资源股份有限公司 | Virtual scene-based vehicle maintenance evaluation method, device and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102136204A (en) * | 2011-02-25 | 2011-07-27 | 中国人民解放军第二炮兵工程学院 | Virtual maintenance distribution interactive simulation support platform of large equipment and collaborative maintenance method |
| CN104504958A (en) * | 2014-12-22 | 2015-04-08 | 中国民航大学 | Airplane digitalized coordination virtual maintenance training device and coordination maintenance method |
| CN110502119A (en) * | 2019-08-28 | 2019-11-26 | 国网上海市电力公司 | Transformer fault case virtual interactive interface method and system based on virtual reality |
| CN110599603A (en) * | 2019-09-20 | 2019-12-20 | 上海大学 | Mechanical equipment visual interaction and equipment state monitoring system and method based on augmented reality |
| KR102165692B1 (en) * | 2019-07-23 | 2020-11-04 | 한화시스템 주식회사 | Military equipment maintenance training system using a virtual reality and operating method of thereof |
-
2021
- 2021-02-09 CN CN202110175968.8A patent/CN112947752B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102136204A (en) * | 2011-02-25 | 2011-07-27 | 中国人民解放军第二炮兵工程学院 | Virtual maintenance distribution interactive simulation support platform of large equipment and collaborative maintenance method |
| CN104504958A (en) * | 2014-12-22 | 2015-04-08 | 中国民航大学 | Airplane digitalized coordination virtual maintenance training device and coordination maintenance method |
| KR102165692B1 (en) * | 2019-07-23 | 2020-11-04 | 한화시스템 주식회사 | Military equipment maintenance training system using a virtual reality and operating method of thereof |
| CN110502119A (en) * | 2019-08-28 | 2019-11-26 | 国网上海市电力公司 | Transformer fault case virtual interactive interface method and system based on virtual reality |
| CN110599603A (en) * | 2019-09-20 | 2019-12-20 | 上海大学 | Mechanical equipment visual interaction and equipment state monitoring system and method based on augmented reality |
Non-Patent Citations (1)
| Title |
|---|
| TECNOMATIX系统在核电维修中的应用研究;刘吉双;陈飞;叶清;蔡国辉;袁鑫;;设备管理与维修;20170106(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112947752A (en) | 2021-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Segura et al. | Visual computing technologies to support the Operator 4.0 | |
| Wolfartsberger et al. | A virtual reality supported 3D environment for engineering design review | |
| KR20200083004A (en) | Engine room education and training system using virtual reality using the virtual reality implementation system | |
| CN107656505A (en) | Use the methods, devices and systems of augmented reality equipment control man-machine collaboration | |
| CN114616541A (en) | Conversational AI platform with rendered graphical output | |
| Jones et al. | Towards a situated, multimodal interface for multiple UAV control | |
| Marín et al. | The UJI online robot: An education and training experience | |
| CN110142769B (en) | ROS platform online mechanical arm demonstration system based on human body posture recognition | |
| CN107257946B (en) | System for virtual debugging | |
| CN112947752B (en) | Collaborative human-computer interaction control method based on intelligent equipment | |
| CN109754654A (en) | History project based on virtual reality instructs system, method and device | |
| Morris et al. | An xri mixed-reality internet-of-things architectural framework toward immersive and adaptive smart environments | |
| Zhang et al. | Evaluation of auditory and visual feedback on task performance in a virtual assembly environment | |
| US7496411B2 (en) | Functional object data, functional object imaging system, and object data transmitting unit, object data receiving unit and managing unit for use in the functional object imaging system | |
| Su et al. | Development of an effective 3D VR-based manipulation system for industrial robot manipulators | |
| Avizzano et al. | A multi-finger haptic interface for visually impaired people | |
| CN109213306A (en) | A kind of robot remote control platform and its design method | |
| JPH11328243A (en) | Plant design support system | |
| Zhang et al. | Hubotverse: Toward internet of human and intelligent robotic things with a digital twin-based mixed reality framework | |
| Wang et al. | Construction of Human-Robot Cooperation Assembly Simulation System Based on Augmented Reality | |
| Zhang et al. | Design and application of electric power skill training platform based on virtual reality technology | |
| CN107738256A (en) | A kind of teach-by-doing apery teaching robot's programing system | |
| CN112233208B (en) | Robot state processing method, apparatus, computing device and storage medium | |
| Kirakosian et al. | Immersive simulation and training of person-to-3d character dance in real-time | |
| Liu et al. | Data and model hybrid-driven virtual reality robot operating system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |