CN106569769A - AR technology-based machine operation instruction information display method and apparatus - Google Patents
AR technology-based machine operation instruction information display method and apparatus Download PDFInfo
- Publication number
- CN106569769A CN106569769A CN201610987399.6A CN201610987399A CN106569769A CN 106569769 A CN106569769 A CN 106569769A CN 201610987399 A CN201610987399 A CN 201610987399A CN 106569769 A CN106569769 A CN 106569769A
- Authority
- CN
- China
- Prior art keywords
- machine
- information
- user
- guidance information
- technology
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005516 engineering process Methods 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 title claims abstract description 49
- 230000000007 visual effect Effects 0.000 claims abstract description 39
- 230000004927 fusion Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 4
- 239000011521 glass Substances 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides an AR technology-based machine operation instruction information display method and apparatus to provide more concrete and direct machine operation instruction information. The method comprises steps of acquiring identifying information of a designated machine in a visual field range and seen by a user via an AR device lens, extracting the operation instruction information of the machine according to the identifying information, displaying the operation instruction information on the lens in an overlaying way to enable the user to see the machine and the operation instruction information at the same time. The machine and the operation instruction information can be combined by the use of the method; the information can be overlaid and displayed in the visual field range of the user; by the use of the method, the user can operate the machine and check the operation instruction information at the same time, so accidents due to the departure of the machine upon a user checking the operation instruction information can be avoided; and therefore, the user can conveniently operate the machine and great work efficiency can be provided.
Description
Technical Field
The invention relates to the technical field of augmented reality, in particular to a method and a device for displaying machine operation guidance information based on AR technology.
Background
Augmented Reality (AR) generates a virtual environment with realistic visual, auditory, strength, touch, movement and other senses by means of a photoelectric technology, an interaction technology, a multi-sensor technology and a computer image and multimedia technology, and the virtual environment is seamlessly combined with the real world to enable information which is difficult to experience in a certain time and space range of the real world to be perceived by human senses as if the virtual information is real, so that space, time and other objective limits are broken through, and the sensory experience beyond Reality is achieved.
For some large machines with complex operations, users often need to operate the machines in combination with operation rules, however, the operation rules are generally displayed by characters and pictures, usually paper or electronic, and the users often need to check the operation rules while operating. Therefore, the existing display method of the operation guidance information is very unfavorable for guiding the operation of the user, and the guidance efficiency is low.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides the machine operation guidance information display method and device based on the AR technology, so that a user can operate a machine more conveniently, and the efficiency of checking the machine operation guidance information by the user is improved.
In a first aspect, the present invention provides a method for displaying machine operation guidance information based on AR technology, including:
acquiring identification information of a designated machine in a visual field range seen by a user through a lens of the AR equipment;
calling operation guide information of the machine according to the identification information;
displaying the operation guidance information on the lens in an overlapping manner so that the user can see the machine and the operation guidance information at the same time.
Optionally, the acquiring identification information of a specific machine in a visual field range seen by a user through lenses of the AR device includes:
taking a picture containing a specified machine in a visual field range seen by a user through a lens of the AR device; or, scanning a specified machine contained in a visual field range seen by a user through a lens of the AR equipment by adopting a three-dimensional stereo scanning technology to obtain a stereo image of the machine;
and identifying the machine from the photo or the stereoscopic image by adopting an image feature identification technology to obtain identification information of the machine.
Optionally, before the step of retrieving the operation guidance information of the machine according to the identification information, the method further includes:
judging the current operating state of the machine according to the pictures or the stereoscopic images;
the calling of the operation guidance information of the machine according to the identification information comprises the following steps:
and calling the operation guidance information of the machine starting with the current operation state of the machine as a node according to the identification information and the current operation state of the machine.
Optionally, the acquiring identification information of a specific machine in a visual field range seen by a user through lenses of the AR device includes:
scanning the code to identify the two-dimensional code or the bar code on a specified machine contained in the visual field range seen by the user through the lens of the AR device, and obtaining the identification information of the machine.
Optionally, the invoking operation guidance information of the machine according to the identification information includes:
inquiring operation guide information of the machine from a local database according to the identification information; or
And downloading the operation guide information of the machine from a remote server according to the identification information.
Optionally, the operation guidance information includes: a combination of one or more of a text description, a pictorial description, a demonstration video, a flat demonstration animation, a stereoscopic demonstration animation, a guide icon that guides a user to use the machine's operational guidance information.
Optionally, the displaying the operation guidance information on the lens in an overlapping manner so that the user can see the machine and the operation guidance information at the same time includes:
and projecting the operation guide information onto the lens by adopting a projection technology, so that the user sees a stereoscopic fusion scene of the machine and the operation guide information.
In a second aspect, the present invention provides an AR technology-based machine operation guidance information presentation apparatus, including:
the identification information acquisition module is used for acquiring identification information of a designated machine in a visual field range seen by a user through the lens of the AR equipment;
the guide information calling module is used for calling the operation guide information of the machine according to the identification information;
and the guide information display module is used for displaying the operation guide information on the lens in an overlapping mode so that the user can see the machine and the operation guide information at the same time.
Optionally, the identification information obtaining module includes:
the photographing image acquisition unit is used for photographing a picture containing a specified machine in a visual field range seen by a user through the lens of the AR equipment; or,
the scanning image acquisition unit is used for scanning a specified machine contained in a visual field range seen by a user through a lens of the AR equipment by adopting a three-dimensional scanning technology to obtain a three-dimensional image of the machine;
and the image processing unit is used for identifying the machine from the photo or the stereoscopic image by adopting an image feature identification technology and obtaining identification information of the machine.
Optionally, the AR technology-based machine operation guidance information display apparatus further includes:
the operation state judgment module is used for judging the current operation state of the machine according to the pictures or the images;
the guidance information calling module comprises:
and the partial information calling unit is used for calling the operation guide information of the machine starting from the current operation state of the machine as a node according to the identification information and the current operation state of the machine.
Optionally, the identification information obtaining module includes:
and the code scanning unit is used for scanning a two-dimensional code or a bar code on a specified machine contained in a visual field range seen by a user through the lens of the AR equipment, and acquiring the identification information of the machine.
Optionally, the guidance information retrieving module includes:
the guide information local calling unit is used for inquiring the operation guide information of the machine from a local database according to the identification information; or,
and the guiding information remote calling unit is used for downloading the operation guiding information of the machine from a remote server according to the identification information.
Optionally, the operation guidance information includes: a combination of one or more of a text description, a pictorial description, a demonstration video, a flat demonstration animation, a stereoscopic demonstration animation, a guide icon that guides a user to use the machine's operational guidance information.
Optionally, the guidance information display module includes:
and the projection unit is used for projecting the operation guide information onto the lens by adopting a projection technology so that the user sees a stereoscopic fusion scene of the machine and the operation guide information.
According to the technical scheme, the machine operation guidance information display method based on the AR technology is used for providing more specific and visual machine operation guidance information for a user, and firstly, the identification information of a specified machine in a visual field range seen by the user through the lens of the AR equipment is obtained; then, calling operation guide information of the machine according to the identification information; finally, the operation guidance information is displayed superimposed on the lens so that the user sees the machine and the operation guidance information simultaneously. The method combines the machine and the operation guidance information, and simultaneously displays the machine and the operation guidance information in a visual field range of a user in an overlapping manner, the virtual operation guidance information and the real machine are fused, so that the user can learn the machine operation guidance information more intuitively and simply, the method for displaying the machine operation guidance information can simultaneously operate the machine and check the operation guidance information, accidents caused by the fact that the user leaves the machine to check the operation guidance information are avoided, the user can operate the machine more conveniently, and the working efficiency is higher.
The machine operation guidance information display device based on the AR technology provided by the invention has the same beneficial effects as the machine operation guidance information display method based on the AR technology.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below. Throughout the drawings, like elements or portions are generally identified by like reference numerals. In the drawings, elements or portions are not necessarily drawn to scale.
Fig. 1 is a flowchart illustrating a method for presenting machine operation guidance information based on AR technology according to a first embodiment of the present invention;
fig. 2 is a schematic diagram illustrating a machine operation guidance information presentation apparatus based on AR technology according to a second embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby.
It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.
For ease of understanding, the general inventive concept of the present invention is described herein:
the invention provides a method and a device for displaying machine operation guidance information based on AR technology by taking AR equipment used by a user as an execution main body, which belong to the same invention concept on the whole, mainly utilize the technology based on AR to enable the user to see the machine operation guidance information and a real machine in the same visual field at the same time, and firstly, acquire identification information of a designated machine in the visual field range seen by the user through a lens of the AR equipment; then, calling operation guide information of the machine according to the identification information; finally, the operation guidance information is displayed superimposed on the lens so that the user sees the machine and the operation guidance information simultaneously. When a user operates a relatively complex large-scale machine, the operation steps are complex, and the operation guide information needs to be checked at any time, at the moment, the method or the device provided by the invention can be utilized to ensure that the user can see the operation guide information while operating the machine, the operation process, the operation attention and the like in the operation guide information can be virtually operated and demonstrated on the real machine without checking paper edition or electronic edition operation rules, and the method for displaying the operation guide information is very convenient and intuitive and has high guide efficiency. It should be noted that the present invention is not limited to the kind of machine, and all machines and devices are within the protection scope of the present invention.
The invention provides a machine operation guidance information display method and device based on AR technology. Embodiments of the present invention will be described below with reference to the drawings.
Fig. 1 shows a flowchart of a method for presenting machine operation guidance information based on AR technology according to a first embodiment of the present invention. As shown in fig. 1, a method for displaying machine operation guidance information based on AR technology according to a first embodiment of the present invention includes the following steps:
step S101: identification information of a designated machine within a field of view of a user through lenses of the AR device is acquired.
In the step, there are three ways to obtain the identification information of the designated machine in the visual field range seen by the user through the lens of the AR device, the first way is to shoot the photo of the designated machine contained in the visual field range seen by the user through the lens of the AR device through the camera mounted on the AR device, and to identify the machine from the photo by adopting the image feature recognition technology to obtain the identification information of the machine, in a typical AR device, a camera is arranged beside the lens, after the AR glasses are worn, the visual field shot by the camera is basically consistent with the external visual field seen by the user through the lens, therefore, when the user faces the designated machine, the picture shot by the camera contains the machine, namely, the process of shooting the photo of the designated machine contained in the visual field range seen by the user through the lens of the AR device is realized, one or more cameras can be arranged, and the cameras are all within the protection scope of the invention; in the second mode, an AR device with a three-dimensional scanning technology is used for scanning a specified machine contained in a visual field range seen by a user through a lens of the AR device to obtain a three-dimensional image of the machine, and then the machine is identified from the three-dimensional image by adopting an image feature identification technology to obtain identification information of the machine; in the third mode, a two-dimensional code or a bar code on a specified machine contained in a visual field range seen by a user through a lens of the AR device is scanned, and identification information of the machine is obtained. The same AR equipment can have one or more of the three ways of acquiring the identification information, and the ways are not mutually conflicted, so that the application range of the AR equipment is increased, the wrong identification caused by the same appearance and different models is avoided, and when the appearances are the same, the correct identification can be carried out by using a scanning or code scanning way.
The identification information refers to information that can identify the machine, such as a model, a name, a specification, a size, and the like of the machine.
Step S102: and calling operation guide information of the machine according to the identification information.
In the process of acquiring the identification information of the designated machine in the visual field range seen by the user through the lenses of the AR device in the first manner or the second manner in step S101, the current operating state of the machine may be determined according to the captured photograph or the scanned stereoscopic image, and then the operation guidance information of the machine starting with the current operating state of the machine as a node may be retrieved according to the identification information and the operating state. Repeated checking of the understood and operated operation guide information is avoided, and the operation guide information is more targeted and simpler.
The operation guidance information of the machine is called according to the identification information, the operation guidance information of the machine is inquired from a local database according to the identification information, or the operation guidance information of the machine is downloaded from a remote server according to the identification information, so that the application range of the AR technology-based machine operation guidance information display method is wider, and the AR technology-based machine operation guidance information display method can be applied to one machine and multiple machines. The operation guidance information with small occupied space can be directly stored in the local database, and the operation guidance information can be conveniently and quickly called from the local database, so that the use of a server is avoided, and the cost is saved; the operation guidance information with larger occupied space can be stored in the remote server, and can be downloaded according to the requirement when in use. In addition, in an embodiment provided by the present invention, the AR device is provided with a local storage, and is further provided with a wireless communication module connected to a remote server, the local storage stores operation guidance information for specifying several models of machines, the remote server is provided in a manufacturer of the machines, and all models of machines manufactured by the manufacturer are stored in the remote server.
Example 1: in a factory, a plurality of large machines are available, the operation of each machine is complicated, and a user generally needs guidance during operation, and at this time, the user can wear the AR device to operate the machine and watch operation guidance information. After a user wears the AR equipment, the user firstly needs to determine which machine to operate, then the AR equipment is used for scanning the machine, the user can see the machine on a lens of the AR equipment, then the AR equipment can acquire a three-dimensional image of the machine, then the three-dimensional image is identified by adopting an image identification technology, identification information of the machine, such as machine name, specification, model and the like, is acquired, and after the identification information is acquired, the AR equipment can search corresponding operation guide information of the machine from a local storage according to the identification information or download the corresponding operation guide information of the machine from a remote server. The operation guidance information may be operation flow, attention, performance parameters, etc. of the machine, which are all within the protection scope of the present invention.
Example 2: a user operates a relatively complex machine, if the operation steps of the machine include seven steps, the user operation is carried out to the second step, how to operate is uncertain in the next step, at the moment, the machine in the current operation state is photographed or scanned by utilizing AR equipment, the AR equipment identifies the machine by adopting an image identification technology according to a photograph obtained by photographing or a scanned three-dimensional image to obtain identification information, meanwhile, the AR equipment judges which step the user specifically operates according to the identification information of the machine and the photograph or the three-dimensional image of the machine in the current operation state, the user operation is judged to be carried out to the second step, the operation of the third step is carried out next, then the identification information is searched from a storage device which stores the machine operation guide information, and the operation guide information of the machine starting from the operation state as a node is called, and displaying the operation guidance information of the third step and the later steps of the machine operation on the AR equipment to provide the next operation guidance information for the user.
Step S103: displaying the operation guidance information on the lens in an overlapping manner so that the user can see the machine and the operation guidance information at the same time.
The operation guidance information in the embodiment of the present invention may be in the form of content of one or more of a text description, a picture description, a demonstration video, a plane demonstration animation, a stereoscopic demonstration animation, and a guide icon of the operation guidance information that guides the user to use the machine.
In terms of content, the operation guidance information may be guidance information of an operation flow, guidance information of an operation notice, and the like, and is within the scope of the present invention.
The operation guidance information is displayed on the lens in an overlapping mode, so that the user can see the machine and the operation guidance information at the same time, the operation guidance information can be projected onto the lens by adopting a projection technology, so that the user can see a three-dimensional fusion scene of the machine and the operation guidance information, a transparent display can be used as a lens, and the operation guidance information is displayed on the transparent display in a semitransparent mode, which is within the protection scope of the invention.
Example 3: when a user breaks down suddenly in the process of using the automobile, the automobile needs to be maintained immediately, the user can wear the AR glasses with the automobile maintenance operation guide information, and the AR glasses can guide the whole maintenance operation process of the user. After a user wears the AR glasses, firstly, the user operates the AR glasses to enable the AR glasses to take a picture or scan the car, confirm the model of the car, and then find out the cause of the car failure, for example, open the car cover in front of the car, check the parts of the car engine, the radiator and the like, in the process of finding out the cause of the failure, the user can see a person with an arrow or a virtual structure on the AR glasses to indicate the user to open the car cover in front of the car, a virtual wrench can appear to instruct the user which screw should be screwed, which direction the user is screwed, to prompt the user where the problem occurs, after finding out the problem that the failure occurs, the user can obtain the guidance information how to perform maintenance from the AR glasses, for example, the arrow points to an electric wire, and simultaneously there is a prompt sound or text indicating that the electric wire is broken, next, there will be a voice or text together with the guiding arrows to guide the user how the wires should be connected to complete the repair work of the fault.
Through the superposition display of the real machine and the operation guidance information, the operation guidance information can be presented in front of a user more friendly, and the user can see the three-dimensional fusion scene of the machine and the operation guidance information, so that the user can feel the content of the operation guidance information which is 3D, informationized, concretized and simplified more truly, the content of the operation guidance information can be better understood, the guidance effect of the operation guidance information on the user can be effectively improved, the guidance efficiency is improved, and better user experience is obtained.
The user utilizes this kind of method to learn machine operation guide information, can oneself accomplish whole study independently, does not need experienced master to explain again, and unclear place can be watched repeatedly, like this, has not only saved the time of having experienced master to teach, and in addition, the user can wear when operating the machine AR glasses remind the user at any time the operation items, operation flow etc. that need notice, can prevent because carelessness, the omission individual step that causes of factors such as carelessness, distraction, obscure, have avoided some unnecessary losses or injury. Meanwhile, by the learning method for displaying the operation guidance information based on the AR equipment, a user can learn the operation guidance information of the machine at any time, the learning progress of the user can be accelerated, the learning efficiency is improved, and certain help is provided for improving the production efficiency of a factory.
In the first embodiment described above, a method for displaying machine operation guidance information based on AR technology is provided, and in response, the present application also provides a device for displaying machine operation guidance information based on AR technology. Please refer to fig. 2, which is a schematic diagram of a device for displaying machine operation guidance information based on AR technology according to a second embodiment of the present invention. Since the apparatus embodiments are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.
A second embodiment of the present invention provides an AR technology-based machine operation guidance information display apparatus, including:
an identification information acquisition module 101, configured to acquire identification information of a specified machine within a visual field range seen by a user through lenses of the AR device;
a guidance information retrieving module 102, configured to retrieve operation guidance information of the machine according to the identification information;
a guidance information display module 103, configured to display the operation guidance information on the lens in an overlapping manner, so that the user can see the machine and the operation guidance information at the same time.
In an embodiment provided by the present invention, the identification information obtaining module 101 includes:
the photographing image acquisition unit is used for photographing a picture containing a specified machine in a visual field range seen by a user through the lens of the AR equipment; or,
the scanning image acquisition unit is used for scanning a specified machine contained in a visual field range seen by a user through a lens of the AR equipment by adopting a three-dimensional scanning technology to obtain a three-dimensional image of the machine;
and the image processing unit is used for identifying the machine from the photo or the stereoscopic image by adopting an image feature identification technology and obtaining identification information of the machine.
In an embodiment of the present invention, the AR technology-based machine operation guidance information presentation apparatus further includes:
the operation state judgment module is used for judging the current operation state of the machine according to the pictures or the images;
the guidance information retrieving module 102 includes:
and the partial information calling unit is used for calling the operation guide information of the machine starting from the current operation state of the machine as a node according to the identification information and the current operation state of the machine.
In an embodiment provided by the present invention, the identification information obtaining module 101 includes:
and the code scanning unit is used for scanning a two-dimensional code or a bar code on a specified machine contained in a visual field range seen by a user through the lens of the AR equipment, and acquiring the identification information of the machine.
In an embodiment provided by the present invention, the guidance information retrieving module 102 includes:
the guide information local calling unit is used for inquiring the operation guide information of the machine from a local database according to the identification information; or,
and the guiding information remote calling unit is used for downloading the operation guiding information of the machine from a remote server according to the identification information.
In an embodiment of the present invention, the operation guidance information includes: a combination of one or more of a text description, a pictorial description, a demonstration video, a flat demonstration animation, a stereoscopic demonstration animation, a guide icon that guides a user to use the machine's operational guidance information.
In an embodiment provided by the present invention, the guidance information display module 103 includes:
and the projection unit is used for projecting the operation guide information onto the lens by adopting a projection technology so that the user sees a stereoscopic fusion scene of the machine and the operation guide information.
The above is a description of an embodiment of the AR technology-based machine operation guidance information display apparatus according to the second embodiment of the present invention.
The machine operation guidance information display device based on the AR technology and the machine operation guidance information display method based on the AR technology provided by the invention have the same inventive concept and the same beneficial effects, and are not repeated herein.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction. It is to be noted that the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer (which may be a personal computer, a server, or a network machine) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.
Claims (10)
1. A machine operation guidance information display method based on AR technology is characterized by comprising the following steps:
acquiring identification information of a designated machine in a visual field range seen by a user through a lens of the AR equipment;
calling operation guide information of the machine according to the identification information;
displaying the operation guidance information on the lens in an overlapping manner so that the user can see the machine and the operation guidance information at the same time.
2. The AR technology-based machine operation guidance information presentation method according to claim 1, wherein the acquiring identification information of a specific machine within a visual field range seen by a user through lenses of an AR device includes:
taking a picture containing a specified machine in a visual field range seen by a user through a lens of the AR device; or, scanning a specified machine contained in a visual field range seen by a user through a lens of the AR equipment by adopting a three-dimensional stereo scanning technology to obtain a stereo image of the machine;
and identifying the machine from the photo or the stereoscopic image by adopting an image feature identification technology to obtain identification information of the machine.
3. The AR technology-based machine operation guidance information presentation method according to claim 2, further comprising, before the step of retrieving operation guidance information of the machine based on the identification information:
judging the current operating state of the machine according to the pictures or the stereoscopic images;
the calling of the operation guidance information of the machine according to the identification information comprises the following steps:
and calling the operation guidance information of the machine starting with the current operation state of the machine as a node according to the identification information and the current operation state of the machine.
4. The AR technology-based machine operation guidance information presentation method according to claim 1, wherein the acquiring identification information of a specific machine within a visual field range seen by a user through lenses of an AR device includes:
scanning the code to identify the two-dimensional code or the bar code on a specified machine contained in the visual field range seen by the user through the lens of the AR device, and obtaining the identification information of the machine.
5. The method for displaying the machine operation guidance information based on the AR technology according to claim 1, wherein the calling the operation guidance information of the machine according to the identification information includes:
inquiring operation guide information of the machine from a local database according to the identification information; or
And downloading the operation guide information of the machine from a remote server according to the identification information.
6. The AR technology-based machine operation guidance information presentation method according to claim 1, wherein the operation guidance information includes: a combination of one or more of a text description, a pictorial description, a demonstration video, a flat demonstration animation, a stereoscopic demonstration animation, a guide icon that guides a user to use the machine's operational guidance information.
7. The AR technology-based machine operation specification presentation method according to claim 1, wherein the displaying of the operation guidance information superimposed on the lens so that the user sees the machine and the operation guidance information simultaneously comprises:
and projecting the operation guide information onto the lens by adopting a projection technology, so that the user sees a stereoscopic fusion scene of the machine and the operation guide information.
8. A machine operation guidance information display device based on AR technology, comprising:
the identification information acquisition module is used for acquiring identification information of a designated machine in a visual field range seen by a user through the lens of the AR equipment;
the guide information calling module is used for calling the operation guide information of the machine according to the identification information;
and the guide information display module is used for displaying the operation guide information on the lens in an overlapping mode so that the user can see the machine and the operation guide information at the same time.
9. The AR technology-based machine operation guidance information presentation device of claim 8, wherein the identification information acquisition module includes:
the photographing image acquisition unit is used for photographing a picture containing a specified machine in a visual field range seen by a user through the lens of the AR equipment; or,
the scanning image acquisition unit is used for scanning a specified machine contained in a visual field range seen by a user through a lens of the AR equipment by adopting a three-dimensional scanning technology to obtain a three-dimensional image of the machine;
and the image processing unit is used for identifying the machine from the photo or the stereoscopic image by adopting an image feature identification technology and obtaining identification information of the machine.
10. The AR technology-based machine operation guidance information presentation device of claim 8, wherein the guidance information retrieving module comprises:
the guide information local calling unit is used for inquiring the operation guide information of the machine from a local database according to the identification information; or,
and the guiding information remote calling unit is used for downloading the operation guiding information of the machine from a remote server according to the identification information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610987399.6A CN106569769A (en) | 2016-11-09 | 2016-11-09 | AR technology-based machine operation instruction information display method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610987399.6A CN106569769A (en) | 2016-11-09 | 2016-11-09 | AR technology-based machine operation instruction information display method and apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106569769A true CN106569769A (en) | 2017-04-19 |
Family
ID=58540910
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610987399.6A Pending CN106569769A (en) | 2016-11-09 | 2016-11-09 | AR technology-based machine operation instruction information display method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106569769A (en) |
Cited By (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107065808A (en) * | 2017-05-27 | 2017-08-18 | 深圳增强现实技术有限公司 | Accessories of a product method for posting, quality detecting method and production line system |
| CN107169139A (en) * | 2017-06-13 | 2017-09-15 | 深圳增强现实技术有限公司 | A kind of document file management system based on AR augmented realities, AR mobile intelligent terminals |
| CN107277172A (en) * | 2017-07-28 | 2017-10-20 | 深圳增强现实技术有限公司 | Machine operates monitor system and its ustomer premises access equipment |
| CN107316165A (en) * | 2017-07-28 | 2017-11-03 | 深圳增强现实技术有限公司 | Machine working specification creates system and operation Modeling Server |
| CN107370821A (en) * | 2017-08-03 | 2017-11-21 | 深圳增强现实技术有限公司 | Machine Operating Guideline system, method and ustomer premises access equipment |
| CN107391707A (en) * | 2017-07-28 | 2017-11-24 | 深圳增强现实技术有限公司 | Machine Operating Guideline system and ustomer premises access equipment |
| CN107422710A (en) * | 2017-07-28 | 2017-12-01 | 歌尔股份有限公司 | The method and system of assembling product or plant maintenance are aided in based on AR |
| CN107463153A (en) * | 2017-08-11 | 2017-12-12 | 深圳增强现实技术有限公司 | A kind of management method of equipment working state, management equipment and management system |
| CN107526590A (en) * | 2017-08-14 | 2017-12-29 | 深圳乐普智能医疗器械有限公司 | Operation guidance information methods of exhibiting, device and the terminal of Medical Devices |
| CN107547554A (en) * | 2017-09-08 | 2018-01-05 | 北京枭龙科技有限公司 | A kind of smart machine remote assisting system based on augmented reality |
| CN107688331A (en) * | 2017-08-30 | 2018-02-13 | 深圳增强现实技术有限公司 | Machine operates monitor system and its communication means |
| CN107730008A (en) * | 2017-09-14 | 2018-02-23 | 珠海格力电器股份有限公司 | Device after-sale guidance method and device, computer readable storage medium and terminal |
| CN107730589A (en) * | 2017-08-18 | 2018-02-23 | 珠海格力电器股份有限公司 | Electrical appliance assembly method and device, storage device, mobile terminal and electrical appliance |
| CN107784885A (en) * | 2017-10-26 | 2018-03-09 | 歌尔科技有限公司 | Operation training method and AR equipment based on AR equipment |
| CN107977685A (en) * | 2017-12-21 | 2018-05-01 | 江西爱驰亿维实业有限公司 | Obtain the method, apparatus and electronic equipment of electronics service manual |
| CN108021991A (en) * | 2017-11-14 | 2018-05-11 | 歌尔科技有限公司 | Obtain the method and device of device information |
| CN108168555A (en) * | 2017-12-08 | 2018-06-15 | 李志新 | Operation guidance method and system based on coordinate setting |
| CN108805294A (en) * | 2017-04-26 | 2018-11-13 | 联想新视界（天津）科技有限公司 | A kind of processing method and processing device of tutorial message for target device |
| CN109451295A (en) * | 2017-08-29 | 2019-03-08 | 深圳市掌网科技股份有限公司 | A kind of method and system obtaining virtual information |
| CN109993023A (en) * | 2017-12-29 | 2019-07-09 | 佛山市顺德区美的电热电器制造有限公司 | A kind of household electrical appliances networking indicating means, AR equipment and computer storage medium |
| CN110175681A (en) * | 2018-09-13 | 2019-08-27 | 中国国际海运集装箱(集团)股份有限公司 | Maintenance method and device for airport ground facilities |
| CN110196580A (en) * | 2019-05-29 | 2019-09-03 | 中国第一汽车股份有限公司 | Assemble guidance method, system, server and storage medium |
| CN110569265A (en) * | 2019-09-10 | 2019-12-13 | 浪潮商用机器有限公司 | server accessory information display method and device, AR glasses and media |
| WO2020057130A1 (en) * | 2018-09-17 | 2020-03-26 | 深圳市泰衡诺科技有限公司 | Application control method and means |
| CN111049687A (en) * | 2019-12-23 | 2020-04-21 | 华自科技股份有限公司 | Equipment maintenance video operation guide file processing method and device and AR terminal |
| CN111273762A (en) * | 2019-08-27 | 2020-06-12 | 上海飞机制造有限公司 | Connector pin sending method and device based on AR equipment, AR equipment and storage medium |
| US10706633B2 (en) | 2018-12-06 | 2020-07-07 | Toyota Motor Engineering & Manufacturing North America, Inc. | Augmented reality systems and methods for vehicle inspection |
| CN111915032A (en) * | 2020-08-07 | 2020-11-10 | 中国十七冶集团有限公司 | AR technology-based equipment use and maintenance method |
| TWI710357B (en) * | 2020-01-17 | 2020-11-21 | 東昕精密科技股份有限公司 | AR dental prosthesis manufacturing guidance system and its application method |
| WO2020237667A1 (en) * | 2019-05-31 | 2020-12-03 | 西门子股份公司 | Method, device, and system for transmitting task execution record, and medium |
| CN112101056A (en) * | 2020-09-29 | 2020-12-18 | 重庆市华驰交通科技有限公司 | Electromechanical equipment identification and information acquisition method and system based on wearable equipment |
| CN112288883A (en) * | 2020-10-30 | 2021-01-29 | 北京市商汤科技开发有限公司 | Method and device for prompting operation guide information, electronic equipment and storage medium |
| CN112684951A (en) * | 2020-12-17 | 2021-04-20 | 宽衍(北京)科技发展有限公司 | Fault information query method and device, AR glasses, server and storage medium |
| WO2022198547A1 (en) * | 2021-03-24 | 2022-09-29 | 成都维泰油气能源技术有限公司 | Assistance system and method for intelligent well control |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101630064A (en) * | 2009-08-11 | 2010-01-20 | 广东工业大学 | Head video perspective three-dimensional display and control method thereof |
| CN104834379A (en) * | 2015-05-05 | 2015-08-12 | 江苏卡罗卡国际动漫城有限公司 | Repair guide system based on AR (augmented reality) technology |
-
2016
- 2016-11-09 CN CN201610987399.6A patent/CN106569769A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101630064A (en) * | 2009-08-11 | 2010-01-20 | 广东工业大学 | Head video perspective three-dimensional display and control method thereof |
| CN104834379A (en) * | 2015-05-05 | 2015-08-12 | 江苏卡罗卡国际动漫城有限公司 | Repair guide system based on AR (augmented reality) technology |
Non-Patent Citations (1)
| Title |
|---|
| 饶勇等: ""增强现实设备维修系统操作状态检测方法研究"", 《机电工程技术》 * |
Cited By (41)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108805294A (en) * | 2017-04-26 | 2018-11-13 | 联想新视界（天津）科技有限公司 | A kind of processing method and processing device of tutorial message for target device |
| CN107065808A (en) * | 2017-05-27 | 2017-08-18 | 深圳增强现实技术有限公司 | Accessories of a product method for posting, quality detecting method and production line system |
| CN107169139A (en) * | 2017-06-13 | 2017-09-15 | 深圳增强现实技术有限公司 | A kind of document file management system based on AR augmented realities, AR mobile intelligent terminals |
| CN107422710A (en) * | 2017-07-28 | 2017-12-01 | 歌尔股份有限公司 | The method and system of assembling product or plant maintenance are aided in based on AR |
| CN107391707A (en) * | 2017-07-28 | 2017-11-24 | 深圳增强现实技术有限公司 | Machine Operating Guideline system and ustomer premises access equipment |
| CN107316165A (en) * | 2017-07-28 | 2017-11-03 | 深圳增强现实技术有限公司 | Machine working specification creates system and operation Modeling Server |
| CN107277172A (en) * | 2017-07-28 | 2017-10-20 | 深圳增强现实技术有限公司 | Machine operates monitor system and its ustomer premises access equipment |
| CN107370821A (en) * | 2017-08-03 | 2017-11-21 | 深圳增强现实技术有限公司 | Machine Operating Guideline system, method and ustomer premises access equipment |
| CN107370821B (en) * | 2017-08-03 | 2020-10-09 | 深圳增强现实技术有限公司 | Machine operation guidance system |
| CN107463153A (en) * | 2017-08-11 | 2017-12-12 | 深圳增强现实技术有限公司 | A kind of management method of equipment working state, management equipment and management system |
| CN107463153B (en) * | 2017-08-11 | 2019-08-20 | 深圳增强现实技术有限公司 | A kind of management method of equipment working state, management equipment and management system |
| CN107526590A (en) * | 2017-08-14 | 2017-12-29 | 深圳乐普智能医疗器械有限公司 | Operation guidance information methods of exhibiting, device and the terminal of Medical Devices |
| CN107526590B (en) * | 2017-08-14 | 2021-03-30 | 深圳乐普智能医疗器械有限公司 | Operation guidance information display method and device of medical equipment and terminal |
| CN107730589A (en) * | 2017-08-18 | 2018-02-23 | 珠海格力电器股份有限公司 | Electrical appliance assembly method and device, storage device, mobile terminal and electrical appliance |
| CN109451295A (en) * | 2017-08-29 | 2019-03-08 | 深圳市掌网科技股份有限公司 | A kind of method and system obtaining virtual information |
| CN107688331A (en) * | 2017-08-30 | 2018-02-13 | 深圳增强现实技术有限公司 | Machine operates monitor system and its communication means |
| CN107547554A (en) * | 2017-09-08 | 2018-01-05 | 北京枭龙科技有限公司 | A kind of smart machine remote assisting system based on augmented reality |
| CN107730008A (en) * | 2017-09-14 | 2018-02-23 | 珠海格力电器股份有限公司 | Device after-sale guidance method and device, computer readable storage medium and terminal |
| CN107784885A (en) * | 2017-10-26 | 2018-03-09 | 歌尔科技有限公司 | Operation training method and AR equipment based on AR equipment |
| CN108021991A (en) * | 2017-11-14 | 2018-05-11 | 歌尔科技有限公司 | Obtain the method and device of device information |
| CN108168555A (en) * | 2017-12-08 | 2018-06-15 | 李志新 | Operation guidance method and system based on coordinate setting |
| CN107977685A (en) * | 2017-12-21 | 2018-05-01 | 江西爱驰亿维实业有限公司 | Obtain the method, apparatus and electronic equipment of electronics service manual |
| CN109993023A (en) * | 2017-12-29 | 2019-07-09 | 佛山市顺德区美的电热电器制造有限公司 | A kind of household electrical appliances networking indicating means, AR equipment and computer storage medium |
| CN110175681A (en) * | 2018-09-13 | 2019-08-27 | 中国国际海运集装箱(集团)股份有限公司 | Maintenance method and device for airport ground facilities |
| WO2020057130A1 (en) * | 2018-09-17 | 2020-03-26 | 深圳市泰衡诺科技有限公司 | Application control method and means |
| US10706633B2 (en) | 2018-12-06 | 2020-07-07 | Toyota Motor Engineering & Manufacturing North America, Inc. | Augmented reality systems and methods for vehicle inspection |
| CN110196580A (en) * | 2019-05-29 | 2019-09-03 | 中国第一汽车股份有限公司 | Assemble guidance method, system, server and storage medium |
| CN110196580B (en) * | 2019-05-29 | 2020-12-15 | 中国第一汽车股份有限公司 | Assembly guidance method, system, server and storage medium |
| WO2020237667A1 (en) * | 2019-05-31 | 2020-12-03 | 西门子股份公司 | Method, device, and system for transmitting task execution record, and medium |
| CN111273762A (en) * | 2019-08-27 | 2020-06-12 | 上海飞机制造有限公司 | Connector pin sending method and device based on AR equipment, AR equipment and storage medium |
| CN110569265A (en) * | 2019-09-10 | 2019-12-13 | 浪潮商用机器有限公司 | server accessory information display method and device, AR glasses and media |
| CN111049687A (en) * | 2019-12-23 | 2020-04-21 | 华自科技股份有限公司 | Equipment maintenance video operation guide file processing method and device and AR terminal |
| CN111049687B (en) * | 2019-12-23 | 2023-03-24 | 华自科技股份有限公司 | Equipment maintenance video operation guide file processing method and device and AR terminal |
| CN113133844A (en) * | 2020-01-17 | 2021-07-20 | 东昕精密科技股份有限公司 | AR dental prosthesis manufacturing guide system and application method thereof |
| TWI710357B (en) * | 2020-01-17 | 2020-11-21 | 東昕精密科技股份有限公司 | AR dental prosthesis manufacturing guidance system and its application method |
| CN111915032A (en) * | 2020-08-07 | 2020-11-10 | 中国十七冶集团有限公司 | AR technology-based equipment use and maintenance method |
| CN112101056A (en) * | 2020-09-29 | 2020-12-18 | 重庆市华驰交通科技有限公司 | Electromechanical equipment identification and information acquisition method and system based on wearable equipment |
| CN112288883A (en) * | 2020-10-30 | 2021-01-29 | 北京市商汤科技开发有限公司 | Method and device for prompting operation guide information, electronic equipment and storage medium |
| CN112684951A (en) * | 2020-12-17 | 2021-04-20 | 宽衍(北京)科技发展有限公司 | Fault information query method and device, AR glasses, server and storage medium |
| WO2022198547A1 (en) * | 2021-03-24 | 2022-09-29 | 成都维泰油气能源技术有限公司 | Assistance system and method for intelligent well control |
| US11756275B2 (en) | 2021-03-24 | 2023-09-12 | Vertechs Oil & Gas Technology Co., Ltd. | Auxiliary system and method for intelligent well control |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106569769A (en) | AR technology-based machine operation instruction information display method and apparatus | |
| CN106355153B (en) | A kind of virtual objects display methods, device and system based on augmented reality | |
| CN106210861B (en) | Method and system for displaying bullet screen | |
| US20170195664A1 (en) | Three-dimensional viewing angle selecting method and apparatus | |
| CN106598229B (en) | Virtual reality scene generation method and device and virtual reality system | |
| CN110716645A (en) | Augmented reality data presentation method and device, electronic equipment and storage medium | |
| CN111610998A (en) | AR scene content generation method, display method, device and storage medium | |
| KR20140082610A (en) | Method and apaaratus for augmented exhibition contents in portable terminal | |
| US10403048B2 (en) | Storage medium, content providing apparatus, and control method for providing stereoscopic content based on viewing progression | |
| WO2015090421A1 (en) | Method and system for providing information associated with a view of a real environment superimposed with a virtual object | |
| JP2016522463A5 (en) | ||
| CN106569671A (en) | AR technology-based method and device for displaying instruction information of consumer goods | |
| WO2016203111A1 (en) | Mediated reality | |
| WO2020012955A1 (en) | Information processing device, information processing method, and program | |
| KR20190078675A (en) | Immersed exhibition system based on AR(augmented reality)-VR(virtual reality) technology and method thereof | |
| CN111652986B (en) | Stage effect presentation method and device, electronic equipment and storage medium | |
| EP2717227A2 (en) | Image processing program, image processing device, image processing system, and image processing method | |
| EP3038061A1 (en) | Apparatus and method to display augmented reality data | |
| CN113382224B (en) | Interactive handle display method and device based on holographic sand table | |
| CN110488489B (en) | Eye registration for a head-mounted housing | |
| KR101879102B1 (en) | Method for implementing augmented reality system | |
| CN115543138A (en) | Display control method and device, augmented reality head-mounted device and medium | |
| CN105334692B (en) | Information processing method and electronic equipment | |
| CN115268650A (en) | Picture screen capturing method and device, head-mounted virtual reality equipment and storage medium | |
| JP2017097854A (en) | Program, recording medium, content providing device, and control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170419 |
|
| RJ01 | Rejection of invention patent application after publication |