CN107656505A - Use the methods, devices and systems of augmented reality equipment control man-machine collaboration - Google Patents
Use the methods, devices and systems of augmented reality equipment control man-machine collaboration Download PDFInfo
- Publication number
- CN107656505A CN107656505A CN201710720950.5A CN201710720950A CN107656505A CN 107656505 A CN107656505 A CN 107656505A CN 201710720950 A CN201710720950 A CN 201710720950A CN 107656505 A CN107656505 A CN 107656505A
- Authority
- CN
- China
- Prior art keywords
- augmented reality
- reality equipment
- industrial robot
- operation information
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003190 augmentative effect Effects 0.000 title claims abstract description 166
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000004891 communication Methods 0.000 claims abstract description 26
- 230000004044 response Effects 0.000 claims abstract description 14
- 238000004519 manufacturing process Methods 0.000 claims description 27
- 230000008859 change Effects 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 17
- 230000033001 locomotion Effects 0.000 claims description 12
- 230000000007 visual effect Effects 0.000 claims description 12
- 210000005252 bulbus oculi Anatomy 0.000 claims description 10
- 238000005516 engineering process Methods 0.000 claims description 10
- 230000002708 enhancing effect Effects 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 5
- 238000006073 displacement reaction Methods 0.000 claims description 5
- 230000004886 head movement Effects 0.000 claims description 4
- 238000013439 planning Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 7
- 210000003128 head Anatomy 0.000 description 51
- 230000008569 process Effects 0.000 description 12
- 239000011521 glass Substances 0.000 description 11
- 238000009776 industrial production Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 10
- 230000003287 optical effect Effects 0.000 description 10
- 238000004590 computer program Methods 0.000 description 9
- 238000003860 storage Methods 0.000 description 7
- 238000001514 detection method Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000002452 interceptive effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 235000013399 edible fruits Nutrition 0.000 description 2
- 210000001508 eye Anatomy 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001953 sensory effect Effects 0.000 description 2
- 238000005728 strengthening Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 241000196324 Embryophyta Species 0.000 description 1
- 235000013290 Sagittaria latifolia Nutrition 0.000 description 1
- 241000209140 Triticum Species 0.000 description 1
- 235000021307 Triticum Nutrition 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 235000015246 common arrowhead Nutrition 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000004148 unit process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
- G05B19/41865—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM] characterised by job scheduling, process planning, material flow
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/32—Operator till task planning
- G05B2219/32252—Scheduling production, machining, job shop
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Manufacturing & Machinery (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a kind of methods, devices and systems that man-machine collaboration is controlled using augmented reality equipment.This method includes：Augmented reality equipment establishes communication connection with industrial robot；Augmented reality equipment obtains operation information, and corresponding operate is performed in response to operation information；Augmented reality equipment generation control instruction is sent to industrial robot, wherein, industrial robot performs respective operations according to control instruction.By the present invention, the effect for being produced by augmented reality equipment support industry and improving efficiency and security is reached.
Description
Technical field
The present invention relates to augmented reality field, and augmented reality equipment control man-machine collaboration is used in particular to one kind
Methods, devices and systems.
Background technology
As the mankind step into industrial 4.0 epoch, industrial production is to efficiency and the demand more and more higher of degree of intelligence.And at this
During individual, the promptness and high efficiency of man-machine interaction and data presentation are most important.
Although in the industrial premises of most modern, machine has been connected into center management system, and factory can collect
The data of machine are simultaneously controlled by machine, but industrial quarters remains unchanged to lack and allows the acquisition information of worker's fast accurate, reading to believe
Breath, the scheme with information exchange.
Currently used technical scheme includes：
1st, it is each production equipment configuration display and control terminal (such as display, control panel, handle, keyboard).
2nd, worker carries the handheld mobile device (such as notebook computer, tablet personal computer, mobile phone or other equipment) of networking.
Scheme one：Due to be that each equipment arranges display device and control device, hardware cost, installation cost, dimension
It is very high to protect cost, while very space-consuming and the energy.
Scheme two：Worker wants long-time hand-held display device, and both hands are occupied, can not carry out industrial production simultaneously.
The defects of two schemes have：
1st, the space of display information is limited by screen size, if it is desired to which bigger space just needs bigger screen
Curtain, cost improve, and portability declines.
2nd, the mode of display information is undesirable, and information can only be presented on screen, and worker needs frequently to bow or turned
Head checks display screen, reduces operating efficiency.
3rd, current display screen is only able to display plane information, it is impossible to shows 3D information.
4th, interactive mode is limited, and the interaction of both the above scheme is required for operation physical button or touching display screen
Curtain, in the case where be inconvenient to use both hands, operate highly difficult (for example having the situation of greasy dirt in wearing gloves or on hand).
5th, above scheme can not realize man-machine collaboration, that is, according to situations such as service speed, the mode of operation of people it is real
When adjust robot operating rate.
For, by unhandy problem during display screen auxiliary operation, not yet being carried at present in industrial production in correlation technique
Go out effective solution.
The content of the invention
It is a primary object of the present invention to provide it is a kind of using augmented reality equipment control man-machine collaboration method, apparatus and
System, to solve in correlation technique in industrial production by unhandy problem during display screen auxiliary operation.
To achieve these goals, according to an aspect of the invention, there is provided a kind of controlled using augmented reality equipment
The method of man-machine collaboration, this method include：Augmented reality equipment establishes communication connection with industrial robot；The augmented reality is set
It is standby to obtain operation information, perform corresponding operate in response to the operation information；The augmented reality equipment generates control instruction
The industrial robot is sent to, wherein, the industrial robot performs respective operations according to the control instruction.
Further, the augmented reality equipment obtains operation information including following any one or more：The enhancing
Real world devices obtain the gesture instruction that user sends；The augmented reality equipment obtains the phonetic order that user sends；The increasing
Strong real world devices are by identifying that user eyeball active state obtains instruction；The augmented reality equipment obtains the industrial robot
The operation information of transmission；The augmented reality equipment parses the image acquisition operations letter gathered in real time by image recognition technology
Breath；The augmented reality equipment obtains the operation information that production line control centre sends.
Further, after the augmented reality equipment obtains operation information, methods described also includes：The enhancing is existing
Real equipment updates display content according to the operation information；The augmented reality equipment uses processing according to the operation information
Device module, generation motion planning and robot control instruction；The augmented reality equipment uses processor die according to the operation information
Block, the operational data of the operator of production line is generated, sent to production line control centre；The augmented reality equipment is according to
Operation information determines the operating rate of the industrial robot, and generation robot speed control instructs, wherein, the industrial machine
The operating rate of people is used for the operation rate for coordinating the operator of current manufacturing lines.
Further, the augmented reality equipment updates display content according to the operation information and included：The enhancing is existing
Real equipment determines next step operation object according to the operation information；The augmented reality equipment is generated and shown and operates in next step
Prompt message；The augmented reality renewal of the equipment simultaneously shows the movement velocity of industrial robot, current location and joint angles；
The augmented reality equipment shows the motor temperature of industrial robot.
Further, the augmented reality equipment generates and shows that the prompt message of operation in next step includes：To be operated
The prompting icon or word or 3D models of operation in next step are shown on object；And/or show in the augmented reality equipment visual field
Show the prompting icon or word or 3D models of operation in next step.
Further, the augmented reality equipment generation control instruction, which is sent to the industrial robot, includes：The increasing
Strong real world devices generation starts, suspended, terminating, speed change, restart, reset, governing speed, the control instruction of change pattern are sent to
The industrial robot.
Further, the control instruction of the augmented reality equipment generation includes：The industrial robot is controlled to change certainly
Body motion state, wherein, the displacement state includes the movement velocity of the industrial robot, current location and joint angle
Degree；The industrial robot is controlled to change operation object；The industrial robot is controlled to change operator scheme；Control the industry
Robot changes operation rate；Control the industrial robot that operating result is sent back into the augmented reality equipment.
To achieve these goals, according to another aspect of the present invention, additionally provide one kind and use augmented reality equipment control
The device of man-machine collaboration processed, the device include：Communication unit, communicated for being established by augmented reality equipment and industrial robot
Connection；Acquiring unit, for obtaining operation information by the augmented reality equipment, correspondence is performed in response to the operation information
Operation；Processing unit, the industrial robot is sent to for generating control instruction by the augmented reality equipment, its
In, the industrial robot performs respective operations according to the control instruction.
Further, the acquiring unit is following any one or more for performing：Pass through the augmented reality equipment
Obtain the gesture instruction that user sends；The phonetic order sent by augmented reality equipment acquisition user；Pass through the increasing
Strong real world devices are by identifying that user eyeball active state obtains instruction；Image recognition skill is passed through by the augmented reality equipment
Art, parse the image acquisition operations information gathered in real time；Production line control centre is obtained by the augmented reality equipment to send
Operation information；The operation information of the industrial robot transmission is obtained by the augmented reality equipment.
To achieve these goals, according to another aspect of the present invention, additionally provide one kind and use augmented reality equipment control
The control system of man-machine collaboration processed, it is characterised in that including：Industrial equipment；Augmented reality equipment, the augmented reality equipment are used
In the method that man-machine collaboration is controlled using augmented reality equipment of the present invention；Industrial robot, for being set with the augmented reality
It is standby to coordinate the control industrial equipment.
The present invention establishes communication connection by augmented reality equipment and industrial robot；Augmented reality equipment obtains operation letter
Breath, corresponding operate is performed in response to operation information；Augmented reality equipment generation control instruction is sent to industrial robot, its
In, industrial robot performs respective operations according to control instruction, solves auxiliary by display screen in industrial production in correlation technique
Unhandy problem when helping operation, and then pass through the production of augmented reality equipment support industry and improve efficiency and security
Effect.
Brief description of the drawings
The accompanying drawing for forming the part of the application is used for providing a further understanding of the present invention, schematic reality of the invention
Apply example and its illustrate to be used to explain the present invention, do not form inappropriate limitation of the present invention.In the accompanying drawings：
Fig. 1 is the flow chart of the method according to embodiments of the present invention that man-machine collaboration is controlled using augmented reality equipment；
Fig. 2 is the schematic diagram of the system according to embodiments of the present invention that man-machine collaboration is controlled using augmented reality equipment；With
And
Fig. 3 is the schematic diagram of the device according to embodiments of the present invention that man-machine collaboration is controlled using augmented reality equipment.
Embodiment
It should be noted that in the case where not conflicting, the feature in embodiment and embodiment in the application can phase
Mutually combination.Describe the present invention in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
In order that those skilled in the art more fully understand application scheme, below in conjunction with the embodiment of the present application
Accompanying drawing, the technical scheme in the embodiment of the present application is clearly and completely described, it is clear that described embodiment is only
The embodiment of the application part, rather than whole embodiments.Based on the embodiment in the application, ordinary skill people
The every other embodiment that member is obtained under the premise of creative work is not made, it should all belong to the model of the application protection
Enclose.
It should be noted that term " first " in the description and claims of this application and above-mentioned accompanying drawing, "
Two " etc. be for distinguishing similar object, without for describing specific order or precedence.It should be appreciated that so use
Data can exchange in the appropriate case, so as to embodiments herein described herein.In addition, term " comprising " and " tool
Have " and their any deformation, it is intended that cover it is non-exclusive include, for example, containing series of steps or unit
Process, method, system, product or equipment are not necessarily limited to those steps clearly listed or unit, but may include without clear
It is listing to Chu or for the intrinsic other steps of these processes, method, product or equipment or unit.
For the ease of description, below to the invention relates to several terms illustrate：
Augmented reality, it is a kind of by " seamless " the integrated new technology of real world information and virtual world information, is
Script it is difficult to which the entity information (visual information, sound etc.) experienced, leads in the certain time spatial dimension of real world
The technologies such as computer vision are crossed, are superimposed again after analog simulation, virtual Information application to real world is felt by human sensory
Know, so as to reach the sensory experience of exceeding reality.Real environment and virtual object have been added to same picture in real time
Or space exists simultaneously.
AR systems have three it is prominent the characteristics of：1. real world and virtual information integration；2. there is real-time, interactive；
3. increase positioning dummy object in three dimension scale space.
The embodiments of the invention provide a kind of method that man-machine collaboration is controlled using augmented reality equipment.
Fig. 1 is the flow chart of the method according to embodiments of the present invention that man-machine collaboration is controlled using augmented reality equipment, such as
Shown in Fig. 1, this method comprises the following steps：
Step S102：Augmented reality equipment establishes communication connection with industrial robot.
Step S104：Augmented reality equipment obtains operation information, and corresponding operate is performed in response to operation information.
Step S106：Augmented reality equipment generation control instruction be sent to industrial robot, wherein, industrial robot according to
Control instruction performs respective operations.
The embodiment establishes communication connection by augmented reality equipment and industrial robot；Augmented reality equipment obtains operation
Information, corresponding operate is performed in response to operation information；Augmented reality equipment generation control instruction is sent to industrial robot, its
In, industrial robot performs respective operations according to control instruction, solves auxiliary by display screen in industrial production in correlation technique
Unhandy problem when helping operation, and then pass through the production of augmented reality equipment support industry and improve efficiency and security
Effect.
In embodiments of the present invention, augmented reality equipment can establish communication with industrial robot in several ways and connect
Connect, including but not limited to mode, the augmented reality equipment such as bluetooth, wireless network, cable network connection can be that augmented reality shows
Show glasses (AR glasses), AR glasses can obtain operational order in several ways, operational order can be user by gesture,
Language, watch the operational order for AR glasses that modes such as (gaze) is sent attentively, operation information (instruction) can also be according to
The instruction that operation of the family for industrial equipment generates.AR glasses operate corresponding to being performed in response to operational order, for example, can
With prompt message corresponding to display etc., AR glasses can also generate control instruction and be sent to industrial robot, to control industrial machine
Device people, which coordinates, completes more multioperation, and industrial robot controls industrial equipment after control instruction is received, according to control instruction.
Alternatively, perform corresponding operation in response to operational order in augmented reality equipment and generate control instruction and be sent to
After industrial robot, augmented reality equipment receives the control result that industrial robot returns, wherein, control result is industrial machine
Device people performs the control result after operation according to control instruction；Augmented reality equipment updates display content according to control result.
Industrial robot is sent to augmented reality equipment after industrial equipment is controlled according to control instruction, by control result, and enhancing is existing
Real equipment updates the display content in glasses after the control result of industrial robot return is received, according to control result.
Alternatively, augmented reality equipment updates display content according to control result to be：Augmented reality equipment is according to control
Result processed determines next step operation object；Augmented reality equipment generates and shows the prompt message of operation in next step；Augmented reality
Renewal of the equipment and the movement velocity for showing industrial robot, current location and joint angles；Augmented reality equipment shows industrial machine
The motor temperature of device people.
Alternatively, augmented reality equipment generates and shown that the prompt message of operation in next step can be：In object to be operated
The prompting icon or word or 3D models that upper display operates in next step；And/or the surrounding space in the augmented reality equipment visual field
The prompting icon or word or 3D models that middle display operates in next step.Display location can be integrally fixed on object to be operated,
It can be the prompting that display next step operation is fixed in other positions.Display can also be other letters except prompt message
The related information of the production such as breath, such as total quantity, total time, current efficiency, robotary.
Strengthening display device can determine that operation and next step operation object etc. are more in next step according to the process of current operation
The details of individual next step operation, it can be prompted according to next step operation, under the display content in enhancing display device includes
Single stepping is prompted, and next step operation indicating can include a variety of operation guidance informations, in order to more facilitate operator to operate,
Next step operation prompt information can be shown in AR glasses, the mode of display can be fixed on object to be operated
Display reminding information guidance operation, it can so make guidance information more directly perceived, it is easier to understand and operate, reduce worker
Operating mistake probability and learning and memory cost.Prompt message can also follow user's movement fixed in glasses in AR glasses
The prompting of operation in next step is shown in the upper right corner in the visual field or the upper left corner, can be the form of list, word or video.
Alternatively, augmented reality equipment updates display content according to operation information；Augmented reality equipment according to operation information,
Use processor module, generation motion planning and robot control instruction；Augmented reality equipment uses processor die according to operation information
Block, the operational data of the operator of production line is generated, sent to production line control centre；Augmented reality equipment is according to operation information
The operating rate of industrial robot is determined, generation robot speed control instructs, wherein, the operating rate of industrial robot is used for
Coordinate the operation rate of the operator of current manufacturing lines.
Alternatively, augmented reality equipment, which obtains operational order, includes following any one：Augmented reality equipment obtains user
The gesture instruction sent；Augmented reality equipment obtains the phonetic order that user sends；Augmented reality equipment is by identifying user's eye
Ball or head movement state obtain instruction；State acquisition operation of the augmented reality equipment by gathering the object to be operated in real time refers to
Order, augmented reality equipment, which can also be read, obtains the operational order that production line control centre sends, or obtains industrial robot
The operational order of transmission.Augmented reality equipment can parse the image acquisition operations letter gathered in real time by image recognition technology
Breath；Augmented reality equipment obtains the operation information that production line control centre sends；Augmented reality equipment obtains industrial machine human hair
The operation information sent, strengthening the instruction of display device can also be obtained by gathering the state of the object to be operated, for example, AR glasses
After controlling industrial robot operating industrial installations, current status information can be sent to AR eyes by industrial equipment by instruction
Mirror.The source of the operational order of augmented reality equipment is varied, is not limited thereto, and the command source of any of which is all
Among protection scope of the present invention.
Alternatively, augmented reality equipment performs corresponding operation and generate control instruction in response to operational order is sent to work
Industry robot can be：Augmented reality equipment in response to operational order perform corresponding to operate and generate and start, suspend, terminate
Control instruction is sent to industrial robot.
Augmented reality equipment to the instruction that industrial robot is sent can be polytype, for example, it may be start, temporarily
Stop, terminate, speed change, restart, reset, the instruction such as governing speed, change pattern control industrial robot performs and starts, suspends, tying
Beam operation etc..
Alternatively, the control instruction of augmented reality equipment generation includes：Industrial robot is controlled to change displacement state,
Wherein, displacement state includes the movement velocity of industrial robot, current location and joint angles；Control industrial robot changes
Become operation object；Industrial robot is controlled to change operator scheme；Industrial robot is controlled to change operation rate；Control industrial machine
Operating result is sent back to augmented reality equipment by people.Joint angles include the angle in each joint of robot, the articulate angle of institute
Degree can determine the posture of current robot.
The technical scheme of the embodiment of the present invention can also realize intelligent man-machine collaboration, by augmented reality equipment, industrial machine
In the system of device people and industrial equipment composition, augmented reality equipment can be anti-by the action or industrial equipment that obtain worker
The signal of feedback obtains the operation rate of current manufacturing lines operator, then determines the work of industrial robot according to operation rate
Speed, then by the operating rate determined be sent to industrial robot in real time adjust coordinate worker work speed so that work
Cooperation of the industry robot for man-machine collaboration process is more intelligent, perceives sensitiveer.In a kind of application scenarios, each production
During the working condition difference of each worker on line, operation rate is different, for example, experience is faster than more rich operation speed
Some, the fewer operation speed of experience is slow, in order to reduce the operating mistake of the slow worker of operation rate, can drop
The cooperation speed of low industrial robot, in order to improve veteran work task efficiency, industrial robot can be improved
Coordinate speed.Or in the case of worker's fatigue, reduce coordinate speed to meet the needs under different scenes in time.Connecing
After receiving the operating rate that augmented reality equipment is sent, industrial robot adjusts cooperating in time according to this parameter
Operating rate, to realize real man-machine collaboration, improve the intelligent of industrial robot.
The embodiment of the present invention additionally provides a kind of preferred embodiment, with reference to the preferred embodiment to the present invention's
Technical scheme illustrates.
Augmented reality equipment (AR heads display device) includes following critical piece：
Optics display module, optical camera group, RGB-D depth cameras, IMU inertia detection units, voice collecting wheat
Gram wind, operating and processing module (chip module), communication module group.
1st, information is presented
The optics display module of AR heads display device is located at worker at the moment, it would be desirable to the virtual information of display, directly throws
Penetrate in the worker visual field.Worker, which does not need rotary head or turned one's head, checks that screen just can directly obtain information.
Because optics display module is on head device, worker does not need long-time hand-held display device (ratio
Such as tablet personal computer), liberate the both hands of worker.
AR heads display device imaging is not shown that the information content is projected directly at regarding for worker by physical display screen size
In wild space, can there are bigger indication range and more preferable display effect.
By the parallax of right and left eyes image, AR heads display device can make the interior of reality have Deep Canvas and three-dimensional effect
Fruit, so as to realize displaying 3D contents.
Optical camera group, the IMU inertia detection units of AR heads display device will gather the vision of worker current location
Image and the movable information of worker, the change in location of worker is calculated by operating and processing module, so that it is determined that virtual content
Correct display location, realize and virtual content is fixed on some position of realistic space.
2nd, interactive controlling
Optical camera group, IMU inertia detection units are able to detect that the head movement of worker, pass through operating and processing module
Determine the focus of worker's sight.
Optical camera group, RGB-D depth cameras can catch posture and the motion of worker's hand, pass through calculation process
Module handles to obtain the instruction that the gesture of worker is sent, including clicks on, pulls, touches, overturns, rolls dummy object.
Voice collecting microphone can gather the voice of worker, understand the finger that worker sent by operating and processing module
Order.
By three above approach, AR heads display device will not need user to manipulate actual control hardware and (such as press
Button, screen, handle etc.), solve that both hands are occupied can not to click on button, or both hands it is unholiness/wearing gloves can not click on
The problem of touch-screen.
3rd, communicate
AR heads display device can be supported to communicate with the high quality of other equipment as smart machine.Main point of communication
For following three main parts：
(1) control centre/server with factory communicates, and obtains information, receives and dispatches instruction and data.Such as warehoused cargo
Positional information, plant area's navigation information, accident device location information, work process Global Information etc..
(2) with there is the device talk (such as industrial robot, electronics lathe etc.) of communication function, by worker by voice,
Operational order made by eyeball, gesture issues actual production equipment, while can also be sent to the job information of equipment
Worker.
(3) communicated with other AR heads display device, because AR heads display device will gather the visual field of worker, position
Confidence ceases, and the communication between AR equipment can apply in remote command, training and teaching, collaborative work etc. well.
4th, image object identifies
AR heads display device has the function of image recognition and object identification, optical camera group therein and RGB-D
Posture of the depth camera except being able to detect that hand, can also detect and identify picture and object.By this One function,
AR heads display device can quickly determine current application scenarios, such as determine which goods checked, determine that worker exists
Which equipment is operated, it is determined that being currently which process etc..Based on this function, AR heads display device can be in specific occasion
Customizing messages is shown, realizes the intellectuality of display, such as worker with an AR heads display device, but go to different
Different machine datas can be shown before machine.
Just whether AR heads display device can check the operating result of worker using image and the function of object identification
Really.For example, for the work such as wiring or installation parts, AR heads display device can be gathered by optical camera group works as
Preceding working result picture, these image informations are compared operating and processing module with correct result images, to determine work
Whether people, which misses what, is fitted without, or whether correct the order of data wire (power line) is.
The method that man-machine collaboration is controlled using augmented reality equipment of the embodiment of the present invention, following technology effect can be realized
Fruit：
1st, AR heads display device display information does not need external physical screen, and the information that can be shown is more, can
Display space is bigger, and the real space around worker can placement information.Cost substantially reduces simultaneously.
2nd, AR heads display device is worn on the head of worker, and worker need not hold other equipment, liberate both hands, work
People can synchronously carry out industrial operation while information, manipulation information is obtained.
3rd, AR heads display device supports that voice, eyeball and gesture control, worker need not be each manipulator's dynamic point
Button or display screen are hit, commercial Application scene greatly increases.
4th, the information that worker needs directly is presented in the visual field of worker by AR heads display device, it is not necessary to is frequently looked into
Screen is seen, improves operating efficiency.
5th, AR heads display device can show 2D or 3D content, and this causes the information of display to be easier to be managed by worker
Solution, has prominent under multiple scenes such as worker training, machine construction explanation, execute-in-place guidance, machine assembling, machine maintenance
Advantage.
6th, AR heads display device can collect operation information, movable information and the field of view information of worker, be industrial production
Intelligent provide data source.
The control method of the augmented reality equipment of the embodiment of the present invention can adapt to the growth requirement of industry internet, AR heads
Portion's display device is optimal mobile terminal, realizes intelligent industrial production on this basis, is greatly improved industrial
Efficiency (particularly work human efficiency), is specifically included：
1st, data message is presented
The presentation of AR information does not need physical screen, and the space of display greatly expands, and the content of display is more rich, while into
Originally substantially reduce.The content that AR is shown is occurred directly in the user visual field, and the mode that worker obtains information is more direct, it is not necessary to
Traditional display screen is frequently checked, improves the efficiency of operation.AR heads display device causes the presentation of information no longer by multiple
Miscellaneous industrial production environment limitation.
AR heads display device can show 3D content, and this causes the information of display to be easier to be understood by worker, in work
There is prominent advantage under multiple scenes such as people's training, machine construction explanation, execute-in-place guidance, machine assembling, machine maintenance.
2nd, man-machine collaboration and Human-machine Control
AR heads display device has redefined the interactive mode of worker and information, and most operations need not grasp
Physical components (button, handle, screen etc.) are controlled, but are directly interacted by the voice, eyeball, gesture of worker with AR contents, greatly
The big scene for extending industry interaction.Simultaneously because using AR heads display device, worker does not need handheld device (such as flat board electricity
Brain etc.), liberated the both hands of worker, while information can have been obtained in the case where both hands carry out industrial operation, operate machine or
Person's other equipment.
3rd, combined network communication ＆ data acquisitions
AR heads display device can with the control centre of factory, on-site production equipment (such as robot, service
Device, digital machine etc.) and other AR heads display device communicated, enter the timely exchange of row information, by worker and machine
Device is networked.
AR heads display device is the terminal of Data Collection simultaneously.By AR heads display device, factory is only possible to collect
To the field of view information of each employee, operation information, positional information etc..On the basis of these data, intelligent production is realized.
The technical scheme of the embodiment of the present invention realizes the simulated production of AR heads display device and robotic arm and work station.
By AR heads display device, worker can obtain the tutorial message of work in real time, robotic arm by with AR heads
Display device, can adjust that the rhythm of oneself is consistent with worker, and work station can be closed by the communication with AR heads display device
Manage progress of arranging work.Worker can be by issuing with the object in the visual field, button interaction, control machine arm or to work station
Instruction.By using AR heads display device, worker no longer needs to remember the operation that each step is cumbersome, while work station can obtain
The job information of each worker is obtained, it is more preferably more efficient to arrange work.
Specifically, in an application scenarios of the embodiment of the present invention：
Binocular AR heads display device establishes the communication with industrial robot.
Binocular AR head display devices initially enter operation interface by identifying gesture instruction, the phonetic order of worker.
Binocular AR head display devices will work related presentation of information in working region.
The mode of display can be it is fixed suspend in space, be fixed on practical operation object (workpiece, instrument) and (differ
Surely stick together, can spatially there is certain layout), follow the worker visual field move.
The information being fixed on practical operation object has actual process directive function, can make word, button, arrow
Head, icon, 3D models or other instructional informations.
The position of display can be determined by following several ways：Reality on the display device identification working position of binocular AR heads
The position of object (optical camera group ＆ operating and processing modules do image recognition)；Binocular AR heads display device identification worker's eye
Ball and gesture defined location (optical camera Zu ＆IMU Guan Xingjiancedanyuan ＆RGB-D depth camera ＆ calculation process moulds
Group).
The voice or gesture instruction of binocular AR heads display device identification worker, and industrial robot is issued into instruction,
Control machine people starts, suspended, power cut-off；Control machine people captures, put, processing component.
Binocular AR heads display device receives the charge that machine human hair is returned, and updates the display content of oneself, including：Renewal meter
Number, timing, current procedures, prompt message etc.；Update the configured information near practical operation object, such as button state, arrow
Direction, the position (being partially moved to another part from one, represent the change of next step operation object) of configured information.
The gesture of binocular AR heads display device detection workman, voice or practical operation object state (such as etc. it is bright
Rise), robotic arm is sent an instruction to, the operation after the progress of notice robot, realizes man-machine efficient collaboration, both (that is machine
Device people is known that worker has accomplished which is walked, to determine how robot will work).
The key problem in technology point of the embodiment of the present invention is included with lower part：
Realize that (actual use is soft to worker with intelligent industrial equipment in industrial production by binocular AR heads display device
Property cooperation robot, but the industrial robot that in general supports communication and instruction control can also be applied to) efficient collaboration, both is
System.
Its high efficiency is embodied in：Prompt message for worker, operator-work is directly displayed in the visual field, actual
The information of operation is to be directly displayed at operation thing physically, and acquisition of information is more direct, indicative stronger；It is double for machinery equipment
Mesh AR heads display device can obtain state (including the phonetic order of operator, gesture instruction, the operation of operating space in real time
State of object etc.), robot is sent to by communication, helps it preferably to arrange the work of oneself.
Acquisition of the binocular AR head display devices for display location, it can make from the actual operation thing of image recognition
Body or come as operator gesture control (click on, pull, rotation etc.).
Renewal of the binocular AR head display devices for display information, it may come from instruction or the shape of robot transmission
State；It may come from the voice or gesture of identification worker；It may come from the state of identification practical operation object.
The AR of the embodiment of the present invention constructs the industrial quarters human computer cooperation system of complete set.The configured information that AR is shown can
To be directly placed on the object or part that are actually needed operation, make guidance information more directly perceived, it is easier to understand, reduce wrong
Probability and learning and memory cost by mistake.Worker passes through the aobvious real-time control that can be realized to robot of AR heads, flexible control.Structure
The human computer cooperation system built be one can be with the system of real-time communication, AR can issue the state of people machine, while can will
The state of machine issues people, realizes efficient cooperation.
It should be noted that in the inventive solutions, in communication：Can not only be communicated with robot, can also and
Control centre communicates, and other AR device talks.Obtain in instruction, except the present finger that can also be robot and actively send
Order, instruction that control centre sends etc. can be made.In display, the operation prompt information except showing next step, it can also show
Many contents, such as the state of robot, the data (total quantity, total time, current efficiency etc.) of overall work, auxiliary information
(such as prompting robot from it is close have a risk of collision, remind worker to rest, temporary derangement maintenance etc.).It is being emitted to machine
In the instruction of device people, except starting the EOT end of timeout, the instruction of many samples is can also be, for example is restarted, resets, governing speed, changes
Change pattern etc.).
The embodiment of the present invention additionally provides a kind of system that man-machine collaboration is controlled using augmented reality equipment.
Fig. 2 is the schematic diagram of the system according to embodiments of the present invention that man-machine collaboration is controlled using augmented reality equipment.Such as
Shown in Fig. 2, the system includes：
Industrial equipment 10；
Augmented reality equipment 20, for performing the side that man-machine collaboration is controlled using augmented reality equipment of the embodiment of the present invention
Method；
Industrial robot 30, the industrial equipment 10 is controlled for coordinating with augmented reality equipment 20.
The control method of the augmented reality equipment of the embodiment is same as the previously described embodiments, will not be repeated here.
It should be noted that can be in such as one group of computer executable instructions the flow of accompanying drawing illustrates the step of
Performed in computer system, although also, show logical order in flow charts, in some cases, can be with not
The order being same as herein performs shown or described step.
The embodiments of the invention provide a kind of device that man-machine collaboration is controlled using augmented reality equipment, the device can be used
In the method that man-machine collaboration is controlled using augmented reality equipment for performing the embodiment of the present invention.
Fig. 3 is the schematic diagram of the device according to embodiments of the present invention that man-machine collaboration is controlled using augmented reality equipment, such as
Shown in Fig. 3, the device includes：
Communication unit 210, for establishing communication connection by augmented reality equipment and industrial robot；
Acquiring unit 220, for obtaining operation information by augmented reality equipment, in response to corresponding to operation information execution
Operation；
Processing unit 230, industrial robot is sent to for generating control instruction by augmented reality equipment, wherein, work
Industry robot performs respective operations according to control instruction.
Alternatively, acquiring unit is following any one or more for performing：User is obtained by augmented reality equipment to send out
The gesture instruction gone out；The phonetic order sent by augmented reality equipment acquisition user；Identification is passed through by augmented reality equipment
User eyeball active state obtains instruction；By augmented reality equipment by image recognition technology, the image gathered in real time is parsed
Obtain operation information；The operation information sent by augmented reality equipment acquisition production line control centre；Set by augmented reality
The standby operation information for obtaining industrial robot and sending.
The embodiment establishes communication connection using communication unit 210 by augmented reality equipment and industrial robot；Obtain
Unit 220 obtains operation information by augmented reality equipment, and corresponding operate is performed in response to operation information；Processing unit 230
Control instruction is generated by augmented reality equipment and is sent to industrial robot, wherein, industrial robot performs according to control instruction
Respective operations, solve in correlation technique in industrial production by unhandy problem during display screen auxiliary operation, Jin Erda
The effect for being produced by augmented reality equipment support industry and improving efficiency and security is arrived.
The control device of the augmented reality equipment includes processor and memory, above-mentioned communication unit, acquiring unit, place
Manage unit etc. to store in memory as program unit, by the said procedure unit of computing device storage in memory
To realize corresponding function.
Kernel is included in processor, is gone in memory to transfer corresponding program unit by kernel.Kernel can set one
Or more, produce raising efficiency and security by adjusting kernel parameter come support industry.
Memory may include computer-readable medium in volatile memory, random access memory (RAM) and/
Or the form such as Nonvolatile memory, such as read-only storage (ROM) or flash memory (flash RAM), memory includes at least one deposit
Store up chip.
It should be understood by those skilled in the art that, embodiments herein can be provided as method, system or computer program
Product.Therefore, the application can use the reality in terms of complete hardware embodiment, complete software embodiment or combination software and hardware
Apply the form of example.Moreover, the application can use the computer for wherein including computer usable program code in one or more
The computer program production that usable storage medium is implemented on (including but is not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of product.
The application is with reference to the flow according to the method for the embodiment of the present application, equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that can be by every first-class in computer program instructions implementation process figure and/or block diagram
Journey and/or the flow in square frame and flow chart and/or block diagram and/or the combination of square frame.These computer programs can be provided
The processors of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices is instructed to produce
A raw machine so that produced by the instruction of computer or the computing device of other programmable data processing devices for real
The device for the function of being specified in present one flow of flow chart or one square frame of multiple flows and/or block diagram or multiple square frames.
These computer program instructions, which may be alternatively stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that the instruction being stored in the computer-readable memory, which produces, to be included referring to
Make the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one square frame of block diagram or
The function of being specified in multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that counted
Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented processing, so as in computer or
The instruction performed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in individual square frame or multiple square frames.
Memory may include computer-readable medium in volatile memory, random access memory (RAM) and/
Or the form such as Nonvolatile memory, such as read-only storage (ROM) or flash memory (flash RAM).Memory is computer-readable Jie
The example of matter.
It should also be noted that, term " comprising ", "comprising" or its any other variant are intended to nonexcludability
Comprising so that process, method, commodity or equipment including a series of elements not only include those key elements, but also wrapping
Include the other element being not expressly set out, or also include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that including key element
Other identical element in process, method, commodity or equipment also be present.
It will be understood by those skilled in the art that embodiments herein can be provided as method, system or computer program product.
Therefore, the application can be using the embodiment in terms of complete hardware embodiment, complete software embodiment or combination software and hardware
Form.Deposited moreover, the application can use to can use in one or more computers for wherein including computer usable program code
The shape for the computer program product that storage media is implemented on (including but is not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
Embodiments herein is these are only, is not limited to the application.To those skilled in the art,
The application can have various modifications and variations.All any modifications made within spirit herein and principle, equivalent substitution,
Improve etc., it should be included within the scope of claims hereof.
Claims (10)
- A kind of 1. method that man-machine collaboration is controlled using augmented reality equipment, it is characterised in that including：Augmented reality equipment establishes communication connection with industrial robot；The augmented reality equipment obtains operation information, and corresponding operate is performed in response to the operation information；Augmented reality equipment generation control instruction is sent to the industrial robot, wherein, the industrial robot according to The control instruction performs respective operations.
- 2. according to the method for claim 1, it is characterised in that the augmented reality equipment obtains operation information including following It is any one or more：The augmented reality equipment obtains the gesture instruction that user sends；The augmented reality equipment obtains the phonetic order that user sends；The augmented reality equipment is by identifying that user eyeball or head movement state obtain instruction；The augmented reality equipment obtains the operation information that the industrial robot is sent；The augmented reality equipment parses the image acquisition operations information gathered in real time by image recognition technology；The augmented reality equipment obtains the operation information that production line control centre sends.
- 3. according to the method for claim 2, it is characterised in that after the augmented reality equipment obtains operation information, Methods described also includes：The augmented reality equipment updates display content according to the operation information；The augmented reality equipment uses processor module, generation motion planning and robot control instruction according to the operation information；The augmented reality equipment, using processor module, generates the work of the operator of production line according to the operation information Data, send to production line control centre；The augmented reality equipment determines the operating rate of the industrial robot, generation robot speed according to the operation information Rate control instruction, wherein, the operating rate of the industrial robot is used for the operation rate for coordinating the operator of current manufacturing lines.
- 4. according to the method for claim 3, it is characterised in that the augmented reality equipment updates according to the operation information Display content includes：The augmented reality equipment determines next step operation object according to the operation information；The augmented reality equipment generates and shows the prompt message of operation in next step；The augmented reality renewal of the equipment simultaneously shows the movement velocity of industrial robot, current location and joint angles；The augmented reality equipment shows the motor temperature of industrial robot.
- 5. according to the method for claim 4, it is characterised in that the augmented reality equipment is generated and shown and operates in next step Prompt message include：The prompting icon or word or 3D models of operation in next step are shown on object to be operated；And/orThe prompting icon or word or 3D models of operation in next step are shown in the augmented reality equipment visual field.
- 6. according to the method for claim 1, it is characterised in that the augmented reality equipment generation control instruction is sent to institute Stating industrial robot includes：Augmented reality equipment generation starts, suspended, terminating, speed change, restarts, resets, the control of governing speed, change pattern Instruction is sent to the industrial robot.
- 7. according to the method for claim 6, it is characterised in that the control instruction of the augmented reality equipment generation includes：The industrial robot is controlled to change displacement state, wherein, the displacement state includes the industrial machine The movement velocity of people, current location and joint angles；The industrial robot is controlled to change operation object；The industrial robot is controlled to change operator scheme；The industrial robot is controlled to change operation rate；Control the industrial robot that operating result is sent back into the augmented reality equipment.
- A kind of 8. device that man-machine collaboration is controlled using augmented reality equipment, it is characterised in that including：Communication unit, for establishing communication connection by augmented reality equipment and industrial robot；Acquiring unit, for obtaining operation information by the augmented reality equipment, correspondence is performed in response to the operation information Operation；Processing unit, the industrial robot is sent to for generating control instruction by the augmented reality equipment, wherein, institute State industrial robot and respective operations are performed according to the control instruction.
- 9. device according to claim 8, it is characterised in that the acquiring unit is used to perform following any one or more ：The gesture instruction sent by augmented reality equipment acquisition user；The phonetic order sent by augmented reality equipment acquisition user；By the augmented reality equipment by identifying that user eyeball or head movement state obtain instruction；The operation information of the industrial robot transmission is obtained by the augmented reality equipment；By the augmented reality equipment by image recognition technology, the image acquisition operations information gathered in real time is parsed；The operation information sent by augmented reality equipment acquisition production line control centre.
- A kind of 10. system that man-machine collaboration is controlled using augmented reality equipment, it is characterised in that including：Industrial equipment；Augmented reality equipment, the augmented reality equipment are used for perform claim and require that the use enhancing any one of 1-7 is existing The method of real equipment control man-machine collaboration；Industrial robot, the industrial equipment is controlled for coordinating with the augmented reality equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710720950.5A CN107656505A (en) | 2017-08-21 | 2017-08-21 | Use the methods, devices and systems of augmented reality equipment control man-machine collaboration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710720950.5A CN107656505A (en) | 2017-08-21 | 2017-08-21 | Use the methods, devices and systems of augmented reality equipment control man-machine collaboration |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107656505A true CN107656505A (en) | 2018-02-02 |
Family
ID=61128186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710720950.5A Pending CN107656505A (en) | 2017-08-21 | 2017-08-21 | Use the methods, devices and systems of augmented reality equipment control man-machine collaboration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107656505A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109732606A (en) * | 2019-02-13 | 2019-05-10 | 深圳大学 | Long-range control method, device, system and the storage medium of mechanical arm |
| CN109782902A (en) * | 2018-12-17 | 2019-05-21 | 中国科学院深圳先进技术研究院 | A kind of operation indicating method and glasses |
| CN109828673A (en) * | 2019-02-20 | 2019-05-31 | 上海昊沧系统控制技术有限责任公司 | The exchange method and system of object are followed in a kind of AR identification |
| CN110047148A (en) * | 2019-04-10 | 2019-07-23 | 珠海梅西互动技术有限公司 | A kind of the emulation interactive visual system and implementation method of virtual robot work station |
| CN111722705A (en) * | 2019-03-22 | 2020-09-29 | 伊顿智能动力有限公司 | Process monitoring |
| CN112368659A (en) * | 2018-06-26 | 2021-02-12 | 发纳科美国公司 | Automated dynamic diagnostic guidance with augmented reality |
| CN112667139A (en) * | 2020-12-11 | 2021-04-16 | 深圳市越疆科技有限公司 | Robot operation method, device, equipment and storage medium based on augmented reality |
| CN112686227A (en) * | 2021-03-12 | 2021-04-20 | 泰瑞数创科技(北京)有限公司 | Product quality inspection method and device based on augmented reality and man-machine comprehensive detection |
| CN113313346A (en) * | 2021-04-19 | 2021-08-27 | 贵州电网有限责任公司 | Visual implementation method of artificial intelligence scheduling operation based on AR glasses |
| CN114706486A (en) * | 2022-04-26 | 2022-07-05 | 四川大学 | Mixed reality industrial control method and device based on gesture recognition |
| CN114792404A (en) * | 2022-04-26 | 2022-07-26 | 北京大学 | AR enhancement auxiliary repair control platform, method, medium and equipment |
| CN115416015A (en) * | 2022-07-26 | 2022-12-02 | 北京航空航天大学 | Virtual-real fusion process guiding method and system for man-machine cooperation drilling and riveting |
| CN115847428A (en) * | 2023-02-20 | 2023-03-28 | 中国信息通信研究院 | AR technology-based mechanical assembly auxiliary guide system and method |
| CN116679641A (en) * | 2023-06-01 | 2023-09-01 | 盐城工学院 | Multi-objective optimization method of automatic assembly production line considering process rigidity constraint |
| CN117008723A (en) * | 2023-07-14 | 2023-11-07 | 广东火种源智能科技有限公司 | Autonomous mobile robot with augmented reality display and interaction technology energized |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101794349A (en) * | 2010-02-09 | 2010-08-04 | 北京邮电大学 | Experimental system and method for augmented reality of teleoperation of robot |
| CN103213127A (en) * | 2013-03-29 | 2013-07-24 | 山东轻工业学院 | Press-robot synchronous movement coordination method |
| CN205519332U (en) * | 2016-03-02 | 2016-08-31 | 东莞市元一自动化设备有限公司 | Full -automatic punching production line |
| CN105975232A (en) * | 2016-05-06 | 2016-09-28 | 深圳市吾悦科技有限公司 | Real-time interaction system and method for augmented reality |
| CN106257356A (en) * | 2016-03-25 | 2016-12-28 | 深圳增强现实技术有限公司 | Intelligence helmet and intelligence donning system |
| CN106363637A (en) * | 2016-10-12 | 2017-02-01 | 华南理工大学 | Fast teaching method and device for robot |
| CN106772996A (en) * | 2017-01-23 | 2017-05-31 | 清华大学 | A kind of augmented reality operating method and system |
| JP2017104944A (en) * | 2015-12-10 | 2017-06-15 | ファナック株式会社 | Robot system provided with video display device for superimposingly displaying image of virtual object on robot video |
| CN107065810A (en) * | 2017-06-05 | 2017-08-18 | 深圳增强现实技术有限公司 | A kind of method and system of augmented reality industrial operation auxiliary |
-
2017
- 2017-08-21 CN CN201710720950.5A patent/CN107656505A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101794349A (en) * | 2010-02-09 | 2010-08-04 | 北京邮电大学 | Experimental system and method for augmented reality of teleoperation of robot |
| CN103213127A (en) * | 2013-03-29 | 2013-07-24 | 山东轻工业学院 | Press-robot synchronous movement coordination method |
| JP2017104944A (en) * | 2015-12-10 | 2017-06-15 | ファナック株式会社 | Robot system provided with video display device for superimposingly displaying image of virtual object on robot video |
| CN205519332U (en) * | 2016-03-02 | 2016-08-31 | 东莞市元一自动化设备有限公司 | Full -automatic punching production line |
| CN106257356A (en) * | 2016-03-25 | 2016-12-28 | 深圳增强现实技术有限公司 | Intelligence helmet and intelligence donning system |
| CN105975232A (en) * | 2016-05-06 | 2016-09-28 | 深圳市吾悦科技有限公司 | Real-time interaction system and method for augmented reality |
| CN106363637A (en) * | 2016-10-12 | 2017-02-01 | 华南理工大学 | Fast teaching method and device for robot |
| CN106772996A (en) * | 2017-01-23 | 2017-05-31 | 清华大学 | A kind of augmented reality operating method and system |
| CN107065810A (en) * | 2017-06-05 | 2017-08-18 | 深圳增强现实技术有限公司 | A kind of method and system of augmented reality industrial operation auxiliary |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112368659A (en) * | 2018-06-26 | 2021-02-12 | 发纳科美国公司 | Automated dynamic diagnostic guidance with augmented reality |
| CN109782902A (en) * | 2018-12-17 | 2019-05-21 | 中国科学院深圳先进技术研究院 | A kind of operation indicating method and glasses |
| CN109732606A (en) * | 2019-02-13 | 2019-05-10 | 深圳大学 | Long-range control method, device, system and the storage medium of mechanical arm |
| CN109828673B (en) * | 2019-02-20 | 2022-12-06 | 上海昊沧系统控制技术有限责任公司 | Interaction method and system for following object in AR recognition |
| CN109828673A (en) * | 2019-02-20 | 2019-05-31 | 上海昊沧系统控制技术有限责任公司 | The exchange method and system of object are followed in a kind of AR identification |
| CN111722705A (en) * | 2019-03-22 | 2020-09-29 | 伊顿智能动力有限公司 | Process monitoring |
| CN110047148A (en) * | 2019-04-10 | 2019-07-23 | 珠海梅西互动技术有限公司 | A kind of the emulation interactive visual system and implementation method of virtual robot work station |
| CN110047148B (en) * | 2019-04-10 | 2024-01-26 | 珠海惠中智能技术有限公司 | Simulation interaction visualization system of virtual robot workstation and implementation method |
| CN112667139A (en) * | 2020-12-11 | 2021-04-16 | 深圳市越疆科技有限公司 | Robot operation method, device, equipment and storage medium based on augmented reality |
| CN112686227B (en) * | 2021-03-12 | 2021-07-06 | 泰瑞数创科技(北京)有限公司 | Product quality inspection method and device based on augmented reality and man-machine comprehensive detection |
| CN112686227A (en) * | 2021-03-12 | 2021-04-20 | 泰瑞数创科技(北京)有限公司 | Product quality inspection method and device based on augmented reality and man-machine comprehensive detection |
| CN113313346A (en) * | 2021-04-19 | 2021-08-27 | 贵州电网有限责任公司 | Visual implementation method of artificial intelligence scheduling operation based on AR glasses |
| CN114706486A (en) * | 2022-04-26 | 2022-07-05 | 四川大学 | Mixed reality industrial control method and device based on gesture recognition |
| CN114792404A (en) * | 2022-04-26 | 2022-07-26 | 北京大学 | AR enhancement auxiliary repair control platform, method, medium and equipment |
| CN114792404B (en) * | 2022-04-26 | 2022-11-15 | 北京大学 | AR enhancement auxiliary repair control platform, method, medium and equipment |
| CN115416015A (en) * | 2022-07-26 | 2022-12-02 | 北京航空航天大学 | Virtual-real fusion process guiding method and system for man-machine cooperation drilling and riveting |
| CN115847428A (en) * | 2023-02-20 | 2023-03-28 | 中国信息通信研究院 | AR technology-based mechanical assembly auxiliary guide system and method |
| CN116679641A (en) * | 2023-06-01 | 2023-09-01 | 盐城工学院 | Multi-objective optimization method of automatic assembly production line considering process rigidity constraint |
| CN117008723A (en) * | 2023-07-14 | 2023-11-07 | 广东火种源智能科技有限公司 | Autonomous mobile robot with augmented reality display and interaction technology energized |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107656505A (en) | Use the methods, devices and systems of augmented reality equipment control man-machine collaboration | |
| US11467709B2 (en) | Mixed-reality guide data collection and presentation | |
| CN107168537B (en) | Cooperative augmented reality wearable operation guidance method and system | |
| US20120293506A1 (en) | Avatar-Based Virtual Collaborative Assistance | |
| JP2022000640A (en) | Information processing device, information processing method, and information processing program | |
| CN117120962A (en) | Controlling two-handed interactions between mapped hand regions of virtual and graphical elements | |
| CN107577045B (en) | The method, apparatus and storage medium of predicting tracing for head-mounted display | |
| JP2023018097A (en) | Augmented reality adjustment of interaction between human and robot | |
| DE202016008297U1 (en) | Two-handed object manipulations in virtual reality | |
| KR102051309B1 (en) | Intelligent technology based augmented reality system | |
| CN106454322A (en) | VR Image processing system and method thereof | |
| CN106363637A (en) | Fast teaching method and device for robot | |
| EP1709519A1 (en) | A virtual control panel | |
| CN107390863A (en) | Control method and device, electronic equipment, the storage medium of equipment | |
| JP2020126455A (en) | Spatial information management device | |
| CN107357434A (en) | Information input equipment, system and method under a kind of reality environment | |
| CN111546338A (en) | Robot control method and device, robot and storage medium | |
| KR20200025960A (en) | Intelligent technology based augmented reality system | |
| US11651528B2 (en) | Systems and methods for providing context-based data for an industrial automation system | |
| US20230091778A1 (en) | Systems and methods for modifying context-based data provided for an industrial automation system | |
| US20230092405A1 (en) | Systems and methods for providing context-based data for an industrial automation system | |
| WO2022176450A1 (en) | Information processing device, information processing method, and program | |
| US11829126B2 (en) | Systems and methods for providing context-based data for an industrial automation system based on a virtual coordinate system | |
| US20230089251A1 (en) | Systems and methods for providing context-based data for an industrial automation system | |
| Kaswan et al. | AI‐Based AR/VR Models in Biomedical Sustainable Industry 4.0 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180202 |