CN110061755A - Operation householder method and system, wearable device and engineering truck - Google Patents
Operation householder method and system, wearable device and engineering truck Download PDFInfo
- Publication number
- CN110061755A CN110061755A CN201910359830.6A CN201910359830A CN110061755A CN 110061755 A CN110061755 A CN 110061755A CN 201910359830 A CN201910359830 A CN 201910359830A CN 110061755 A CN110061755 A CN 110061755A
- Authority
- CN
- China
- Prior art keywords
- wearable device
- image
- commanding
- commander
- conductor image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 239000004020 conductor Substances 0.000 claims abstract description 121
- 230000004044 response Effects 0.000 claims abstract description 24
- 238000004891 communication Methods 0.000 claims abstract description 13
- 230000006870 function Effects 0.000 claims description 17
- 238000003860 storage Methods 0.000 claims description 9
- 230000003190 augmentative effect Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 7
- 230000004927 fusion Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 15
- 230000008569 process Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000002360 preparation method Methods 0.000 description 4
- 239000000725 suspension Substances 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000003825 pressing Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000155 melt Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B1/00—Details of transmission systems, not covered by a single one of groups H04B3/00 - H04B13/00; Details of transmission systems not characterised by the medium used for transmission
- H04B1/38—Transceivers, i.e. devices in which transmitter and receiver form a structural unit and in which at least one part is used for functions of transmitting and receiving
- H04B1/3827—Portable transceivers
- H04B1/385—Transceivers carried on the body, e.g. in helmets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
- H04N7/185—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source from a mobile camera, e.g. for remote control
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Present disclose provides a kind of operation householder methods and system, wearable device and engineering truck, are related to engineering truck technical field, and the operation householder method includes: that wearable device and engineering truck operate indoor terminal device and establish communication connection；The real-time video picture of operation field is shared to the terminal device by the wearable device；In response to the operation of operation field commanding, the wearable device determines at least one of the tag image of conductor image and operative goals；At least one of the conductor image and the tag image are fused in the real-time video picture of operation field by the wearable device, with assist operators' operation.
Description
Technical field
This disclosure relates to engineering truck technical field, especially a kind of operation householder method and system, wearable device and
Engineering truck.
Background technique
It is greatly developed along with basic economy, people's livelihood engineering, the engineering truck of such as crane is in engineering construction, track
The fields such as traffic and rescue play an important role.
Crane hanging component operation belongs to one kind of dangerous operation.The generation of accident in order to prevent needs person to person, people and rises
Heavy-duty machine reaches the coordinating and unifying.The completion of job task need commanding, department Suo work and operator close fit and mutually
Cooperation, the degree of cooperation of each work post personnel directly affect the efficiency and safety of lifting operation.
Traditional personnel's fit system is mainly carried out by video monitoring, audio intercommunication, gesture guidance.
Summary of the invention
Inventors noted that being limited by construction cost and space, camera can only be deployed in local key area, therefore regard
Frequency monitoring can only cover key area, can not cover whole operating environment.Therefore, operator can not be clear in control cabinl pulpit
See suspended object and ambient enviroment in ground.Commanding is influenced although audio intercom system can be carried, when using to carry out
Limbs commander movement, also will affect the operation of operator.In addition, operator apart from commanding farther out when, it is more difficult to it is clear
Commander's gesture of commanding is seen, to influence the accurate judgement and operation decision to suspended object.Therefore using traditional personnel
Fit system, operating efficiency are low.
To solve the above-mentioned problems, the embodiment of the present disclosure proposes following solution.
According to the one side of the embodiment of the present disclosure, a kind of operation householder method is provided, comprising: wearable device and engineering truck
The indoor terminal device of operation establishes communication connection；The wearable device shares to the real-time video picture of operation field
The terminal device；In response to the operation of operation field commanding, the wearable device determines conductor image and operation mesh
At least one of target tag image；The wearable device is by least one in the conductor image and the tag image
In a real-time video picture for being fused to operation field, with assist operators' operation.
In some embodiments, the operation in response to operation field commanding, the wearable device determination refer to
At least one of the tag image for waving image and operative goals includes: that the first behaviour of command function is opened in response to commanding
Make, the wearable device determines the conductor image according to commander's operation information of commanding；It is opened in response to commanding
The second operation of mark function is opened, the wearable device is according to the gesture motion determination of commanding's label operative goals
Tag image.
In some embodiments, commander's operation information includes at least one of commander's voice and commander's gesture.
In some embodiments, the wearable device determines that the commander schemes according to commander's operation information of commanding
As including: that the wearable device identifies commander's voice, to obtain keyword；It is corresponding out according to the Keywords matching
Commander's instruction；The conductor image is matched from conductor image library according to the commander's instruction matched.
In some embodiments, commander's instruction that the basis matches matches commander's figure from conductor image library
The first conductor image is matched from conductor image library according to the commander's instruction matched as including: the wearable device；Institute
It states wearable device and the second conductor image is matched from conductor image library according to commander's gesture；The wearable device is sentenced
Whether first conductor image of breaking and second conductor image are identical；In first conductor image and second commander
In the identical situation of image, using first conductor image or second conductor image as the conductor image.
In some embodiments, the wearable device marks the gesture motion of operative goals to determine institute according to commanding
State tag image includes: to judge whether the gesture of commanding is specified gesture；If the gesture of commanding is specified gesture, sentence
Whether the gesture motion of disconnected commanding is required movement；If the gesture motion of commanding is required movement, in finger tip
The tag image is generated at position in a manner of augmented reality.
According to the another aspect of the embodiment of the present disclosure, a kind of wearable device is provided, comprising: communication module is used for and work
The indoor terminal device of journey vehicle operating establishes communication connection；Sharing module, for the real-time video picture of operation field to be total to
It enjoys to the terminal device；Determining module, for the operation in response to operation field commanding, the wearable device is determined
At least one of conductor image and the tag image of operative goals；Fusion Module is used for the conductor image and the mark
At least one of note image is fused in the real-time video picture of operation field.
In some embodiments, the determining module is used for: the first operation of command function is opened in response to commanding,
The wearable device determines the conductor image according to commander's operation information of commanding；It opens and marks in response to commanding
Cite sb. for meritorious service can second operation, the wearable device according to commanding mark operative goals gesture motion determine the label
Image.
In some embodiments, commander's operation information includes at least one of commander's voice and commander's gesture.
In some embodiments, the determining module is used for: commander's voice is identified, to obtain keyword；According to institute
It states Keywords matching and goes out corresponding commander's instruction；The commander is matched from conductor image library according to the commander's instruction matched
Image.
In some embodiments, the determining module is used for: being instructed from conductor image library according to the commander that matches
Allot the first conductor image；The second conductor image is matched from conductor image library according to commander's gesture；It is described wearable
Equipment judges whether first conductor image and second conductor image are identical；In first conductor image and described the
In the identical situation of two conductor images, using first conductor image or second conductor image as the conductor image.
In some embodiments, the determining module is used for: whether the gesture for judging commanding is specified gesture；If referring to
The gesture for waving personnel is specified gesture, judges whether the gesture motion of commanding is required movement；If the gesture of commanding
Movement is required movement, then generates the tag image in a manner of augmented reality at the position of finger tip.
According to the another aspect of the embodiment of the present disclosure, a kind of wearable device is provided, comprising: memory；And it is coupled to
The processor of the memory, the processor is configured to above-mentioned any one based on the instruction execution for being stored in the memory
Method described in a embodiment.
According to the another aspect of the embodiment of the present disclosure, a kind of operation auxiliary system is provided, comprising: any one above-mentioned implementation
Wearable device described in example；And terminal device, it is communicated to connect for being established with the wearable device；It receives described wearable
The real-time video picture of the operation field of collaborative share is simultaneously shown.
According to the embodiment of the present disclosure in another aspect, providing a kind of engineering truck, comprising: any one above-mentioned embodiment institute
The operation auxiliary system stated.
In some embodiments, the engineering truck includes crane.
According to the also one side of the embodiment of the present disclosure, a kind of computer readable storage medium is provided, calculating is stored thereon with
Method described in any one above-mentioned embodiment is realized in machine program instruction, the instruction when being executed by processor.
In the embodiment of the present disclosure, wearable device melts at least one of the tag image of operative goals and conductor image
It closes in the real-time video picture of operation field, preferably carries out operation decision so as to assist operators, improve work
The accuracy and operating efficiency of industry.In addition, wearable device is more just relative to the mode in fixed position deployment camera
It takes, and easy to operate, substantially increases the portability of operation.
Below by drawings and examples, the technical solution of the disclosure is described in further detail.
Detailed description of the invention
In order to illustrate more clearly of the embodiment of the present disclosure or technical solution in the prior art, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Disclosed some embodiments for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is the flow diagram according to the operation householder method of some embodiments of the disclosure；
Fig. 2 is the effect diagram shared according to the picture of some embodiments of the disclosure；
Fig. 3 is the structural schematic diagram according to the wearable device of some embodiments of the disclosure；
Fig. 4 is the structural schematic diagram according to the wearable device of the disclosure other embodiments；
Fig. 5 is the structural schematic diagram according to the operation auxiliary system of some embodiments of the disclosure.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present disclosure, the technical solution in the embodiment of the present disclosure is carried out clear, complete
Site preparation description, it is clear that described embodiment is only disclosure a part of the embodiment, instead of all the embodiments.It is based on
Embodiment in the disclosure, those of ordinary skill in the art without creative labor it is obtained it is all its
His embodiment belongs to the range of disclosure protection.
Unless specifically stated otherwise, positioned opposite, the digital table of the component and step that otherwise illustrate in these embodiments
Up to the unlimited the scope of the present disclosure processed of formula and numerical value.
Simultaneously, it should be appreciated that for ease of description, the size of various pieces shown in attached drawing is not according to reality
Proportionate relationship draw.
Technology, method and apparatus known to person of ordinary skill in the relevant may be not discussed in detail, but suitable
In the case of, the technology, method and apparatus should be considered as authorizing part of specification.
It is shown here and discuss all examples in, any occurrence should be construed as merely illustratively, without
It is as limitation.Therefore, the other examples of exemplary embodiment can have different values.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, then in subsequent attached drawing does not need that it is further discussed.
Fig. 1 is the flow diagram according to the operation householder method of some embodiments of the disclosure.
In step 102, wearable device and engineering truck operate indoor terminal device and establish communication connection.
For example, wearable device can be set by WiFi module, bluetooth module or MiracastP2P module etc. with terminal
It is standby to establish wireless communication connection.It is connect in another example wearable device can also establish wire communication with terminal device.
As an example, wearable device for example can be intelligent glasses etc., can be worn by commanding.Terminal device example
It such as can be smart phone, Intelligent flat, intelligent glasses, car-mounted terminal can be integrated into engineering truck.Here, engineering truck
It for example can include but is not limited to crane, such as wheeled crane etc..
In step 104, the real-time video picture of operation field is shared to terminal device by wearable device.
It is set for example, the real-time video picture of operation field can be shared to terminal by way of throwing screen by wearable device
It is standby.For example, the throwing screen that wearable device can be initiated in response to terminal device is requested, into throwing screen mode.In turn, wearable to set
The standby operation that the default application program in wearable device is opened in response to commanding, it is automatic to start camera and shooting operation
The real-time video picture at scene.With the movement of commanding, the real-time video picture of operation field can also change.
In step 106, in response to the operation of operation field commanding, wearable device determines conductor image and operation mesh
At least one of target tag image.
Operator could be aware that the commander of commanding acts according to conductor image, according to the tag image of operative goals
It could be aware that the position of operative goals.As an example, operative goals for example can be target to be hung.
For example, opening the first operation of command function in response to commanding, wearable device is according to the finger of commanding
It waves operation information and determines conductor image.Here, commander's operation information may include at least one commanded in voice and commander's gesture
Kind.It will be described in detail the mode of determining conductor image hereinafter.First operation for example can be short-press key, long-pressing key, double-click
Or any one mode of operation of multi-hit Trackpad.
In another example in response to the second operation of commanding's turn-on flag function, wearable device is marked according to commanding
The gesture motion of note operative goals determines the tag image of operative goals.Second operation for example can be short-press key, long-pressing by
Any one mode of operation of key, double-click or multi-hit Trackpad, and it is different from the first operation.
In step 108, at least one of conductor image and tag image are fused to operation field by wearable device
In real-time video picture.
For example, conductor image can be fused to not the predeterminated position for influencing observation, tag image is fused to and engineering
The relative position of vehicle is able to reflect the position at actual job scene.
In some embodiments, the third for closing command function in response to commanding operates, and conductor image is existing from operation
It disappears in the real-time video picture of field.In some embodiments, the 4th operation of mark function, mark are closed in response to commanding
Note image disappears from the real-time video picture of operation field.Third operation and the 4th operation can be and the first, second operation
Different any operations.
Fig. 2 is the effect diagram shared according to the picture of some embodiments of the disclosure.
As shown in Fig. 2, including the label figure of engineering truck itself, operative goals in the real-time video picture of operation field
Picture and conductor image.In the real-time video picture of operation field, the relative position and reality of conductor image and engineering truck
Operation field is the same.Conductor image for example can be preset images of gestures.As an example, conductor image can be located at view in real time
Lower right corner of frequency picture etc. and engineering truck and the nonoverlapping position of tag image.
In above-described embodiment, wearable device merges at least one of the tag image of operative goals and conductor image
Into the real-time video picture of operation field, operation decision is preferably carried out so as to assist operators, improves operation
Accuracy and operating efficiency.In addition, wearable device is more portable relative to the mode in fixed position deployment camera,
And it is easy to operate, substantially increase the portability of operation.
Wearable device according to commander's operation information of commanding determine conductor image can by different modes come
It realizes.It will be described in detail respectively below.
In some implementations, wearable device determines conductor image according to commander's voice of commanding.
Firstly, wearable device can identify commander's voice of commanding, to obtain keyword；Then, according to key
Word matches corresponding commander's instruction；Later, it is instructed according to the commander matched and matches conductor image from conductor image library.
Conductor image for example can be the image of GB5082-85 standard.
For example, as shown in table 1 below, commander's voice is " preparation ", and the keyword identified can be " preparation ", commander's instruction
It can be preparation, corresponding conductor image can be the image raised one's hand.In another example commander's voice is " operating suspension hook upwards ",
The keyword identified can be " suspension hook is upward " or " hook is upward ", and commander's instruction can be " suspension hook rising ", corresponding commander
Image can be the image upward comprising suspension hook and arm.
Table 1
In other implementations, wearable device determines conductor image according to commander's gesture of commanding.Example
Such as, wearable device can directly match corresponding conductor image according to commander's gesture from conductor image library.For example, can be with
Commander's gesture is identified, commander's gesture is then converted into corresponding conductor image.
In other implementation, wearable device is according to both commander's voice of commanding and commander's gesture come really
Determine conductor image.
For example, wearable device can be instructed according to the commander matched matches the first commander figure from conductor image library
Picture.In addition, wearable device can match the second conductor image according to commander's gesture from conductor image library.Then, it can wear
It wears equipment and judges whether the first conductor image is identical as the second conductor image.It is identical as the second conductor image in the first conductor image
In the case where, using the first conductor image or the second conductor image as conductor image.The conductor image that such mode determines is more
Accurately.
Wearable device is described below and marks the gesture motion of operative goals to determine the reality of tag image according to commanding
Existing mode.
First, it is determined that whether the gesture of commanding is specified gesture.For example, specified gesture can be specified hand shape wheel
Exterior feature, or the profile to specify finger.For example, can consider that gesture is effective, as refers to after identifying specified hand shape profile
Determine gesture.
If the gesture of commanding is specified gesture, judge whether the gesture motion of commanding is required movement.For example,
Required movement can be the movement similar to down button.
If the gesture motion of commanding is required movement, mark is generated in a manner of augmented reality at the position of finger tip
Remember image.At the position of finger tip i.e. commanding be directed toward operative goals.The tag image that is generated in a manner of augmented reality with
The relative position of engineering truck can reflect the two in the relative position of operation field.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with its
The difference of its embodiment, the same or similar part cross-reference between each embodiment.For wearable device
For embodiment, since it is substantially corresponding with embodiment of the method, so being described relatively simple, related place is implemented referring to method
The part explanation of example.
Fig. 3 is the structural schematic diagram according to the wearable device of some embodiments of the disclosure.
As shown in figure 3, wearable device includes communication module 301, sharing module 302, determining module 303 and Fusion Module
304。
Communication module 301 is used for the indoor terminal device of operation with engineering truck and establishes communication connection.Sharing module 302
For the real-time video picture of operation field to be shared to terminal device by way of throwing screen.Determining module 303 is for responding
In the operation of operation field commanding, wearable device determines at least one in conductor image and the tag image of operative goals
It is a.The real-time video that Fusion Module 304 is used at least one of conductor image and tag image being fused to operation field is drawn
In face.
In some embodiments, determining module 303 is used for: the first operation of command function is opened in response to commanding,
Wearable device determines conductor image according to commander's operation information of commanding；In response to commanding's turn-on flag function
Second operation, wearable device mark the gesture motion of operative goals to determine tag image according to commanding.For example, commander behaviour
Making information includes at least one of commander's voice and commander's gesture.
In some embodiments, determining module 303 is used to determine conductor image according to such as under type: identification commander's voice,
To obtain keyword；Go out corresponding commander's instruction according to Keywords matching；It is instructed according to the commander matched from conductor image library
In match conductor image.
In some embodiments, determining module 303 is used to determine conductor image according to such as under type: according to the finger matched
It waves instruction and matches the first conductor image from conductor image library；The second finger is matched from conductor image library according to commander's gesture
Wave image；Wearable device judges whether the first conductor image is identical as the second conductor image；In the first conductor image and second
In the identical situation of conductor image, using the first conductor image or the second conductor image as conductor image.
In some embodiments, determining module 303 is used to determine tag image according to such as under type: judging commanding
Whether gesture is specified gesture；If the gesture of commanding is specified gesture, judge whether the gesture motion of commanding is finger
Fixed movement；If the gesture motion of commanding is required movement, mark is generated in a manner of augmented reality at the position of finger tip
Remember image.
Fig. 4 is the structural schematic diagram according to the wearable device of the disclosure other embodiments.
As shown in figure 4, the wearable device 400 of the embodiment is including memory 401 and is coupled to the memory 401
Processor 402, processor 402 are configured as executing any one aforementioned embodiment based on the instruction being stored in memory 401
Operation householder method.
Memory 401 is such as may include system storage, fixed non-volatile memory medium.System storage is for example
Operating system, application program, Boot loader (BootLoader) and other programs etc. are can store.
Wearable device 400 can also include input/output interface 403, network interface 404, memory interface 405 etc..These
It can for example be connected by bus 406 between interface 403,404,405 and between memory 401 and processor 402.Input
The input-output equipment such as output interface 403 is display, mouse, keyboard, touch screen provide connecting interface.Network interface 404 is
Various networked devices provide connecting interface.The external storages such as memory interface 405 is SD card, USB flash disk provide connecting interface.
Fig. 5 is the structural schematic diagram according to the operation auxiliary system of some embodiments of the disclosure.
As shown in figure 5, operation auxiliary system includes the wearable device 501 and terminal device of any one above-mentioned embodiment
502.Terminal device 502 is used to establish with wearable device 501 and communicate to connect；It is existing to receive the shared operation of wearable device 501
The real-time video picture of field is simultaneously shown.Wearable device 501 can be by assisting any wearing, and terminal device 502 is located at operating room
It is interior.
The embodiment of the present disclosure additionally provides a kind of engineering truck, comprising: the operation auxiliary system of any one above-mentioned embodiment
System.Engineering truck for example can be crane etc..
The embodiment of the present disclosure additionally provides a kind of computer readable storage medium, is stored thereon with computer program instructions,
Wherein, the operation householder method of any one above-mentioned embodiment is realized when which is executed by processor.
Those skilled in the art should be understood that embodiment of the disclosure can provide as method, system or computer journey
Sequence product.Therefore, complete hardware embodiment, complete software embodiment or combining software and hardware aspects can be used in the disclosure
The form of embodiment.Moreover, it wherein includes the calculating of computer usable program code that the disclosure, which can be used in one or more,
Machine can use the meter implemented in non-transient storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of calculation machine program product.
The disclosure is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present disclosure
Figure and/or block diagram describe.It should be understood that a process or multiple processes in flow chart can be realized by computer program instructions
And/or the function of being specified in one box or multiple boxes in block diagram.These computer program instructions be can provide to general meter
Calculation machine, special purpose computer, Embedded Processor or other programmable data processing devices processor to generate a machine, make
It obtains and is generated by the instruction that computer or the processor of other programmable data processing devices execute for realizing in flow chart one
The device for the function of being specified in a process or multiple processes and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
The foregoing is merely the preferred embodiments of the disclosure, not to limit the disclosure, all spirit in the disclosure and
Within principle, any modification, equivalent replacement, improvement and so on be should be included within the protection scope of the disclosure.
Claims (17)
1. a kind of operation householder method, comprising:
Wearable device and engineering truck operate indoor terminal device and establish communication connection；
The real-time video picture of operation field is shared to the terminal device by the wearable device；
In response to the operation of operation field commanding, the wearable device determines the label figure of conductor image and operative goals
At least one of as；
At least one of the conductor image and the tag image are fused to the reality of operation field by the wearable device
When video pictures in, with assist operators' operation.
2. operation householder method according to claim 1, wherein the operation in response to operation field commanding,
The wearable device determines that at least one of the tag image of conductor image and operative goals includes:
The first operation of command function is opened in response to commanding, the wearable device is operated according to the commander of commanding
Information determines the conductor image；
In response to the second operation of commanding's turn-on flag function, the wearable device marks operation mesh according to commanding
Target gesture motion determines the tag image.
3. operation householder method according to claim 2, wherein commander's operation information includes commander's voice and commander
At least one of gesture.
4. operation householder method according to claim 3, wherein the wearable device is grasped according to the commander of commanding
Determine that the conductor image includes: as information
The wearable device identifies commander's voice, to obtain keyword；
Go out corresponding commander's instruction according to the Keywords matching；
The conductor image is matched from conductor image library according to the commander's instruction matched.
5. operation householder method according to claim 4, wherein the commander that the basis matches instructs from conductor image
The conductor image is matched in library includes:
The wearable device matches the first conductor image according to the commander's instruction matched from conductor image library；
The wearable device matches the second conductor image according to commander's gesture from conductor image library；
The wearable device judges whether first conductor image and second conductor image are identical；
In first conductor image situation identical with second conductor image, with first conductor image or described
Second conductor image is as the conductor image.
6. operation householder method according to claim 2, wherein the wearable device marks operation according to commanding
The gesture motion of target determines that the tag image includes:
Whether the gesture for judging commanding is specified gesture；
If the gesture of commanding is specified gesture, judge whether the gesture motion of commanding is required movement；
If the gesture motion of commanding is required movement, the mark is generated in a manner of augmented reality at the position of finger tip
Remember image.
7. a kind of wearable device, comprising:
Communication module establishes communication connection for operating indoor terminal device with engineering truck；
Sharing module, for the real-time video picture of operation field to be shared to the terminal device；
Determining module, for the operation in response to operation field commanding, the wearable device determines conductor image and work
At least one of the tag image of industry target；
Fusion Module, at least one of the conductor image and the tag image to be fused to the real-time of operation field
In video pictures.
8. wearable device according to claim 7, wherein the determining module is used for:
The first operation of command function is opened in response to commanding, the wearable device is operated according to the commander of commanding
Information determines the conductor image；
In response to the second operation of commanding's turn-on flag function, the wearable device marks operation mesh according to commanding
Target gesture motion determines the tag image.
9. wearable device according to claim 8, wherein commander's operation information includes commander's voice and commander's hand
At least one of gesture.
10. wearable device according to claim 9, wherein the determining module is used for:
Commander's voice is identified, to obtain keyword；
Go out corresponding commander's instruction according to the Keywords matching；
The conductor image is matched from conductor image library according to the commander's instruction matched.
11. wearable device according to claim 10, wherein the determining module is used for:
The first conductor image is matched from conductor image library according to the commander's instruction matched；
The second conductor image is matched from conductor image library according to commander's gesture；
The wearable device judges whether first conductor image and second conductor image are identical；
In first conductor image situation identical with second conductor image, with first conductor image or described
Second conductor image is as the conductor image.
12. wearable device according to claim 8, wherein the determining module is used for:
Whether the gesture for judging commanding is specified gesture；
If the gesture of commanding is specified gesture, judge whether the gesture motion of commanding is required movement；
If the gesture motion of commanding is required movement, the mark is generated in a manner of augmented reality at the position of finger tip
Remember image.
13. a kind of wearable device, comprising:
Memory；And
It is coupled to the processor of the memory, the processor is configured to the instruction based on storage in the memory,
Perform claim requires method described in 1-6 any one.
14. a kind of operation auxiliary system, comprising:
Wearable device described in claim 8-13 any one；With
Terminal device is communicated to connect for establishing with the wearable device；It is existing to receive the shared operation of the wearable device
The real-time video picture of field is simultaneously shown.
15. a kind of engineering truck, comprising: operation auxiliary system described in claim 14.
16. engineering truck according to claim 15, wherein the engineering truck includes crane.
17. a kind of computer readable storage medium, is stored thereon with computer program instructions, wherein the instruction is held by processor
Method as claimed in any one of claims 1 to 6 is realized when row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910359830.6A CN110061755B (en) | 2019-04-30 | 2019-04-30 | Operation assisting method and system, wearable device and engineering vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910359830.6A CN110061755B (en) | 2019-04-30 | 2019-04-30 | Operation assisting method and system, wearable device and engineering vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110061755A true CN110061755A (en) | 2019-07-26 |
| CN110061755B CN110061755B (en) | 2021-04-16 |
Family
ID=67321779
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910359830.6A Active CN110061755B (en) | 2019-04-30 | 2019-04-30 | Operation assisting method and system, wearable device and engineering vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110061755B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112883754A (en) * | 2019-11-29 | 2021-06-01 | 华晨宝马汽车有限公司 | Auxiliary operation system of forklift |
| CN114415837A (en) * | 2022-01-25 | 2022-04-29 | 中国农业银行股份有限公司 | Operation auxiliary system and method |
| WO2022199264A1 (en) * | 2021-03-22 | 2022-09-29 | International Business Machines Corporation | Multi-user interactive ad shopping using wearable device gestures |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2813563Y (en) * | 2005-04-20 | 2006-09-06 | 苗小平 | Tower crane visual monitoring device |
| CN104063039A (en) * | 2013-03-18 | 2014-09-24 | 朱慧灵 | Human-computer interaction method of wearable computer intelligent terminal |
| CN106339094A (en) * | 2016-09-05 | 2017-01-18 | 山东万腾电子科技有限公司 | Interactive remote expert cooperation maintenance system and method based on augmented reality technology |
| CN107203272A (en) * | 2017-06-23 | 2017-09-26 | 山东万腾电子科技有限公司 | Wearable augmented reality task instruction system and method based on myoelectricity cognition technology |
| CN206654664U (en) * | 2017-03-20 | 2017-11-21 | 河南师范大学 | A kind of control tower crane based on virtual reality technology |
| CN107886172A (en) * | 2017-10-19 | 2018-04-06 | 中车青岛四方机车车辆股份有限公司 | System is supported in a kind of remote visualization maintenance |
| CN108509025A (en) * | 2018-01-26 | 2018-09-07 | 吉林大学 | A kind of crane intelligent Lift-on/Lift-off System based on limb action identification |
| CN108879440A (en) * | 2018-06-20 | 2018-11-23 | 国网山东省电力公司济宁供电公司 | Intelligent examination and repair system and method based on wearable augmented reality display terminal and cloud platform |
| CN109405804A (en) * | 2018-11-05 | 2019-03-01 | 徐州重型机械有限公司 | Operation householder method and system |
| CN109446876A (en) * | 2018-08-31 | 2019-03-08 | 百度在线网络技术（北京）有限公司 | Sign language information processing method, device, electronic equipment and readable storage medium storing program for executing |
-
2019
- 2019-04-30 CN CN201910359830.6A patent/CN110061755B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2813563Y (en) * | 2005-04-20 | 2006-09-06 | 苗小平 | Tower crane visual monitoring device |
| CN104063039A (en) * | 2013-03-18 | 2014-09-24 | 朱慧灵 | Human-computer interaction method of wearable computer intelligent terminal |
| CN106339094A (en) * | 2016-09-05 | 2017-01-18 | 山东万腾电子科技有限公司 | Interactive remote expert cooperation maintenance system and method based on augmented reality technology |
| CN206654664U (en) * | 2017-03-20 | 2017-11-21 | 河南师范大学 | A kind of control tower crane based on virtual reality technology |
| CN107203272A (en) * | 2017-06-23 | 2017-09-26 | 山东万腾电子科技有限公司 | Wearable augmented reality task instruction system and method based on myoelectricity cognition technology |
| CN107886172A (en) * | 2017-10-19 | 2018-04-06 | 中车青岛四方机车车辆股份有限公司 | System is supported in a kind of remote visualization maintenance |
| CN108509025A (en) * | 2018-01-26 | 2018-09-07 | 吉林大学 | A kind of crane intelligent Lift-on/Lift-off System based on limb action identification |
| CN108879440A (en) * | 2018-06-20 | 2018-11-23 | 国网山东省电力公司济宁供电公司 | Intelligent examination and repair system and method based on wearable augmented reality display terminal and cloud platform |
| CN109446876A (en) * | 2018-08-31 | 2019-03-08 | 百度在线网络技术（北京）有限公司 | Sign language information processing method, device, electronic equipment and readable storage medium storing program for executing |
| CN109405804A (en) * | 2018-11-05 | 2019-03-01 | 徐州重型机械有限公司 | Operation householder method and system |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112883754A (en) * | 2019-11-29 | 2021-06-01 | 华晨宝马汽车有限公司 | Auxiliary operation system of forklift |
| CN112883754B (en) * | 2019-11-29 | 2024-04-12 | 华晨宝马汽车有限公司 | Auxiliary operation system of forklift |
| WO2022199264A1 (en) * | 2021-03-22 | 2022-09-29 | International Business Machines Corporation | Multi-user interactive ad shopping using wearable device gestures |
| US11769134B2 (en) | 2021-03-22 | 2023-09-26 | International Business Machines Corporation | Multi-user interactive ad shopping using wearable device gestures |
| CN114415837A (en) * | 2022-01-25 | 2022-04-29 | 中国农业银行股份有限公司 | Operation auxiliary system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110061755B (en) | 2021-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9892559B2 (en) | Portable terminal device, and portable control device | |
| KR102248474B1 (en) | Voice command providing method and apparatus | |
| CN110061755A (en) | Operation householder method and system, wearable device and engineering truck | |
| KR102576654B1 (en) | Electronic apparatus and controlling method thereof | |
| US9990773B2 (en) | Terminal, information processing apparatus, display control method, and storage medium | |
| CN107801413A (en) | The terminal and its processing method being controlled to electronic equipment | |
| CN104182051B (en) | Head-wearing type intelligent equipment and the interactive system with the head-wearing type intelligent equipment | |
| KR101646356B1 (en) | Apparatus and Method for Controlling of Vehicle Using Wearable Device | |
| KR20150025293A (en) | Method for controlling a custom screen and an electronic device | |
| US20160124506A1 (en) | Electronic device and method for controlling external object | |
| EP2767914B1 (en) | Electronic device and object recognition method in electronic device | |
| US11842446B2 (en) | VR scene and interaction method thereof, and terminal device | |
| KR102559407B1 (en) | Computer readable recording meditum and electronic apparatus for displaying image | |
| CN111125442A (en) | Data labeling method and device | |
| CN114882437A (en) | Recognition model training method and device, electronic equipment and storage medium | |
| JP2016122392A (en) | Information processing apparatus, information processing system, control method and program of the same | |
| CN112116690A (en) | Video special effect generation method and device and terminal | |
| KR20150092962A (en) | Method for processing data and an electronic device thereof | |
| CN106557672A (en) | The solution lock control method of head mounted display and device | |
| KR20150025456A (en) | An electronic device and operating metod thereof | |
| CN109656364A (en) | It is a kind of for the method and apparatus of augmented reality content to be presented on a user device | |
| KR102646344B1 (en) | Electronic device for image synthetic and operating thereof | |
| KR102236203B1 (en) | Method for providing service and an electronic device thereof | |
| US20160078795A1 (en) | Method of controlling display of electronic device and electronic device thereof | |
| CN111914180A (en) | User characteristic determination method, device, equipment and medium based on graph structure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |