CN107491174A - Method, apparatus, system and electronic equipment for remote assistance - Google Patents
Method, apparatus, system and electronic equipment for remote assistance Download PDFInfo
- Publication number
- CN107491174A CN107491174A CN201710770766.1A CN201710770766A CN107491174A CN 107491174 A CN107491174 A CN 107491174A CN 201710770766 A CN201710770766 A CN 201710770766A CN 107491174 A CN107491174 A CN 107491174A
- Authority
- CN
- China
- Prior art keywords
- label information
- terminal
- information
- image
- remote assistance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
Abstract
A kind of method, apparatus, system and electronic equipment for remote assistance of disclosure.It is related to computer information processing field, this method includes：Present image is obtained by camera device；Label information is obtained, the label information includes mark image information and marker location information；Image procossing is carried out to the present image according to the label information, obtains identification data；And according to the identification data, correspondence position of the label information in display screen is shown.Method, apparatus, system and electronic equipment disclosed in the present application for remote assistance, it is possible to increase the degree of accuracy of remote assistance and operating efficiency.
Description
Technical field
The present invention relates to computer information processing, in particular to a kind of method, apparatus for remote assistance, it is
System and electronic equipment.
Background technology
China has used a large amount of plant equipment, and generally hold to high intensity as a manufacture big country in all trades and professions
Reforwarding turns, inevitably various failures, but specialized maintenance technical staff lacks, and field maintenance and inevitable band
Carry out greatly increasing on time and traveling expense, depend merely on voice and live fixing camera communication efficiency is very low, although
Also occurred shooting photo and transmission of video to backstage with mobile phone, or the mode using head mounted image-sensing head transmission video, but
Due to troublesome in poeration, it is difficult to discharge both hands, shooting orientation is difficult to calibrate, it is less in linking up real-time the problems such as, never have for many years
Good means solve the assistance communication problem of long-distance service.In fields such as tele-medicine, long-distance educations, there is also similar
Instruct communication problem urgently to be resolved hurrily.
Solve the problems, such as that remote sound has had history for many years as multi-person interactive using video conferencing system, but be limited to network
Condition and hardware limitation, seldom apply always in maintenance of equipment remote guide field.With Intelligent hardware technology and wireless network
Technology (such as：3G/4G communication networks) development, have been provided with corresponding technical conditions at present.The coming years will attract this
The explosive growth of application.
Prior art is generally by the use of AR glasses as collection in worksite equipment, with reference to traditional net meeting system
Realize that real-time interactive long-distance service instructs.In addition to speech communication, scene and the video pictures that can distally see other side, or
The picture at scene is all seen in person scene and distal end, but live video is to pass to the mode that the webserver passes scene back again.Can also
The person's handwriting of distal end or shared digital material are transferred back on the display unit at live end by blank.
But there is problems with the technology during utilization：
1. scene and distal end are it is seen that the real-time pictures of other side, and actual field and the camera that need not see distal end
Corresponding picture, this just wastes the network traffics of video all the way, adds the image analysis performance of the wearable hardware in scene
Loss, Field Force is also caused not know about whether the camera of oneself targeted by the object for wishing to shoot.
2. distal end needs to instruct scene, existing technology can only be indicated by voice, occasionally had and cut by screen
The mode taken, distal end intercepts the instantaneous picture at scene, then passes through shared or blank side after drawing on instantaneous tableaux
Formula passes scene back, and asynchronous guidance scene is operated, complex for operation step without real-time.It is difficult to video elementary at the scene
The mode being superimposed in upper or direct environment at the scene with augmented reality solves the problems, such as real-time communication.
Checked 3. can not directly open the digital material of remote transmission in end equipment at the scene.Such as：Picture, video,
3D models etc..Distal end can not control the content that scene is shown completely, cause Field Force's needs in switching display content frequent
Wearable device is operated, and wearable device is generally not easy to gloves or on hand had operated during greasy dirt.Even if can
With Voice command, but because field maintenance environment is generally more noisy, difficulty occurs in Voice command.
Therefore, it is necessary to a kind of new method, apparatus, system and electronic equipment for remote assistance.
Above- mentioned information is only used for strengthening the understanding of the background to the present invention, therefore it disclosed in the background section
It can include not forming the information to prior art known to persons of ordinary skill in the art.
The content of the invention
In view of this, the present invention provides a kind of method, apparatus, system and electronic equipment for remote assistance, Neng Gouti
The degree of accuracy and operating efficiency that high-altitude is assisted.
Other characteristics and advantage of the present invention will be apparent from by following detailed description, or partially by the present invention
Practice and acquistion.
According to an aspect of the invention, it is proposed that a kind of method for remote assistance, this method includes：Pass through camera device
Obtain present image；Label information is obtained, label information includes mark image information and marker location information；According to label information
Image procossing is carried out to present image, obtains identification data；And according to identification data, by label information display screen pair
Position is answered to be shown.
In a kind of exemplary embodiment of the disclosure, display screen includes：The display screen of wearable display device.
In a kind of exemplary embodiment of the disclosure, image procossing is carried out to present image according to label information, obtained
Identification data, including：According to mark image information, corresponding region in present image is determined；Image procossing is carried out to corresponding region,
It is determined that corresponding target；And identification data is generated by target.
In a kind of exemplary embodiment of the disclosure, according to identification data, by label information display screen correspondence
Position is shown, including：By positional information and identification data, display location information is obtained by geometrical principle algorithm；With
And by display location information, correspondence position of the label information in display screen is shown.
In a kind of exemplary embodiment of the disclosure, according to identification data, by label information display screen correspondence
Position is shown, including：It is transparent display or semi-transparent display apparatus to set display screen；And according to identification number
According to correspondence position of the label information in display screen is shown.
According to an aspect of the invention, it is proposed that a kind of method for remote assistance, this method includes：Obtain picture number
According to；Operation is marked on the image data, generates label information, label information includes mark image information and marker bit confidence
Breath；Label information is sent to predetermined end.
In a kind of exemplary embodiment of the disclosure, operation is marked on the image data, generates label information, bag
Include：Dynamically labeled operation is carried out on the image data, generates label information；And static tay behaviour is carried out on the image data
Make, generate label information.
In a kind of exemplary embodiment of the disclosure, marking operation passes through wearable display apparatus processes.
In a kind of exemplary embodiment of the disclosure, view data is obtained, including：Pass through the mobile display device of distal end
Obtain view data；And view data is obtained by database.
According to an aspect of the invention, it is proposed that a kind of method for remote assistance, this method includes：First terminal and
Two terminals establish long-range connection；The present image obtained by first terminal is sent to second terminal；Enter at second terminal
Line flag operates, and generates label information, and label information includes mark image information and marker location information；Second terminal will mark
Information is sent to first terminal；And correspondence position of the label information in the display screen of first terminal is shown.
In a kind of exemplary embodiment of the disclosure, first terminal, including：Wearable display device.
According to an aspect of the invention, it is proposed that a kind of system for remote assistance, the system includes：First terminal and
Two terminals establish long-range connection；The present image obtained by first terminal is sent to second terminal；Enter at second terminal
Line flag operates, and generates label information, and label information includes mark image information and marker location information；Second terminal will mark
Information is sent to first terminal；And correspondence position of the label information in the display screen of first terminal is shown.
According to an aspect of the invention, it is proposed that a kind of device for remote assistance, the device includes：Photographing module, use
In obtaining present image by camera device；Receiving module, for obtaining label information, label information includes mark image information
With marker location information；Processing module, for carrying out image procossing to present image according to label information, obtain identification data；
And display module, for according to identification data, correspondence position of the label information in display screen to be shown.
According to an aspect of the invention, it is proposed that a kind of device for remote assistance, the device includes：Receiving module, use
In acquisition view data；Mark module, for operation to be marked on the image data, generate label information, label information bag
Include mark image information and marker location information；Sending module, for label information to be sent to predetermined end.
According to an aspect of the invention, it is proposed that a kind of electronic equipment, the electronic equipment includes：One or more processors；
Storage device, for storing one or more programs；When one or more programs are executed by one or more processors so that one
Individual or multiple processors realize such as methodology above.
According to an aspect of the invention, it is proposed that a kind of computer-readable medium, is stored thereon with computer program, its feature
It is, method as mentioned in the above is realized when program is executed by processor.
According to the method, apparatus, system and electronic equipment for remote assistance of the present invention, it is possible to increase remote assistance
The degree of accuracy and operating efficiency.
It should be appreciated that the general description and following detailed description of the above are only exemplary, this can not be limited
Invention.
Brief description of the drawings
Its example embodiment is described in detail by referring to accompanying drawing, above and other target of the invention, feature and advantage will
Become more fully apparent.Drawings discussed below is only some embodiments of the present invention, for the ordinary skill of this area
For personnel, on the premise of not paying creative work, other accompanying drawings can also be obtained according to these accompanying drawings.
Fig. 1 is a kind of flow chart of method for remote assistance according to an exemplary embodiment.
Fig. 2 is a kind of display schematic diagram of method for remote assistance according to an exemplary embodiment.
Fig. 3 is a kind of flow chart of method for remote assistance according to another exemplary embodiment.
Fig. 4 is a kind of flow chart of method for remote assistance according to another exemplary embodiment.
Fig. 5 is a kind of block diagram of device for remote assistance according to an exemplary embodiment.
Fig. 6 is a kind of block diagram of device for remote assistance according to another exemplary embodiment.
Fig. 7 is a kind of block diagram of system for remote assistance according to an exemplary embodiment.
Fig. 8 is the block diagram of a kind of electronic equipment according to an exemplary embodiment.
Fig. 9 is a kind of computer-readable medium schematic diagram according to an exemplary embodiment.
Specific embodiment
Example embodiment is described more fully with referring now to accompanying drawing.However, example embodiment can be real in a variety of forms
Apply, and be not understood as limited to embodiment set forth herein；On the contrary, these embodiments are provided so that the present invention will be comprehensively and complete
It is whole, and the design of example embodiment is comprehensively communicated to those skilled in the art.Identical reference represents in figure
Same or similar part, thus repetition thereof will be omitted.
In addition, described feature, structure or characteristic can be incorporated in one or more implementations in any suitable manner
In example.In the following description, there is provided many details fully understand so as to provide to embodiments of the invention.However,
It will be appreciated by persons skilled in the art that technical scheme can be put into practice without one or more in specific detail,
Or other methods, constituent element, device, step etc. can be used.In other cases, side known in being not shown in detail or describe
Method, device, realization are operated to avoid fuzzy each aspect of the present invention.
Block diagram shown in accompanying drawing is only functional entity, not necessarily must be corresponding with physically separate entity.
I.e., it is possible to realize these functional entitys using software form, or realized in one or more hardware modules or integrated circuit
These functional entitys, or these functional entitys are realized in heterogeneous networks and/or processor device and/or microcontroller device.
Flow chart shown in accompanying drawing is merely illustrative, it is not necessary to including all contents and operation/step,
It is not required to perform by described order.For example, some operation/steps can also decompose, and some operation/steps can close
And or partly merging, therefore the order actually performed is possible to be changed according to actual conditions.
It should be understood that although herein various assemblies may be described using term first, second, third, etc., these groups
Part should not be limited by these terms.These terms are to distinguish a component and another component.Therefore, first group be discussed herein below
Part can be described as teaching of second component without departing from disclosure concept.As used herein, term " and/or " include it is associated
All combinations for listing any one and one or more in project.
It will be understood by those skilled in the art that accompanying drawing is the schematic diagram of example embodiment, module or flow in accompanying drawing
Necessary to not necessarily implementing the present invention, therefore it cannot be used for limiting the scope of the invention.
Disclosure example embodiment is described in detail below in conjunction with the accompanying drawings.
Fig. 1 is a kind of flow chart of method for remote assistance according to an exemplary embodiment.
As shown in figure 1, in S102, present image is obtained by camera device.Camera may be, for example, wearable MR
The camera carried on glasses, camera can also for example include but is not limited to desktop computer, notebook computer, super basis, flat board
The camera carried in computer, mobile phone, wearable AR glasses, wearable VR glasses and/or other wearable devices.
In S104, label information is obtained, label information includes mark image information and marker location information.Label information
It may be, for example, dynamically labeled information or static tay information, may be, for example, the figure of arbitrary graphic pattern, the word of input and/or insertion
Piece, animation and video record and/or electronic document, such as, but not limited to 3D models, word, excel, powerpoint, pdf are done
Service message subdocument etc..Label information can also be for example including marker location information, and marker location information can be used for one spy of record
The position of earnest body, marker location information can also for example be used for the position of one object of mark in favor of follow-up processing.
In S106, image procossing is carried out to present image according to label information, obtains identification data.Including：According to mark
Remember image information, determine corresponding region in present image；Image procossing is carried out to corresponding region, it is determined that corresponding target；It is and logical
Cross target generation identification data.The image currently obtained can be carried out at image for example by the image information in label information
Reason, the region same or like with image in label information is obtained, this region can for example be defined as first area, can also be for example right
First area carries out further image recognition, obtains the target object in first area.Mesh that can for example by identifying
Mark object generation identification data.Wherein, image recognition algorithm can be for example including Sift local feature descriptions subalgorithm, Sift features
It is the local feature of image, to translating, rotating, scaling, brightness change, blocks and noise etc. has good consistency,
A certain degree of stability is also kept to visible change, affine transformation.Can for example, extraction label information in local feature,
Local feature in label information is compared with local feature in realtime graphic, and then mark is determined in realtime graphic
Remember the corresponding region in information.And then carry out follow-up processing.
In S108, according to identification data, correspondence position of the label information in display screen is shown.Display screen
Including：The display screen of wearable display device, it can for example include the display screen of the wearable display device of virtual reality,
And the display screen of the wearable display device of augmented reality.The present invention is not limited.
, can be for example, being shown currently at display screen end when user uses the wearable display device of virtual reality
Image information, in the content and label information of the internal processing program of the wearable display device of virtual reality according to label information
Positional information, show label information on the screen of the display screen of the wearable display device of virtual reality.User can be straight
The image content connected on display screen obtains label information.The use of set use also may be, for example, wearable in this scene
Formula AR glasses, wearable VR glasses and/or other wearable devices, the present invention are not limited.
, can also be for example, setting the display screen to be when user uses the wearable display device of virtual reality
Obvious showing device or semi-transparent display apparatus；And according to identification data, correspondence position of the label information in display screen is entered
Row displaying.Also interpupillary distance regulation can be carried out, for example, according to the instruction of user to adapt to different users.Fig. 2 is according to one
A kind of display schematic diagram of method for remote assistance shown in exemplary embodiment.As illustrated, can also be for example, passing through position
Confidence ceases and identification data, and display location information is obtained by geometrical principle algorithm；And by display location information, it will mark
Information is shown in the correspondence position of display screen.Outwards looked by transparent or semitransparent glasses with causing user
When, the label information showed on a display screen can just cover the target object of distal end.
According to the method for remote assistance of the present invention, by obtaining realtime graphic in user's end, and image is entered
Row image procossing, so as to the mode for being shown label information on the display screen of user, it is possible to increase remote assistance
The degree of accuracy and operating efficiency.
It will be clearly understood that the present disclosure describe how formation and using particular example, but the principle of the present invention is not limited to
Any details of these examples.On the contrary, the teaching based on present disclosure, these principles can be applied to many other
Embodiment.
Fig. 3 is a kind of flow chart of method for remote assistance according to another exemplary embodiment.
As described in Figure 3, in S302, view data is obtained.Can be for example, obtaining image by the mobile display device of distal end
Data, the real time data at scene for example can be obtained by the mobile device of distal end, the mobile device of distal end may include but be not limited to
Desktop computer, notebook computer, super basis, tablet personal computer, mobile phone, wearable AR glasses, wearable VR glasses and/or can
Wearable MR helmets etc..Can also for example, by database obtain view data, can for example, carry out remote engineering instruct when,
, can be for example, distal end for example can be taken the photograph between equipment operation by prior when the residing region of field engineering administration is known region
System operates, the drawing on incorporation engineering, establishes the 3-D view model of operation room, is stored in database, you can adjust by database
With view data obtain the operation of image.Can be for example, realizing 3-D view model by three-dimensional panorama virtual reality algorithm.
Three-dimensional panorama virtual reality is also referred to as that outdoor scene is virtual, is the real scene virtual reality technology based on panoramic picture.Panorama
(Panorama) be one or more groups of photomosaics the 360 degree of shootings of camera ring into a panoramic picture, pass through computer skill
Art realizes the real scene reduction display mode of comprehensive interactive viewing.
In S304, operation is marked on the image data, generates label information, label information includes mark image letter
Breath and marker location information.Dynamically labeled operation is carried out on the image data, generates label information；And enter on the image data
Row static tay operates, and generates label information.Marking operation can for example pass through wearable display apparatus processes.User utilizes
Mouse, stylus and/or finger etc. in the arbitrarily drawn pattern of transparent control layer, such as straight line, curve, closed curve, triangle,
Quadrangle, circle etc.；Or utilize input through keyboard word；Or inserted using mouse, stylus, finger and/or keyboard operation
Enter existing picture, animation, video record and/or electronic document, such as, but not limited to 3D models, word, excel,
Powerpoint, pdf office electronic document etc..Meanwhile when being operated at marking operation end, when in the transparent control layer
When inside detecting operational order, the corresponding coordinate information of this operation is obtained.The behaviour that the coordinate information is carried out including the use of person
Make, such as user the operation such as is clicked on using mouse, stylus and/or finger, pulled, moved, unclamped and drawing out arbitrarily
Pattern, such as the coordinate information corresponding to straight line, curve, closed curve, triangle, quadrangle, circle etc.；Or user
Utilize the coordinate information corresponding to input through keyboard word；Or using mouse, stylus, finger and/or keyboard operation insertion
Some pictures, animation, video record and/or electronic document, such as, but not limited to 3D models, word, excel, powerpoint,
Coordinate information corresponding to pdf office electronic documents etc..After above-mentioned coordinate information is obtained, label information is generated.
In S306, label information is sent to predetermined end.Can be for example, label information be sent to the display device of distal end
In.
According to the method for remote assistance of the present invention, by setting acquisition image at tech support, on image
Label information is set, and is sent to the mode of predetermined end, it is possible to increase the degree of accuracy of remote assistance and operating efficiency.
Fig. 4 is a kind of flow chart of method for remote assistance according to another exemplary embodiment.
As shown in figure 4, in S402, first terminal and second terminal establish long-range connection.By first terminal and second eventually
Long-range connection is established at end, and the first terminal and second terminal can include 3G, 4G cellular network communication by mobile communications network
Network, WiFi wireless networks and/or Blue-tooth communication method are directly connected to, and can also be kept connecting by remote server transfer.
Wherein, the first terminal can be computer, mobile device and/or wearable device, including but not limited to desktop computer,
Notebook computer, super basis, tablet personal computer, mobile phone, wearable AR glasses, wearable VR glasses and/or wearable VR
Helmet etc., second terminal can be computer, mobile device and/or wearable device, including but not limited to desktop computer,
Notebook computer, super basis, tablet personal computer, mobile phone, wearable AR glasses, wearable VR glasses and/or wearable MR
Helmet etc..It can also be created for example, after first terminal and second terminal establish long-range connection on first terminal screen saturating
It is obvious to show forms, and the Transparence Display forms are placed in screen top layer, the Transparence Display forms are used to show and second terminal
Information corresponding to the control operation of upper progress, the word including but not limited to inputted, description or the pattern of insertion, video
And/or electronic document, or the operation such as click, dragging, mobile and release
In S404, the present image obtained by first terminal is sent to second terminal.By the first terminal
Shown image on screen, including static picture and/or dynamic video image synchronous driving to the second terminal control window
The image displa layer of body.
In S406, operation is marked at second terminal, generates label information, label information includes mark image letter
Breath and marker location information.Control forms can be created for example on second terminal screen, the control forms include image displa layer
With transparent control layer.Wherein, the image displa layer of forms is controlled to come from first eventually for display on the second terminal screen
The image at end, including static picture and/or dynamic video image, the transparent control layer are used to be controlled in second terminal
Operation.When the second terminal detects operational order in the transparent control layer, the corresponding coordinate letter of this operation is obtained
Breath, and send it to first terminal.According to the needs of remote assistance, the user of the second terminal is in the saturating of second terminal
Operated in bright key-course, send operational order.Specific operation is for example, the user of second terminal utilizes mouse, touch-control
Pen and/or finger etc. are in the arbitrarily drawn pattern of transparent control layer, such as straight line, curve, closed curve, triangle, quadrangle, circle
Shape etc.；Or utilize input through keyboard word；Or insert existing figure using mouse, stylus, finger and/or keyboard operation
Piece, animation, video record and/or electronic document, such as, but not limited to 3D models, word, excel, powerpoint, pdf are done
Service message subdocument etc.., can be for example by obtaining when detecting operational order in the transparent control layer in second terminal
Take the coordinate information generation label information that this operation is corresponding.The operation that the coordinate information is carried out including the use of person, such as make
User the operation such as is clicked on using mouse, stylus and/or finger, pulled, being moved, being unclamped and drawing out arbitrary graphic pattern, such as
Coordinate information corresponding to straight line, curve, closed curve, triangle, quadrangle, circle etc.；Or user utilizes keyboard
Input the coordinate information corresponding to word；Or utilization mouse, stylus, finger and/or the existing picture of keyboard operation insertion,
Animation, video record and/or electronic document, such as, but not limited to 3D models, word, excel, powerpoint, pdf office electricity
Label information corresponding to subdocument etc..
In S408, second terminal sends label information to first terminal.
In S410, correspondence position of the label information in the display screen of first terminal is shown.First terminal connects
Receive the label information sent by second terminal, according to these label informations, the relevant position on first terminal Transparence Display forms
The corresponding operation of display, for example, second terminal user draw out arbitrary graphic pattern, input word and/or insertion picture,
Animation and video record and/or electronic document, such as, but not limited to 3D models, word, excel, powerpoint, pdf office
Electronic document etc..
According to the method for remote assistance of the present invention, by obtaining image at technical support engineer, on image
Label information is set, and is sent to predetermined end, field engineer obtains realtime graphic, and carries out image procossing to image, with
Just mode label information shown on the display screen used, it is possible to increase the degree of accuracy of remote assistance is imitated with work
Rate.
It will be appreciated by those skilled in the art that realize that all or part of step of above-described embodiment is implemented as being performed by CPU
Computer program.When the computer program is performed by CPU, the above-mentioned work(that the above method provided by the invention is limited is performed
Energy.Described program can be stored in a kind of computer-readable recording medium, and the storage medium can be read-only storage, magnetic
Disk or CD etc..
Further, it should be noted that above-mentioned accompanying drawing is only the place included by method according to an exemplary embodiment of the present invention
Reason schematically illustrates, rather than limitation purpose.It can be readily appreciated that above-mentioned processing shown in the drawings is not intended that or limited at these
The time sequencing of reason.In addition, being also easy to understand, these processing for example can be performed either synchronously or asynchronously in multiple modules.
Following is apparatus of the present invention embodiment, can be used for performing the inventive method embodiment.It is real for apparatus of the present invention
The details not disclosed in example is applied, refer to the inventive method embodiment.
Fig. 5 is a kind of block diagram of device for remote assistance according to an exemplary embodiment.
Photographing module 502 is used to obtain present image by camera device.
Receiving module 504 is used to obtain label information, and label information includes mark image information and marker location information.
Processing module 506 is used to carry out image procossing to present image according to label information, obtains identification data.
Display module 508 is used for according to identification data, and correspondence position of the label information in display screen is shown.
Fig. 6 is a kind of block diagram of device for remote assistance according to another exemplary embodiment.
Receiving module 602 is used to obtain view data.
Mark module 604 is used to operation be marked on the image data, generates label information, and label information includes mark
Image information and marker location information.
Sending module 606 is used to send label information to predetermined end.
Fig. 7 is a kind of block diagram of system for remote assistance according to an exemplary embodiment.
First terminal 702 and second terminal 704 establish long-range connection；The present image that will be obtained by first terminal 702
Send to second terminal 704；Operation is marked at second terminal 704, generates label information, label information, which includes mark, schemes
As information and marker location information；Second terminal 704 sends label information to first terminal 702；And label information is existed
The correspondence position of the display screen of first terminal 702 is shown.
Fig. 8 is the block diagram of a kind of electronic equipment according to an exemplary embodiment.
The electronic equipment 200 according to the embodiment of the invention is described referring to Fig. 8.The electronics that Fig. 8 is shown
Equipment 200 is only an example, should not bring any restrictions to the function and use range of the embodiment of the present invention.
As shown in figure 8, electronic equipment 200 is showed in the form of universal computing device.The component of electronic equipment 200 can wrap
Include but be not limited to：At least one processing unit 210, at least one memory cell 220, (including the storage of connection different system component
Unit 220 and processing unit 210) bus 230, display unit 240 etc..
Wherein, the memory cell is had program stored therein code, and described program code can be held by the processing unit 210
OK so that the processing unit 210 perform described in the above-mentioned electronic prescription circulation processing method part of this specification according to this
The step of inventing various illustrative embodiments.For example, the processing unit 210 can perform the step as shown in Fig. 1 or Fig. 3
Suddenly.
The memory cell 220 can include the computer-readable recording medium of volatile memory cell form, such as random access memory
Unit (RAM) 2201 and/or cache memory unit 2202, it can further include read-only memory unit (ROM) 2203.
The memory cell 220 can also include program/practical work with one group of (at least one) program module 2205
Tool 2204, such program module 2205 includes but is not limited to：Operating system, one or more application program, other programs
Module and routine data, the realization of network environment may be included in each or certain combination in these examples.
Bus 230 can be to represent the one or more in a few class bus structures, including memory cell bus or storage
Cell controller, peripheral bus, graphics acceleration port, processing unit use any bus structures in a variety of bus structures
Local bus.
Electronic equipment 200 can also be with one or more external equipments 300 (such as keyboard, sensing equipment, bluetooth equipment
Deng) communication, the equipment communication interacted with the electronic equipment 200 can be also enabled a user to one or more, and/or with causing
Any equipment that the electronic equipment 200 can be communicated with one or more of the other computing device (such as router, modulation /demodulation
Device etc.) communication.This communication can be carried out by input/output (I/O) interface 250.Also, electronic equipment 200 can be with
By network adapter 260 and one or more network (such as LAN (LAN), wide area network (WAN) and/or public network,
Such as internet) communication.Network adapter 260 can be communicated by bus 230 with other modules of electronic equipment 200.Should
Understand, although not shown in the drawings, can combine electronic equipment 200 uses other hardware and/or software module, including it is but unlimited
In：Microcode, device driver, redundant processing unit, external disk drive array, RAID system, tape drive and number
According to backup storage system etc..
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented
Mode can be realized by software, can also be realized by way of software combines necessary hardware.Therefore, according to the disclosure
The technical scheme of embodiment can be embodied in the form of software product, the software product can be stored in one it is non-volatile
Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are to cause a calculating
Equipment (can be personal computer, server or network equipment etc.) performs the above-mentioned electronics according to disclosure embodiment
Prescription circulation processing method.
Fig. 9 is a kind of computer-readable medium schematic diagram according to an exemplary embodiment.
With reference to shown in figure 9, the program product for being used to realize the above method according to the embodiment of the present invention is described
400, it can use portable compact disc read only memory (CD-ROM) and including program code, and can in terminal device,
Such as run on PC.However, the program product not limited to this of the present invention, in this document, readable storage medium storing program for executing can be with
Be it is any include or the tangible medium of storage program, the program can be commanded execution system, device either device use or
It is in connection.
Described program product can use any combination of one or more computer-readable recording mediums.Computer-readable recording medium can be readable letter
Number medium or readable storage medium storing program for executing.Readable storage medium storing program for executing for example can be but be not limited to electricity, magnetic, optical, electromagnetic, infrared ray or
System, device or the device of semiconductor, or any combination above.The more specifically example of readable storage medium storing program for executing is (non exhaustive
List) include：It is electrical connection, portable disc, hard disk, random access memory (RAM) with one or more wires, read-only
Memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read only memory
(CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
The computer-readable recording medium can include believing in a base band or as the data that a carrier wave part is propagated
Number, wherein carrying readable program code.The data-signal of this propagation can take various forms, including but not limited to electromagnetism
Signal, optical signal or above-mentioned any appropriate combination.Readable storage medium storing program for executing can also be any beyond readable storage medium storing program for executing
Computer-readable recording medium, the computer-readable recording medium can send, propagate either transmit for being used by instruction execution system, device or device or
Person's program in connection.The program code included on readable storage medium storing program for executing can be transmitted with any appropriate medium, bag
Include but be not limited to wireless, wired, optical cable, RF etc., or above-mentioned any appropriate combination.
Can being combined to write the program operated for performing the present invention with one or more programming languages
Code, described program design language include object oriented program language-Java, C++ etc., include routine
Procedural programming language-such as " C " language or similar programming language.Program code can be fully in user
Perform on computing device, partly perform on a user device, the software kit independent as one performs, is partly calculated in user
Its upper side point is performed or performed completely in remote computing device or server on a remote computing.It is remote being related to
In the situation of journey computing device, remote computing device can pass through the network of any kind, including LAN (LAN) or wide area network
(WAN) user calculating equipment, is connected to, or, it may be connected to external computing device (such as utilize ISP
To pass through Internet connection).
Above computer computer-readable recording medium carries one or more program, when said one or multiple programs are by one
When the equipment performs so that the computer-readable medium implements function such as：Present image is obtained by camera device；Obtain mark
Remember information, label information includes mark image information and marker location information；Image is carried out to present image according to label information
Processing, obtain identification data；And according to identification data, correspondence position of the label information in display screen is shown.Or
The computer-readable medium implements function such as：Obtain view data；Operation, generation mark letter are marked on the image data
Breath, label information include mark image information and marker location information；Label information is sent to predetermined end.
It will be appreciated by those skilled in the art that above-mentioned each module can be distributed in device according to the description of embodiment, also may be used
To carry out respective change uniquely different from one or more devices of the present embodiment.The module of above-described embodiment can be merged into
One module, can also be further split into multiple submodule.
The description of embodiment more than, those skilled in the art is it can be readily appreciated that example embodiment described herein
It can be realized, can also be realized by way of software combines necessary hardware by software.Therefore, implemented according to the present invention
The technical scheme of example can be embodied in the form of software product, and the software product can be stored in a non-volatile memories
In medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) or on network, including some instructions are make it that a computing device (can
To be personal computer, server, mobile terminal or network equipment etc.) perform method according to embodiments of the present invention.
Detailed description more than, those skilled in the art is it can be readily appreciated that according to embodiments of the present invention being used for is remote
Method, apparatus, system and the electronic equipment that journey is assisted have one or more of the following advantages.
According to some embodiments, the method for remote assistance of the invention, realtime graphic is obtained by field engineer,
And image procossing is carried out to image, so as to the mode for being shown label information on the display screen used, it is possible to increase remote
The degree of accuracy and operating efficiency that journey is assisted.
According to other embodiments, the method for remote assistance of the invention, acquisition image is set by distal end engineer,
Label information is set on image, and is sent to the mode of predetermined end, it is possible to increase the degree of accuracy of remote assistance and operating efficiency.
According to still other embodiments, the method for remote assistance of the invention, acquisition image is set by distal end engineer,
Label information is set on image, and is sent to predetermined end, field engineer obtains realtime graphic, and carries out image to image
Processing, so as to the mode for being shown label information on the display screen used, it is possible to increase the degree of accuracy of remote assistance with
Operating efficiency.
The exemplary embodiment of the present invention is particularly shown and described above.It should be appreciated that the invention is not restricted to
Detailed construction, set-up mode or implementation method described herein；On the contrary, it is intended to cover included in appended claims
Various modifications and equivalence setting in spirit and scope.
In addition, structure, ratio, size shown by this specification Figure of description etc., only coordinating specification institute
Disclosure, for skilled in the art realises that with reading, being not limited to the enforceable qualifications of the disclosure, therefore
Do not have technical essential meaning, the modification of any structure, the change of proportionate relationship or the adjustment of size, do not influenceing the disclosure
Under the technique effect that can be generated and achieved purpose, it all should still fall and obtain and can cover in the technology contents disclosed in the disclosure
In the range of.Meanwhile in this specification it is cited such as " on ", " first ", the term of " second " and " one ", be also only and be easy to
Narration understands, and is not used to limit the enforceable scope of the disclosure, and its relativeness is altered or modified, without substantive change
Under technology contents, when being also considered as the enforceable category of the present invention.
Claims (10)
- A kind of 1. method for remote assistance, it is characterised in that including：Present image is obtained by camera device；Label information is obtained, the label information includes mark image information and marker location information；Image procossing is carried out to the present image according to the label information, obtains identification data；AndAccording to the identification data, correspondence position of the label information in display screen is shown.
- 2. the method as described in claim 1, it is characterised in that the display screen includes：The display screen of wearable display device.
- 3. the method as described in claim 1, it is characterised in that described to be carried out according to the label information to the present image Image procossing, identification data is obtained, including：According to the mark image information, corresponding region in the present image is determined；Image procossing is carried out to the corresponding region, it is determined that corresponding target；AndIdentification data is generated by the target.
- 4. the method as described in claim 1, it is characterised in that it is described according to the identification data, the label information is existed The correspondence position of display screen is shown, including：By the positional information and the identification data, display location information is obtained by geometrical principle algorithm；AndBy the display location information, correspondence position of the label information in display screen is shown.
- A kind of 5. method for remote assistance, it is characterised in that including：Obtain view data；Be marked operation in described image data, generate label information, the label information include mark image information with Marker location information；The label information is sent to predetermined end.
- A kind of 6. method for remote assistance, it is characterised in that including：First terminal and second terminal establish long-range connection；The present image obtained by the first terminal is sent to second terminal；Be marked operation at the second terminal, generate label information, the label information include mark image information with Marker location information；The second terminal sends the label information to the first terminal；AndCorrespondence position of the label information in the display screen of the first terminal is shown.
- 7. a kind of system for remote assistance, including first terminal and second terminal, it is characterised in that including：First terminal and second terminal establish long-range connection；The present image obtained by the first terminal is sent to second terminal；Be marked operation at the second terminal, generate label information, the label information include mark image information with Marker location information；The second terminal sends the label information to the first terminal；AndCorrespondence position of the label information in the display screen of the first terminal is shown.
- 8. a kind of device for remote assistance, it is characterised in that the remote assistance method includes：Photographing module, for obtaining present image by camera device；Receiving module, for obtaining label information, the label information includes mark image information and marker location information；Processing module, for carrying out image procossing to the present image according to the label information, obtain identification data；AndDisplay module, for according to the identification data, correspondence position of the label information in display screen to be shown.
- 9. a kind of device for remote assistance, it is characterised in that the remote assistance method includes：Receiving module, for obtaining view data；Mark module, for operation to be marked in described image data, label information is generated, the label information includes mark Remember image information and marker location information；Sending module, for the label information to be sent to predetermined end.
- 10. a kind of electronic equipment, it is characterised in that including：One or more processors；Storage device, for storing one or more programs, when one or more of programs are by one or more of processing When device performs so that one or more of processors realize the remote assistance side as any one of claim 1 to 5 Method, or realize the remote assistance method as any one of claim 6 to 9.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2016107977295 | 2016-08-31 | ||
| CN201610797729 | 2016-08-31 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107491174A true CN107491174A (en) | 2017-12-19 |
| CN107491174B CN107491174B (en) | 2021-12-24 |
Family
ID=60645953
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710770766.1A Active CN107491174B (en) | 2016-08-31 | 2017-08-31 | Method, device and system for remote assistance and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107491174B (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107566793A (en) * | 2017-08-31 | 2018-01-09 | 中科云创（北京）科技有限公司 | Method, apparatus, system and electronic equipment for remote assistance |
| CN108322475A (en) * | 2018-02-27 | 2018-07-24 | 成都普望智能科技有限公司 | A kind of equipment remote collaboration method for maintaining and system based on MR live streamings |
| CN108776567A (en) * | 2018-06-08 | 2018-11-09 | 上海数圃信息科技有限公司 | Visual indicia method, system, electric terminal and computer readable storage medium |
| CN109462637A (en) * | 2018-10-17 | 2019-03-12 | 广东电网有限责任公司 | MR expert's remote collaboration method and system based on power equipment 3D model library |
| CN109635957A (en) * | 2018-11-13 | 2019-04-16 | 广州裕申电子科技有限公司 | A kind of equipment maintenance aid method and system based on AR technology |
| WO2019085623A1 (en) * | 2017-10-31 | 2019-05-09 | 上海掌门科技有限公司 | Interaction method and device |
| CN109885951A (en) * | 2019-02-28 | 2019-06-14 | 中科云创(厦门)科技有限公司 | Equipment fault diagnosis method and device |
| CN109993812A (en) * | 2019-03-26 | 2019-07-09 | 南京罗拉穿云物联网科技有限公司 | Floor drawing generating method and device based on Internet of Things |
| CN110138831A (en) * | 2019-03-29 | 2019-08-16 | 亮风台（上海）信息科技有限公司 | A kind of method and apparatus carrying out remote assistance |
| CN110474979A (en) * | 2019-08-13 | 2019-11-19 | 北京软通智城科技有限公司 | A kind of remote assisting system, method, platform and storage medium |
| CN111243763A (en) * | 2020-02-03 | 2020-06-05 | 深圳安泰创新科技股份有限公司 | Mark display method, device, equipment and computer storage medium |
| CN111324269A (en) * | 2020-02-21 | 2020-06-23 | 冒守华 | Marking method, system, storage medium and terminal equipment based on visualization |
| CN112181146A (en) * | 2020-09-29 | 2021-01-05 | 重庆市华驰交通科技有限公司 | Wearable device display picture control method and system for electromechanical device maintenance |
| CN112383734A (en) * | 2020-10-29 | 2021-02-19 | 岭东核电有限公司 | Video processing method, video processing device, computer equipment and storage medium |
| CN113986057A (en) * | 2021-10-22 | 2022-01-28 | 北京三快在线科技有限公司 | Image processing method and device, readable storage medium and unmanned equipment |
| WO2022061858A1 (en) * | 2020-09-28 | 2022-03-31 | Orange | Method for assisting a first user provided with a terminal at a scene |
| CN114356089A (en) * | 2021-12-30 | 2022-04-15 | Oppo广东移动通信有限公司 | Augmented reality glasses control method and device, storage medium and electronic equipment |
| CN114465910A (en) * | 2022-02-10 | 2022-05-10 | 北京为准智能科技有限公司 | Machining equipment calibration method based on augmented reality technology |
| CN117311892A (en) * | 2023-11-30 | 2023-12-29 | 深圳富士伟业科技有限公司 | Remote assistance method, system, electronic device and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102662618A (en) * | 2012-03-29 | 2012-09-12 | 北京奇虎科技有限公司 | Image processing method and device in remote assistance process |
| US20130113738A1 (en) * | 2011-11-08 | 2013-05-09 | Electronics And Telecommunications Research Institute | Method and apparatus for controlling content on remote screen |
| CN103945109A (en) * | 2013-01-17 | 2014-07-23 | 佳能株式会社 | Image pickup apparatus, remote control apparatus, and methods of controlling image pickup apparatus and remote control apparatus |
| CN104800068A (en) * | 2015-05-22 | 2015-07-29 | 京东方科技集团股份有限公司 | Device and system used for remotely determining acupuncture point |
-
2017
- 2017-08-31 CN CN201710770766.1A patent/CN107491174B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130113738A1 (en) * | 2011-11-08 | 2013-05-09 | Electronics And Telecommunications Research Institute | Method and apparatus for controlling content on remote screen |
| CN102662618A (en) * | 2012-03-29 | 2012-09-12 | 北京奇虎科技有限公司 | Image processing method and device in remote assistance process |
| CN103945109A (en) * | 2013-01-17 | 2014-07-23 | 佳能株式会社 | Image pickup apparatus, remote control apparatus, and methods of controlling image pickup apparatus and remote control apparatus |
| CN104800068A (en) * | 2015-05-22 | 2015-07-29 | 京东方科技集团股份有限公司 | Device and system used for remotely determining acupuncture point |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107566793A (en) * | 2017-08-31 | 2018-01-09 | 中科云创（北京）科技有限公司 | Method, apparatus, system and electronic equipment for remote assistance |
| WO2019085623A1 (en) * | 2017-10-31 | 2019-05-09 | 上海掌门科技有限公司 | Interaction method and device |
| CN108322475A (en) * | 2018-02-27 | 2018-07-24 | 成都普望智能科技有限公司 | A kind of equipment remote collaboration method for maintaining and system based on MR live streamings |
| CN108776567A (en) * | 2018-06-08 | 2018-11-09 | 上海数圃信息科技有限公司 | Visual indicia method, system, electric terminal and computer readable storage medium |
| CN109462637A (en) * | 2018-10-17 | 2019-03-12 | 广东电网有限责任公司 | MR expert's remote collaboration method and system based on power equipment 3D model library |
| CN109635957A (en) * | 2018-11-13 | 2019-04-16 | 广州裕申电子科技有限公司 | A kind of equipment maintenance aid method and system based on AR technology |
| CN109885951A (en) * | 2019-02-28 | 2019-06-14 | 中科云创(厦门)科技有限公司 | Equipment fault diagnosis method and device |
| CN109993812A (en) * | 2019-03-26 | 2019-07-09 | 南京罗拉穿云物联网科技有限公司 | Floor drawing generating method and device based on Internet of Things |
| CN110138831A (en) * | 2019-03-29 | 2019-08-16 | 亮风台（上海）信息科技有限公司 | A kind of method and apparatus carrying out remote assistance |
| CN110474979A (en) * | 2019-08-13 | 2019-11-19 | 北京软通智城科技有限公司 | A kind of remote assisting system, method, platform and storage medium |
| CN111243763A (en) * | 2020-02-03 | 2020-06-05 | 深圳安泰创新科技股份有限公司 | Mark display method, device, equipment and computer storage medium |
| CN111243763B (en) * | 2020-02-03 | 2023-09-22 | 深圳安泰创新科技股份有限公司 | Mark display method, device, equipment and computer storage medium |
| CN111324269A (en) * | 2020-02-21 | 2020-06-23 | 冒守华 | Marking method, system, storage medium and terminal equipment based on visualization |
| CN111324269B (en) * | 2020-02-21 | 2023-06-02 | 冒守华 | Marking method, system, storage medium and terminal equipment based on visualization |
| WO2022061858A1 (en) * | 2020-09-28 | 2022-03-31 | Orange | Method for assisting a first user provided with a terminal at a scene |
| CN112181146A (en) * | 2020-09-29 | 2021-01-05 | 重庆市华驰交通科技有限公司 | Wearable device display picture control method and system for electromechanical device maintenance |
| CN112383734A (en) * | 2020-10-29 | 2021-02-19 | 岭东核电有限公司 | Video processing method, video processing device, computer equipment and storage medium |
| CN113986057A (en) * | 2021-10-22 | 2022-01-28 | 北京三快在线科技有限公司 | Image processing method and device, readable storage medium and unmanned equipment |
| CN114356089A (en) * | 2021-12-30 | 2022-04-15 | Oppo广东移动通信有限公司 | Augmented reality glasses control method and device, storage medium and electronic equipment |
| CN114465910A (en) * | 2022-02-10 | 2022-05-10 | 北京为准智能科技有限公司 | Machining equipment calibration method based on augmented reality technology |
| CN114465910B (en) * | 2022-02-10 | 2022-09-13 | 北京为准智能科技有限公司 | Machining equipment calibration method based on augmented reality technology |
| CN117311892A (en) * | 2023-11-30 | 2023-12-29 | 深圳富士伟业科技有限公司 | Remote assistance method, system, electronic device and storage medium |
| CN117311892B (en) * | 2023-11-30 | 2024-03-08 | 深圳富士伟业科技有限公司 | Remote assistance method, system, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107491174B (en) | 2021-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107491174A (en) | Method, apparatus, system and electronic equipment for remote assistance | |
| CN107566793A (en) | Method, apparatus, system and electronic equipment for remote assistance | |
| JP5698733B2 (en) | Three-space input detection, representation, and interpretation: Gesture continuum incorporating free space, proximity, and surface contact modes | |
| CN108334199A (en) | The multi-modal exchange method of movable type based on augmented reality and device | |
| US20160358383A1 (en) | Systems and methods for augmented reality-based remote collaboration | |
| CN105075246B (en) | The method that Tele-immersion formula is experienced is provided using mirror metaphor | |
| MacIntyre et al. | Future multimedia user interfaces | |
| EP3769509B1 (en) | Multi-endpoint mixed-reality meetings | |
| Andersen et al. | Virtual annotations of the surgical field through an augmented reality transparent display | |
| JP5304341B2 (en) | Information visualization system, method and program | |
| Behringer et al. | Augmented Reality: Placing artificial objects in real scenes | |
| EP2974509B1 (en) | Personal information communicator | |
| KR20150105308A (en) | Operating environment with gestural control and multiple client devices. displays, and users | |
| CN108769517A (en) | A kind of method and apparatus carrying out remote assistant based on augmented reality | |
| WO2021093679A1 (en) | Visual positioning method and device | |
| CN110138831A (en) | A kind of method and apparatus carrying out remote assistance | |
| CN104464414A (en) | Augmented reality teaching system | |
| Izkara et al. | Mobile augmented reality, an advanced tool for the construction sector | |
| Lapointe et al. | A literature review of AR-based remote guidance tasks with user studies | |
| CN113253842A (en) | Scene editing method and related device and equipment | |
| Wani et al. | Augmented reality for fire and emergency services | |
| CN107463261A (en) | Three-dimensional interaction system and method | |
| WO2022174517A1 (en) | Crowd counting method and apparatus, computer device and storage medium | |
| Hachet et al. | 3D interaction with and from handheld computers | |
| CN108920598A (en) | Panorama sketch browsing method, device, terminal device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |