CN104899361B

CN104899361B - A kind of remote control method and device

Info

Publication number: CN104899361B
Application number: CN201510257853.8A
Authority: CN
Inventors: 宋星光; 廉士国
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2015-05-19
Filing date: 2015-05-19
Publication date: 2018-01-16
Anticipated expiration: 2035-05-19
Also published as: US9785266B2; US20160342224A1; CN104899361A; EP3096489A1; EP3096489B1

Abstract

The present invention relates to control technology field, more particularly to a kind of remote control method and device, in the case where the information in addition to screen content of terminal and demonstrator's demo content can be presented, to realize the remote control to terminal.The embodiment of the present invention, which provides a kind of remote control method, to be included：Demonstration graph picture is obtained, the demonstration graph picture is the image for presentation terminal operation；According to the detection parameters information of terminal, presentation screen picture is detected in the demonstration graph picture, the presentation screen picture is the screen-picture of terminal described in the demonstration graph picture；According to the parameter information of the presentation screen picture and the screen parameter information of the terminal, the mapping relations between the pixel on the display screen of the pixel and the terminal on the presentation screen picture are determined；Operation information according to user to presentation screen picture, and the mapping relations, operation control terminal.

Description

A kind of remote control method and device

Technical field

The present invention relates to multimedia technology field, more particularly to a kind of remote control method and device.

Background technology

Currently, electronic equipment species and function are more and more, are brought great convenience to people's life.When devising end Product is held, can be with remote demonstration to visitor in the case where product is presented directly to visitor by inconvenience.Demonstrator is general The use of one side products Presentation is needed, while allowing user's operation control terminal, carries out own experience.

In existing remote control mode, control terminal needs to be connected by setting up with controlled terminal, obtains to controlled terminal Control authority, and show on the display screen of control terminal the display screen content of controlled terminal.Although this remote control mode Remote control can be realized, but can not meet the needs of terminal demonstration.For example in this way, user can not control The information (such as shell of terminal) outside the screen content of terminal (namely controlled terminal) is seen at end, also, controlled terminal is by eventually After end control gives control terminal, demonstrator will be unable to use process in controlled terminal presentation terminal, it is clear that this remote control side Formula have impact on the effect of the remote demonstration of terminal.

The content of the invention

The embodiment of the present invention provides a kind of remote control method and device, to can present terminal screen content it In the case of outer information and demonstrator's demo content, the remote control to terminal is realized.

First aspect, there is provided a kind of remote control method, including：

Demonstration graph picture is obtained, the demonstration graph picture is the image for presentation terminal operation；

According to the detection parameters information of terminal, presentation screen picture, the presentation screen are detected in the demonstration graph picture Picture is the screen-picture of terminal described in the demonstration graph picture；

According to the parameter information of the presentation screen picture and the screen parameter information of the terminal, the demonstration screen is determined Mapping relations between pixel on curtain picture and the pixel on the display screen of the terminal；

Operation information according to user to the presentation screen picture, and the mapping relations, manipulate the terminal.

With reference in a first aspect, in the first possible implementation, the operation information include operational motion information and First operating position information, the first operating position information include the first pixel operated on the presentation screen picture The information of point set；

Operation information according to user to the presentation screen picture, and the mapping relations, manipulate the terminal, bag Include：

According to the information of the first pixel point set, and the mapping relations, the first pixel point set is obtained The information of corresponding the second pixel point set on the display screen of the terminal；

According to the information of the second pixel point set and the operational motion information, the terminal is manipulated.

With reference in a first aspect, in second of possible implementation, the operation information includes the first operative position confidence Breath, the first operating position information include the information of the first pixel point set operated on the presentation screen picture；

On the display screen of the terminal, track icon is moved to the opening position of the second pixel point set.

With reference in a first aspect, or the first of first aspect or second of possible implementation, in the third possible reality In existing mode, the parameter information of the presentation screen picture includes at least one in following information：The width of presentation screen picture Reference image vegetarian refreshments on angle value, the height value of the presentation screen picture and the presentation screen picture is in the demonstration graph picture In coordinate value, the screen parameter information includes the screen resolution of the terminal；

According to the parameter information of the presentation screen picture and the screen parameter information of the terminal, the demonstration screen is determined Mapping relations between pixel on curtain picture and the pixel on the display screen of the terminal, including：Drilled according to described Show that the width value of screen-picture, the height value of the presentation screen picture, the reference image vegetarian refreshments on the presentation screen picture exist The screen resolution of coordinate value and the terminal in the demonstration graph picture, determines the pixel on the presentation screen picture The mapping relations between pixel on point and the display screen.

With reference to the third possible implementation of first aspect, in the 4th kind of possible implementation, according to following Formula determines that the mapping between the pixel on the display screen of the pixel and the terminal on the presentation screen picture is closed System：

Wherein, (X, Y) is the coordinate value of the pixel on the presentation screen picture, and (Gx, Gy) is corresponding with (X, Y) The coordinate value of pixel on the display screen, TrX are the width value of the display screen, and TrY is the display screen Height value, CW are the width value of the presentation screen picture, and CH is the height value of the presentation screen picture, and (PtX, PtY) is Coordinate value of the reference image vegetarian refreshments in the demonstration graph picture, the seat that the reference image vegetarian refreshments corresponds on the display screen Mark origin.

With reference in a first aspect, or the first of first aspect or second of possible implementation, in the 5th kind of possible reality In existing mode, the screen parameter information of the terminal includes the screen resolution of the laying state information of the terminal, the terminal Rate, the parameter information of the presentation screen picture include the width value of presentation screen picture, the height of the presentation screen picture Coordinate value of the reference image vegetarian refreshments in the demonstration graph picture in value and the presentation screen picture；

According to the parameter information of the presentation screen picture and the screen parameter information of the terminal, the demonstration screen is determined Mapping relations between pixel on curtain picture and the pixel on the display screen of the terminal, including：

According to the width value of the presentation screen picture, the height value of the presentation screen picture, the reference image vegetarian refreshments The screen resolution of coordinate value, the terminal and the laying state information of the terminal in the demonstration graph picture, it is determined that The mapping relations between the pixel on pixel and the display screen on the presentation screen picture.

With reference to the 5th kind of possible implementation of first aspect, in the 6th kind of possible implementation, the placement Status information includes inclination angle of the laying state with respect to reference state of the terminal, wherein, the inclination angle is to be drilled described On diagram picture, angle that the laying state of the terminal rotates in the counterclockwise direction relative to reference state, in putting for the terminal When configuration state is the reference state, the direction where the top margin of the display screen is horizontal direction, the display screen Direction where side is vertical direction, and the pixel on the presentation screen picture and the terminal are determined according to below equation Display screen on pixel between mapping relations：

Wherein, (X, Y) is the coordinate value of the pixel on the presentation screen picture, and (Gx, Gy) is corresponding with (X, Y) The coordinate value of pixel on the display screen, the width value of display screen described in TrX, TrY are the height of the display screen Angle value, CW are the width value of the presentation screen picture, and CH is the height value of the presentation screen picture, and (PtX, PtY) is institute State coordinate value of the reference image vegetarian refreshments in the demonstration graph picture, the coordinate that the reference image vegetarian refreshments corresponds on the display screen Origin, θ are the angle value that angle value β based on the inclination angle is determined, θ be pixel (X, Y) with reference image vegetarian refreshments (PtX, PtY the angle value of the first acute angle between the line between) and the direction where the top margin of the display screen.

With reference to the 6th kind of possible implementation of first aspect, in the 7th kind of possible implementation, based on described The angle value β at inclination angle determines that the angle value θ of first acute angle includes：

When the angle value β at the inclination angle is 0, the angle value θ for determining first acute angle is α, wherein, α is The angle value of second acute angle of line and horizontal direction between pixel (X, Y) and reference image vegetarian refreshments (PtX, PtY)；Or Person

When the angle value β at the inclination angle, which is more than the 0 and angle value β at the inclination angle, is less than π, described first is determined The angle value θ of acute angle is that the angle value β at the inclination angle subtracts the difference after angle value α；Or

When the angle value β at the inclination angle is less than 3 pi/2 more than or equal to π and the inclination angle angle value β, it is determined that The angle value θ of first acute angle is that the angle value β at the inclination angle is added with angle value α, subtracts the value after π again；Or Person

When the angle value β at the inclination angle is less than 2 π more than or equal to the angle value β of 3 pi/2s and the inclination angle, really The angle value θ of fixed first acute angle is that the angle value β at the inclination angle subtracts angle value α, subtracts the value after π again.

With reference in a first aspect, or any one in first to the 7th kind of possible implementation of first aspect, In eight kinds of possible implementations, the detection parameters information according to terminal, presentation screen is detected in the demonstration graph picture Picture, including：

All objects with quadrangular configuration are detected in the demonstration graph picture；

In all objects with quadrangular configuration detected, the institute with terminal quadrangular configuration is found out State presentation screen picture, the terminal quadrangular configuration and the detection parameters information matches of the terminal.

With reference to the 8th kind of possible implementation of first aspect, in the 9th kind of possible implementation, drilled described All objects with quadrangular configuration are detected in diagram picture, including：

All objects with quadrangular configuration are detected in the range of the image detection of the determination of the demonstration graph picture.

With reference to the 9th kind of possible implementation of first aspect, in the tenth kind of possible implementation, according to including At least one of following information determines described image detection range：The presentation screen picture in the demonstration graph as in Demonstrator's hand position information, the default regional extent of the demonstration terminal operation in mobile message, the demonstration graph picture Information.

Second aspect, there is provided a kind of remote operation controller, including：

Acquisition module, for obtaining demonstration graph picture, the demonstration graph picture is the image for presentation terminal operation；

Detection module, for the detection parameters information according to terminal, in the demonstration graph that the acquisition module is got Presentation screen picture is detected as in, the presentation screen picture is the screen-picture of terminal described in the demonstration graph picture；

Determining module, for the parameter information of the presentation screen picture that is detected according to the detection module and described The screen parameter information of terminal, determine the pixel on the display screen of the pixel and the terminal on the presentation screen picture Mapping relations between point；

Operational module, it is true for the operation information according to user to the presentation screen picture, and the determining module The fixed mapping relations, manipulate the terminal.

With reference to second aspect, in the first possible implementation, the first operating position information includes described drill Show the information of the first pixel point set operated on screen-picture；

The operational module is specifically used for：

According to the information of the first pixel point set, and the mapping relations, the first pixel point set is obtained The information of corresponding the second pixel point set on the display screen of the terminal；According to the information of the second pixel point set and The operational motion information, manipulates the terminal.

With reference to second aspect, in second of possible implementation, the operation information includes the first operative position confidence Breath, the first operating position information include the information of the first pixel point set operated on the presentation screen picture；

The operational module is specifically used for：

According to the information of the first pixel point set, and the mapping relations, the first pixel point set is obtained The information of corresponding the second pixel point set on the display screen of the terminal；On the display screen of the terminal, by track Icon is moved to the opening position of the second pixel point set.

With reference to second aspect, or the first of second aspect or second of possible implementation, in the third possible reality In existing mode, the parameter information of the presentation screen picture includes at least one in following information：The width of presentation screen picture Reference image vegetarian refreshments on angle value, the height value of the presentation screen picture and the presentation screen picture is in the demonstration graph picture In coordinate value, the screen parameter information includes the screen resolution of the terminal；

The determining module is specifically used for：

Drawn according to the width value of the presentation screen picture, the height value of the presentation screen picture, the presentation screen The screen resolution of coordinate value and the terminal of the reference image vegetarian refreshments in the demonstration graph picture on face, it is determined that described drill Show the mapping relations between the pixel on the pixel and the display screen on screen-picture.

With reference to the third possible implementation of second aspect, in the 4th kind of possible implementation, the determination Module is specifically used for being determined according to below equation on the display screen of the pixel and the terminal on the presentation screen picture Pixel between mapping relations：

Wherein, (X, Y) is the coordinate value of the pixel on the presentation screen picture, and (Gx, Gy) is corresponding with (X, Y) The coordinate value of pixel on the display screen, the width value of display screen described in TrX, TrY are the height of the display screen Angle value, CW are the width value of the presentation screen picture, and CH is the height value of the presentation screen picture, and (PtX, PtY) is institute State coordinate value of the reference image vegetarian refreshments in the demonstration graph picture, the coordinate that the reference image vegetarian refreshments corresponds on the display screen Origin.

With reference to second aspect, or the first of second aspect or second of possible implementation, in the 5th kind of possible reality In existing mode, the screen parameter information of the terminal includes the screen resolution of the laying state information of the terminal, the terminal Rate, the parameter information of the presentation screen picture include the width value of presentation screen picture, the height of the presentation screen picture Coordinate value of the reference image vegetarian refreshments in the demonstration graph picture in value and the presentation screen picture；

The determining module is specifically used for：

With reference to the 5th kind of possible implementation of second aspect, in the 6th kind of possible implementation, the placement Status information includes inclination angle of the laying state with respect to reference state of the terminal, wherein, the inclination angle is to be drilled described On diagram picture, angle that the laying state of the terminal rotates in the counterclockwise direction relative to reference state, in putting for the terminal When configuration state is the reference state during reference state, the direction where the top margin of the display screen is horizontal direction, described Direction where the side of display screen is vertical direction, and the determining module is specifically used for drilling according to determining below equation Show the mapping relations between the pixel on the display screen of the pixel and the terminal on screen-picture：

Wherein, (X, Y) is the coordinate value of the pixel on the presentation screen picture, and (Gx, Gy) is corresponding with (X, Y) The coordinate value of pixel on the display screen, the width value of display screen described in TrX, TrY are the height of the display screen Angle value, CW are the width value of the presentation screen picture, and CH is the height value of the presentation screen picture, and (PtX, PtY) is institute State coordinate value of the reference image vegetarian refreshments in the demonstration graph picture, the coordinate that the reference image vegetarian refreshments corresponds on the display screen Origin.θ is the angle value that angle value β based on the inclination angle is determined, θ be pixel (X, Y) with reference image vegetarian refreshments (PtX, PtY the angle value of the first acute angle between the line between) and the direction where the top margin of the display screen.

With reference to the 6th kind of possible implementation of second aspect, in the 7th kind of possible implementation, the determination Module is specifically used for：

With reference to any one in second aspect, or first to the 7th kind of possible implementation of second aspect, In eight kinds of possible implementations, the detection module is specifically used for：

All objects with quadrangular configuration are detected in the demonstration graph picture；Described all have four what is detected In the object of side shape profile, the presentation screen picture with terminal quadrangular configuration, the terminal quadrangle wheel are found out The wide and detection parameters information matches of the terminal.

With reference to the 8th kind of possible implementation of second aspect, in the 9th kind of possible implementation, the detection Module is specifically used for：

With reference to the 9th kind of possible implementation of second aspect, in the tenth kind of possible implementation, shown detection Module is specifically used for determining described image detection range according to including at least one of following information：The presentation screen picture The mobile message in the demonstration graph picture, demonstrator's hand position of the demonstration terminal operation in the demonstration graph picture Information, default regional extent information.

The third aspect, there is provided a kind of remote control equipment, including：Processor, memory and bus；The memory storage Execute instruction, when the remote control equipment is run, pass through bus communication between the processor and the memory so that The following execute instruction of computing device：

With reference to the third aspect, in the first possible implementation, the operation information include operational motion information and First operating position information, the first operating position information include the first pixel operated on the presentation screen picture The information of point set；In the execute instruction of the computing device, according to operation of the user to the presentation screen picture Information, and the mapping relations, the terminal is manipulated, including：

With reference to the third aspect, in second of possible implementation, the operation information includes the first operative position confidence Breath, the first operating position information include the information of the first pixel point set operated on the presentation screen picture； In the execute instruction of the computing device, the operation information according to user to the presentation screen picture, and it is described Mapping relations, the terminal is manipulated, including：

With reference to the third aspect, or the first of the third aspect or second of possible implementation, in the third possible reality In existing mode, the parameter information of the presentation screen picture includes at least one in following information：The width of presentation screen picture Reference image vegetarian refreshments on angle value, the height value of the presentation screen picture and the presentation screen picture is in the demonstration graph picture In coordinate value, the screen parameter information includes the screen resolution of the terminal；

In the execute instruction of the computing device, according to the parameter information of the presentation screen picture and the end The screen parameter information at end, determine the pixel on the display screen of the pixel and the terminal on the presentation screen picture Between mapping relations, including：

With reference to the third possible implementation of the third aspect, in the 4th kind of possible implementation, the processing In the execute instruction that device performs, the pixel on the presentation screen picture and the terminal are determined according to below equation The mapping relations between pixel on display screen：

With reference to the third aspect, or the first of the third aspect or second of possible implementation, in the 5th kind of possible reality In existing mode, the screen parameter information of the terminal includes the screen resolution of the laying state information of the terminal, the terminal Rate, the parameter information of the presentation screen picture include the width value of presentation screen picture, the height of the presentation screen picture Coordinate value of the reference image vegetarian refreshments in the demonstration graph picture in value and the presentation screen picture；The computing device In the execute instruction, according to the parameter information of the presentation screen picture and the screen parameter information of the terminal, institute is determined The mapping relations between the pixel on the display screen of the pixel and the terminal on presentation screen picture are stated, including：

With reference to the 5th kind of possible implementation of the third aspect, in the 6th kind of possible implementation, the placement Status information includes inclination angle of the laying state with respect to reference state of the terminal, wherein, the inclination angle is to be drilled described On diagram picture, angle that the laying state of the terminal rotates in the counterclockwise direction relative to reference state, in putting for the terminal When configuration state is the reference state, the direction where the top margin of the display screen is horizontal direction, the display screen Direction where side is vertical direction, in the execute instruction of the computing device, according to determining below equation Mapping relations between pixel on presentation screen picture and the pixel on the display screen of the terminal：

With reference to the 6th kind of possible implementation of the third aspect, in the 7th kind of possible implementation, the processing In the execute instruction that device performs, the angle value β based on the inclination angle determines the angle value θ of first acute angle, Including：

With reference to any one in the third aspect, or first to the 7th kind of possible implementation of the third aspect, In eight kinds of possible implementations, in the execute instruction of the computing device, according to the detection parameters information of terminal, Presentation screen picture is detected in the demonstration graph picture, including：

With reference to the 8th kind of possible implementation of first aspect, in the 9th kind of possible implementation, the processing In the execute instruction that device performs, all objects with quadrangular configuration are detected in the demonstration graph picture, including：

With reference to the 9th kind of possible implementation of the third aspect, in the tenth kind of possible implementation, the processing In the execute instruction that device performs, described image detection range is determined according to including at least one of following information：It is described The mobile message in the demonstration graph picture of presentation screen picture, the demonstration terminal operation in the demonstration graph picture are drilled The person's of showing hand position information, default regional extent information.

The scheme provided according to any of the above-described aspect, the demonstration for presentation terminal operation is gathered using image capture device Image, is presented to the visitor of distal end, and in the demonstration graph picture detection terminal presentation screen picture, establish in demonstration graph picture Mapping relations between pixel on the presentation screen picture of terminal and the pixel on the display screen of terminal reality；According to User manipulates the terminal to the operation information of the presentation screen picture of terminal, and the mapping relations.So as to use this hair Bright embodiment, the information (such as terminal enclosure pattern) outside the screen content of terminal can not only be presented in demonstration graph picture and drill The person's of showing demo content, the remote control to terminal can also be realized.

Brief description of the drawings

Fig. 1 is the remote control method flow chart that the embodiment of the present invention one provides；

Fig. 2 is the remote control method flow chart that the embodiment of the present invention two provides；

Fig. 3 (a) is the schematic diagram operated to the terminal screen picture in demonstration graph picture；

Fig. 3 (b) be on the screen of terminal, the position that trace ball moves according to laser spots moves, schematic diagram；

When Fig. 4 (a) is angle of inclination beta=0, the schematic diagram of mapping relations is calculated；

The schematic diagram of mapping relations is calculated when Fig. 4 (b) is 0 ＜ β≤pi/2；

The schematic diagram of mapping relations is calculated when Fig. 4 (c) is pi/2 ＜ β ＜ π；

The schematic diagram of mapping relations is calculated when Fig. 4 (d) is π≤β 3 pi/2s of ＜；

The schematic diagram of mapping relations is calculated when Fig. 4 (e) is β >=3 pi/2；

Fig. 5 is the system architecture schematic diagram for the long-range remote control of support that the embodiment of the present invention three provides；

Fig. 6 is application schematic diagram of the embodiment of the present invention in the case where demonstrating scene；

Fig. 7 is application schematic diagram of the embodiment of the present invention in family's long-distance video chat scenario；

Fig. 8 is application schematic diagram of the embodiment of the present invention in remote visible handwritten signature scene；

Fig. 9 is the remote operation controller structural representation that the embodiment of the present invention four provides；

Figure 10 is the remote control device structure schematic diagram that the embodiment of the present invention five provides.

Embodiment

The basic thought of the embodiment of the present invention is：Demonstration graph for presentation terminal operation is gathered using image capture device Picture, is presented to the visitor of distal end, and in the demonstration graph picture detection terminal presentation screen picture, establish in demonstration graph picture eventually Mapping relations between pixel on the presentation screen picture at end and the pixel on the display screen of terminal reality；According to Family manipulates the terminal to the operation information of the presentation screen picture of terminal, and the mapping relations.So as to using the present invention Embodiment, the information (such as terminal enclosure pattern) outside the screen content of terminal and demonstration can not only be presented in demonstration graph picture Person's demo content, the remote control to terminal can also be realized.

The embodiment of the present invention is described in further detail with reference to Figure of description.

Embodiment one

As shown in figure 1, the remote control method flow chart provided for the embodiment of the present invention one, comprises the following steps：

S101：Demonstration graph picture is obtained, the demonstration graph picture is the image for presentation terminal operation.

In specific implementation, demonstration graph picture during image capture device collection demonstrator presentation terminal operation can be used. Here demonstrator is the user to other user's presentation terminal operating process.

The terminal of the embodiment of the present invention can be mobile terminal, or the terminal with touch-screen.

S102：According to the detection parameters information of terminal, presentation screen picture, the demonstration are detected in the demonstration graph picture Screen-picture is the screen-picture of terminal described in the demonstration graph picture.

In specific implementation, the executive agent of the embodiment of the present invention can be independently of the control device outside terminal, than Such as can be independent PC (Personal Computer, PC) etc. or the function mould on terminal Block, do not limit in the embodiment of the present invention.

, can be according to the ratio of width to height (width and the height of the detection parameters information of terminal, such as terminal in specific implementation process The ratio of degree, here, the width value of terminal may be considered the length value of the top margin of the display screen of terminal, and height value can be recognized For be terminal display screen side length value), (here, screen resolution can include the aobvious of terminal to screen resolution Resolution ratio (corresponding width value) of the display screen curtain on the direction where top margin, and the resolution ratio on the direction where side are (right Answer height value)), the one or more in laying state information (such as transverse screen, portrait layout, place inclination angle etc.) etc., in demonstration graph Detection matches the presentation screen picture (i.e. the screen-picture of terminal in demonstration graph picture) of the detection parameters information of the terminal as in.This In, in the demonstration graph picture that image capture device collects, the image of demonstrator may be included, the image of terminal (includes terminal Presentation screen picture) and other background images, the embodiment of the present invention needs therefrom to detect and user's operation is related drills Show screen-picture, make place mat for following step S103.

S103：According to the parameter information of the presentation screen picture and the screen parameter information of the terminal, it is determined that described Mapping relations between pixel on presentation screen picture and the pixel on the display screen of the terminal.

Here, the parameter information of the presentation screen picture can include at least one in following information：Presentation screen Reference image vegetarian refreshments on the width value of picture, the height value of presentation screen picture and the presentation screen picture is in the demonstration Coordinate value in image；The screen parameter information of terminal can include at least one in following information：The screen of the terminal Resolution ratio, the laying state information of the terminal, the width value and height value of the terminal.

After the presentation screen picture of the terminal during S102 detects demonstration graph picture, it can establish on the presentation screen picture Coordinate value of the pixel in the demonstration graph picture display screen actual with terminal on pixel the terminal display screen On coordinate value between mapping relations, so as to based on the mapping relations, by presentation screen picture of the user in demonstration graph picture On operating position be converted to the operating position on the actual display screen of terminal.

S104：Operation information according to user to the presentation screen picture, and the mapping relations, manipulate the end End.

Here user is the user operated to the presentation screen picture in demonstration graph picture, is corresponding with demonstrator Visitor.

Here operation information can include operational motion information and the first operating position information, the first operative position confidence Breath includes the information of the first pixel point set operated on the presentation screen picture；

In S104, the operation information according to user to the presentation screen picture, and the mapping relations, described in manipulation Terminal, it can include：According to the information of the first pixel point set, and the mapping relations, first pixel is obtained The information of corresponding the second pixel point set on the display screen of the terminal of point set；According to the second pixel point set Information and the operational motion information, manipulate the terminal.Or only it can be performed according to the information of the first pixel point set unrestrained Trip operation, that is, according to the information of the first pixel point set, and the mapping relations, obtain first pixel The information of corresponding the second pixel point set on the display screen of the terminal of set；According to the letter of the second pixel point set Breath, the track icon on the display screen of the terminal is moved.The description of embodiment as described below.

Below by embodiment two, the thought of the embodiment of the present invention is described in further detail.

Embodiment two

As shown in Fig. 2 the remote control method flow chart provided for the embodiment of the present invention two, comprises the following steps：

S201：All objects with quadrangular configuration are detected in the demonstration graph picture of acquisition；In all tools detected Have in the object of quadrangular configuration, find out the presentation screen picture with terminal quadrangular configuration, the terminal quadrangle wheel The wide and detection parameters information matches of the terminal.

Quadrangular configuration described in the embodiment of the present invention can be the profile for the quadrangle that four angles have certain radian.

Here, the detection parameters information of terminal can include in the ratio of width to height of terminal, screen resolution, laying state information One or more.Wherein, if detection parameters information includes the screen resolution of terminal, not including the ratio of width to height, then screen is passed through The side where the ratio of width to height of terminal, such as the corresponding top margin in display screen of screen resolution can be calculated in curtain resolution ratio Upward number of pixels is 600 (being width value), and the number of pixels on the direction where the side of display screen is 800 (being height value), then the ratio of width to height of terminal is 3:4.In specific implementation, the acquiescence laying state of terminal can be set, such as Giving tacit consent to laying state is：Direction where the top margin of the display screen of terminal is horizontal direction, the side of the display screen of terminal The direction at place is vertical direction, requires that demonstrator (user of presentation terminal operation) will in presentation terminal in this case Terminal is placed under the laying state of acquiescence.If being not provided with the acquiescence laying state of terminal, can by laying state information come The current placed angle of instruction terminal, namely terminal laying state with respect to the inclination angle of reference state, reference state can be just It is the acquiescence laying state.

In specific implementation, if the executive agent of the embodiment of the present invention is independently of the entity outside terminal, need to obtain The parameter information of terminal transmission is taken, specifically, parameter information can be transferred to the embodiment of the present invention by terminal by being wirelessly transferred Executive agent.

Alternatively, all objects with quadrangular configuration are detected in demonstration graph picture, including：

In specific implementation, it is all with quadrangular configuration to detect to travel through each pixel in demonstration graph picture Object, but because screen profile detection is time-consuming bigger, in specific implementation, image detection scope can be determined first, true Object of the detection with quadrangular configuration in the range of fixed image detection.

It is alternatively possible to image detection scope is determined according at least one of following information：

The presentation screen picture described in the demonstration of mobile message, the demonstration graph as in of the demonstration graph as in Demonstrator's hand position information, the default regional extent information of terminal operation.

In specific implementation, demonstrator typically understands mobile terminal when presentation terminal operates, and the background in demonstration graph picture Picture will not generally change, and therefore, can determine a figure based on the presentation screen picture transition information in demonstration graph picture As detection range so that it is determined that image detection in the range of include the picture changed；In addition, the spy of application scenarios is considered Point, terminal typically can be near hand, therefore can determine that image is examined by detecting in demonstration graph picture demonstrator's hand position Scope is surveyed, hand position detection can use existing such as Gesture Recognition Algorithm based on Kinect etc..In addition to this it is possible to Made according to demonstrator's height, presentation screen size, restriction to demonstrator present position etc. to pre-set a regional extent For image detection scope.In specific implementation, image detection scope can be determined based on the comprehensive analysis of above-mentioned three kinds of information, Such as using be based respectively on the image detection scope that each information obtains common factor or intersection as the image detection finally determined Scope.

It is determined that after image detection scope, can according to the screen resolution (the ratio of width to height of counterpart terminal) of terminal, place Status information (such as transverse screen, portrait layout, place inclination angle etc.), terminal the area of presentation screen picture (can be based on terminal Scaling when being shot corresponding to screen resolution and demonstration graph picture determines the area) etc., the detection in demonstration graph picture The presentation screen picture of detection parameters information with the terminal.

S202：According to the parameter information of the presentation screen picture and the screen parameter information of the terminal, it is determined that described Mapping relations between pixel on presentation screen picture and the pixel on the display screen of the terminal.

Here it is possible to according to the presentation screen picture of terminal in the demonstration graph picture detected and the screen parameter of the terminal Information, the pixel on the display screen of the terminal corresponding to the pixel on presentation screen picture is determined, closed based on the mapping System performs associative operation instruction.Two kinds of specific mapping modes will be introduced after S203.

S203：It is (described to drill to the first operating position information of the presentation screen picture of terminal in demonstration graph picture according to user Show the information of the first pixel point set operated on screen-picture), and the mapping relations, determine first operation The second operating position information (the second picture on the display screen of the terminal on the display screen of terminal corresponding to positional information The information of vegetarian refreshments set)；According to the second operating position information and the operational motion information, the terminal is manipulated.

Alternatively, corresponding the second pixel point set on the display screen of the terminal of the first pixel point set is obtained Information after, track icon on the display screen of the terminal, can also be moved to the position of the second pixel point set Put place.

In specific implementation, the operational motion indicated by operational motion information can be click on operation, slide etc..Remove Outside this, be also based on the second operating position information show on the display screen visitor (based on demonstration graph as in terminal The user that presentation screen picture is operated) roam operation；Here roam operation refers to that operation instruction icon shields in demonstration The operation moved on curtain picture, such as, visitor uses laser pen, and presentation screen picture and shift position are pointed to laser spots. For roam operation, in order that demonstrator sees the roam operation track of visitor, can be set on the display screen of terminal Track icon, the track icon is set to be moved according to the operating position of roam operation.As shown in Fig. 3 (a), visitor's laser Pen gets laser spots and points to the presentation screen picture of terminal in demonstration graph picture and move, at this moment, in the display screen of terminal The position that upper track icon (can be trace ball) is moved according to laser spots moves, as shown in Fig. 3 (b).

Below, to determining that the mode of mapping relations is described further in S202.

In S202, according to the parameter information of the presentation screen picture and the screen parameter information of the terminal, it is determined that Mapping relations between pixel on the presentation screen picture and the pixel on the display screen of the terminal.The present invention Following two modes are provided in embodiment and determine the mapping relations：

Mode one：The laying state of terminal is default conditions：For example the top margin of the display screen of terminal is (such as Fig. 4 (a) institutes Show) where direction be horizontal direction, the direction where the side of the display screen of terminal is vertical direction；The screen parameter Information includes the screen resolution of the terminal；The parameter information of the presentation screen picture includes：The width of presentation screen picture Angle value, the presentation screen picture height value and the presentation screen picture on reference image vegetarian refreshments in the demonstration graph picture In coordinate value；

Specifically, according to the width value of the presentation screen picture (pixel number can be used to represent), described drill Show that the height value (can be represented using pixel number) of screen-picture, the reference image vegetarian refreshments on the presentation screen picture are existed The screen resolution of coordinate value and the terminal in the demonstration graph picture, determine the presentation screen of terminal in demonstration graph picture Mapping relations between pixel on picture and the pixel on the display screen of the terminal.

For example, as shown in Fig. 4 (a), the pixel on the presentation screen picture and institute can be determined according to below equation State the mapping relations between the pixel on the display screen of terminal：

Wherein, (X, Y) is the coordinate value of the pixel on the presentation screen picture, and (Gx, Gy) is the display screen On pixel coordinate value, TrX be terminal in the resolution ratio where the top margin of the display screen on direction, as display screen The width value of curtain, TrY are height of the terminal in the resolution ratio where the side of the display screen on direction, as display screen Value；CW is the width value of the presentation screen picture, and CH is the height value of the presentation screen picture；(PtX, PtY) is described Coordinate value of the reference image vegetarian refreshments in the demonstration graph picture；The coordinate that the reference image vegetarian refreshments corresponds on the display screen is former Point.

In Fig. 4 (a), the origin of coordinates in demonstration graph picture is demonstration graph as the pixel in the upper left corner；In demonstration graph picture, Axis of abscissas direction is horizontal direction to the right, and axis of ordinates direction is direction vertically downward.On the display screen of terminal The origin of coordinates be terminal acquiescence laying state (direction where the top margin of display screen be horizontal direction, display screen side Direction where side is vertical direction, such as under portrait layout state) under when the terminal display upper left corner pixel, at end In the display screen at end, the horizontal direction to the right when axis of abscissas direction is in acquiescence laying state, axis of ordinates side To the direction vertically downward being as in when giving tacit consent to laying state.

Explanation is needed exist for, above-mentioned formula is only one of example of mode one, any thought based on mode one, is led to The example that the simple deformation of above-mentioned formula obtains is crossed to all fall in the scope of protection of the present invention.

Mode two, the laying state of terminal can have certain inclination with respect to reference state (than such as above-mentioned default conditions) Angle；The screen parameter information includes the screen resolution and laying state information of the terminal；The presentation screen picture Parameter information includes the width value of presentation screen picture, the height value of the presentation screen picture and the presentation screen picture On coordinate value of the reference image vegetarian refreshments in the demonstration graph picture；

Specifically, according to the width value of the presentation screen picture, the height value of the presentation screen picture, the reference Coordinate value, the screen resolution of the terminal and the laying state letter of the terminal of the pixel in the demonstration graph picture Breath, determines the mapping relations between the pixel on the pixel and the display screen on the presentation screen picture.

In specific implementation, the laying state information can include the laying state of the terminal with respect to reference state Inclination angle, wherein, the inclination angle is on the demonstration graph picture, and the laying state of the terminal is with respect to reference state along the inverse time The angle that pin direction rotates, when the laying state of the terminal is the reference state, where the top margin of the display screen Direction be horizontal direction, the direction where the side of the display screen is vertical direction；It can be determined according to below equation Mapping relations between pixel on the presentation screen picture and the pixel on the display screen of the terminal：

Wherein, (X, Y) is the coordinate value of the pixel on the presentation screen picture, and (Gx, Gy) is corresponding with (X, Y) The coordinate value of pixel on the display screen, the width value of display screen described in TrX, TrY are the height of the display screen Angle value, CW are the width value of the presentation screen picture, and CH is the height value of the presentation screen picture, and (PtX, PtY) is institute State coordinate value of the reference image vegetarian refreshments in the demonstration graph picture, the coordinate that the reference image vegetarian refreshments corresponds on the display screen Origin.θ is the angle value that angle value β based on the inclination angle is determined, θ be pixel (X, Y) with reference image vegetarian refreshments (PtX, PtY the angle value of the first acute angle between the line between) and the direction where the top margin of the display screen (can also Be line between pixel (X, Y) and reference image vegetarian refreshments (PtX, PtY) and the direction where the top margin of the display screen it Between obtuse angle angle angle value, now sin θ, cos θ take absolute value in above-mentioned formula).

In specific implementation, the difference of the angle value β based on inclination angle, θ specific calculation is also different.With with reference to shape Exemplified by state is acquiescence laying state described in aforesaid way one：

As shown in Fig. 4 (a), β=0, now：θ=α；

As shown in Fig. 4 (b), 0 ＜ β≤pi/2, now：θ=β-α；

As shown in Fig. 4 (c), pi/2 ＜ β ＜ π, now：θ=β-α；

As shown in Fig. 4 (d), the pi/2s of π≤β ＜ 3, now：θ=alpha+beta-π；

As shown in Fig. 4 (e), the pi/2 of β >=3, now：θ=β-α-π；

Wherein,Or,

In Fig. 4 (a)~(e), the origin of coordinates in demonstration graph picture is demonstration graph as the pixel in the upper left corner；In demonstration graph As in, axis of abscissas direction is horizontal direction to the right, and axis of ordinates direction is direction vertically downward.The display screen of terminal On the origin of coordinates be terminal (direction where the top margin of display screen is horizontal direction, display screen in acquiescence laying state Side where direction be vertical direction, such as under portrait layout state) under when the terminal display upper left corner pixel, In the display screen of terminal, the horizontal direction to the right when axis of abscissas direction is in acquiescence laying state, ordinate Direction vertically downward when direction of principal axis is in acquiescence laying state.

In order to which the thought that the embodiment of the present invention supports remote control is better described, make lower be situated between from the angle of system below Continue.

Embodiment three

As shown in figure 5, the system architecture schematic diagram of the support remote control provided for the embodiment of the present invention three.In the system In, the executive agent of above method step is independently of the equipment outside terminal, for example is PC (Personal Computer, PC), parameter information can be transferred to PC by terminal by wireless socket (Socket).

Within the system, realizing the main modular of function of the embodiment of the present invention includes：

Terminal algorithm module 51, is deployed in terminal, for by terminal parameter information, (such as screen resolution, laying state to be believed Breath etc.) be transferred to mapping block 53, and respond the transmission of mapping block 53 manipulation instruction (as the response click of visitor, slip, Roam operation etc.)；

Terminal screen detection module 52, PC ends are deployed in, for the image determined in image detection range computation module 55 Detection quadrangular configuration in detection range, in all quadrangular configurations detected, search what matching mapping block 53 transmitted The quadrangular configuration of terminal parameter information, and the presentation screen using the picture in the quadrangular configuration found as the terminal Picture；

Mapping block 53, be deployed in PC ends, for receiving terminal algoritic module 51 transmit terminal parameter information, and by its It is transferred to terminal screen detection module 52；And the terminal detected according to terminal parameter information and terminal screen detection module 52 Presentation screen picture, generate demonstration graph picture in terminal presentation screen picture on pixel and terminal display screen on Mapping relations between pixel；According to the mapping relations of generation, the operation information that manipulation information detecting module 54 transmits is turned Change the manipulation instruction for terminal into, and be transferred to terminal algorithm module 51；

Information detecting module 54 is manipulated, is deployed in PC ends, for detecting manipulation information (the hand-held laser of such as visitor of user The roaming of pen or clicking operation etc.), and manipulation information is sent to mapping block 53；

Image detection range computation module 55, is deployed in PC ends, according to demonstrator's hand position information of detection, it is determined that figure As detection range, and it is transferred to terminal screen detection module 52.

Explanation on application scenarios of the embodiment of the present invention：

The embodiment of the present invention can be applied not only to the scene of end product demonstration, in other scenes for needing remote control Under it is equally applicable, for example can be applicable to the auxiliary operation scene of similar emotional affection care, remote visible handwritten signature scene etc..

Scene one：End product is demonstrated.

As shown in fig. 6, demonstrator picks up terminal, by terminal display towards image capture device, network passes in PC The demonstration graph picture that image capture device gathers is transferred to distal end by defeated module, and is presented to distal end by the display screen of distal end Visitor.Visitor can be clearly visible that the presentation screen picture of terminal by display screen, and can use interactive tool Operation remotely is performed to the presentation screen picture；Interactive device detects the operation information of visitor, and is transferred to the PC of demo end Machine；The pixel of the presentation screen picture of terminal and terminal is aobvious in PC combination far-end operation information and the demonstration graph picture determined The mapping relations between pixel on display screen curtain, generation are directed to the manipulation instruction of terminal, and are transferred to terminal；Terminal response connects The manipulation instruction received, demonstrator can see the operation of local visitor by the feedback of the display screen of terminal.

Scene two：Assist operation.

As shown in fig. 7, in family's long-distance video chat scenario, A ends member can see B ends by Video chat window The terminal that member holds, the PC that the terminal is used for Video chat with B ends member pass through wired or wireless connection.A ends member can With the terminal held by mouse control B end members, implementation is consistent with the mode of above-described embodiment introduction：It is used at A ends Detection terminal in the demonstration graph picture that the PC of Video chat receives, and the mapping relations are generated, based on the mapping relations, will examine The operation information generation of the A ends member measured is sent to the terminal at B ends, the terminal response at B ends should for the operational order of terminal Operational order.

Scene three：Remote visible handwritten signature.

As digital handwriting signature becomes more and more extensive, the local mode for carrying out digital handwriting signature increasingly can not Meet signature demand, such as, if be only by local signature file transmit to distal end, either in security or with All it has been short of in the experience of family, remote visible handwritten signature can be realized using the embodiment of the present invention.

As shown in figure 8, on the screen of local terminal, signer (visitor of corresponding above-mentioned scene one) is it can be seen that distal end Document to be signed on terminal display, signer can hold signature tool (such as signature pen) in the signature position of document to be signed Upper signature, and operation information is transmitted to distal end (such as signature trace information).In distal end, according to the mapping relations of generation, will sign The operation information of name person is mapped as showing the signature of signer in the manipulation instruction for terminal, namely control terminal display Track.

In addition to track of signing, the visual biological features such as terminal camera collection signer face can also be used, use terminal Microphone gathers signer voice data, the multi-modal signature with reference to various features generation to specific file destination.Signing During certification, it again may be by gathering the face of user, sound and the data such as track of signing be authenticated；This multi-modal label Name mode can improve the security of signature authentication.

Based on same inventive concept, a kind of corresponding with remote control method remotely grasp is additionally provided in the embodiment of the present invention Device is controlled, because the principle of device solution problem is similar to remote control method of the embodiment of the present invention, therefore the reality of the device The implementation for the method for may refer to is applied, part is repeated and repeats no more.

Example IV

As shown in figure 9, the remote operation controller structural representation provided for the embodiment of the present invention four, including：

Acquisition module 91, for obtaining demonstration graph picture, the demonstration graph picture is the image for presentation terminal operation；

Detection module 92, for the detection parameters information according to terminal, got in the acquisition module 91 described in drill Presentation screen picture is detected in diagram picture, the presentation screen picture is the screen-picture of terminal described in the demonstration graph picture；

Determining module 93, for the parameter information of the presentation screen picture that is detected according to the detection module 92 and The screen parameter information of the terminal, on the display screen for determining the pixel and the terminal on the presentation screen picture Mapping relations between pixel；

Operational module 94, for the operation information according to user to the presentation screen picture, and the determining module 93 mapping relations determined, manipulate the terminal.

Alternatively, the operation information includes operational motion information and the first operating position information, first operative position Confidence breath includes the information of the first pixel point set operated on the presentation screen picture；

The operational module 94 is specifically used for：

Alternatively, the operation information includes the first operating position information, and the first operating position information includes described The information of the first pixel point set operated on presentation screen picture；

The operational module 94 is specifically used for：

Alternatively, the parameter information of the presentation screen picture includes at least one in following information：Presentation screen is drawn Reference image vegetarian refreshments on the width value in face, the height value of the presentation screen picture and the presentation screen picture is drilled described Coordinate value in diagram picture, the screen parameter information include the screen resolution of the terminal；

The determining module 93 is specifically used for：

Alternatively, the determining module 93 is specifically used for determining the pixel on the presentation screen picture according to below equation The mapping relations between pixel on point and the display screen of the terminal：

Alternatively, the screen parameter information of the terminal includes the laying state information of the terminal, the screen of the terminal Curtain resolution ratio, the parameter information of the presentation screen picture include width value, the presentation screen picture of presentation screen picture Height value and the presentation screen picture on coordinate value of the reference image vegetarian refreshments in the demonstration graph picture；

The determining module 93 is specifically used for：

According to the width value of the presentation screen picture, the height value of the presentation screen picture, the reference image vegetarian refreshments The screen resolution of coordinate value, the terminal and the laying state information of the terminal in the demonstration graph picture, it is determined that The mapping relations between the pixel on pixel and the display screen on the presentation screen picture；

Alternatively, the laying state information includes inclination angle of the laying state with respect to reference state of the terminal, its In, the inclination angle is that the laying state of the terminal revolves in the counterclockwise direction with respect to reference state on the demonstration graph picture The angle turned, when the laying state of the terminal is the reference state, the direction where the top margin of the display screen is Horizontal direction, the direction where the side of the display screen are vertical direction, the determining module 93 be specifically used for according to Lower formula determines the mapping between the pixel on the display screen of the pixel and the terminal on the presentation screen picture Relation：

Alternatively, the determining module 93 is specifically used for：

Alternatively, the detection parameters information of the terminal includes at least one of following information：The width of the terminal is high Than, the laying state information of the screen resolution of the terminal, the terminal.

Alternatively, the mapping parameters information of the terminal includes at least one of following information：The screen of the terminal Resolution ratio, the laying state information of the terminal, the width value and height value of the terminal.

Alternatively, the detection module 92 is specifically used for：

Alternatively, the detection module 92 is specifically used for determining described image according to including at least one of following information Detection range：The mobile message in the demonstration graph picture of the presentation screen picture, the demonstration institute in the demonstration graph picture State demonstrator's hand position information of terminal operation, default regional extent information.

Embodiment five

As shown in Figure 10, the structural representation of remote control equipment 100 provided for the embodiment of the present invention five, including：Processing Device 101, memory 102 and bus 103；The memory 102 stores execute instruction, when the remote control equipment 100 is run When, communicated between the processor 101 and the memory 102 by bus 103 so that the processor 101 performs as follows Execute instruction：

Alternatively, the operation information includes operational motion information and the first operating position information, first operative position Confidence breath includes the information of the first pixel point set operated on the presentation screen picture；The processor 101 performs The execute instruction in, the operation information according to user to the presentation screen picture, and the mapping relations, manipulate institute Terminal is stated, including：

In the execute instruction that the processor 101 performs, believed according to operation of the user to the presentation screen picture Breath, and the mapping relations, manipulate the terminal, including：

In the execute instruction that the processor 101 performs, according to the parameter information of the presentation screen picture and institute The screen parameter information of terminal is stated, determines the picture on the display screen of the pixel and the terminal on the presentation screen picture Mapping relations between vegetarian refreshments, including：

Alternatively, in the execute instruction that the processor 101 performs, the presentation screen is determined according to below equation Mapping relations between pixel on picture and the pixel on the display screen of the terminal：

Alternatively, the screen parameter information of the terminal includes the laying state information of the terminal, the screen of the terminal Curtain resolution ratio, the parameter information of the presentation screen picture include width value, the presentation screen picture of presentation screen picture Height value and the presentation screen picture on coordinate value of the reference image vegetarian refreshments in the demonstration graph picture；The processor In 101 execute instructions performed, believed according to the parameter information of the presentation screen picture and the screen parameter of the terminal Breath, determine that the mapping between the pixel on the display screen of the pixel and the terminal on the presentation screen picture is closed System, including：

Alternatively, the laying state information includes inclination angle of the laying state with respect to reference state of the terminal, its In, the inclination angle is that the laying state of the terminal revolves in the counterclockwise direction with respect to reference state on the demonstration graph picture The angle turned, when the laying state of the terminal is the reference state, the direction where the top margin of the display screen is Horizontal direction, the direction where the side of the display screen are vertical direction, and the execution that the processor 101 performs refers to In order, the pixel on the display screen of the pixel and the terminal on the presentation screen picture is determined according to below equation Between mapping relations：

Alternatively, in the execute instruction that the processor 101 performs, the angle value β based on the inclination angle is determined The angle value θ of first acute angle includes：

Alternatively, in the execute instruction that the processor 101 performs, according to the detection parameters information of terminal, in institute Detection presentation screen picture in demonstration graph picture is stated, including：

Alternatively, in the execute instruction that the processor 101 performs, being detected in the demonstration graph picture all has The object of quadrangular configuration, including：

Alternatively, in the execute instruction that the processor 101 performs, according to including at least one of following information Determine described image detection range：The mobile message in the demonstration graph picture of the presentation screen picture, the demonstration graph Demonstrator's hand position information, the default regional extent information of the demonstration terminal operation as in.

It should be understood by those skilled in the art that, embodiments of the invention can be provided as method, system or computer program Product.Therefore, the present invention can use the reality in terms of complete hardware embodiment, complete software embodiment or combination software and hardware Apply the form of example.Moreover, the present invention can use the computer for wherein including computer usable program code in one or more The computer program production that usable storage medium is implemented on (including but is not limited to magnetic disk storage, CD-ROM, optical memory etc.) The form of product.

The present invention is the flow with reference to method, apparatus (system) and computer program product according to embodiments of the present invention Figure and/or block diagram describe.It should be understood that can be by every first-class in computer program instructions implementation process figure and/or block diagram Journey and/or the flow in square frame and flow chart and/or block diagram and/or the combination of square frame.These computer programs can be provided The processors of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices is instructed to produce A raw machine so that produced by the instruction of computer or the computing device of other programmable data processing devices for real The device for the function of being specified in present one flow of flow chart or one square frame of multiple flows and/or block diagram or multiple square frames.

These computer program instructions, which may be alternatively stored in, can guide computer or other programmable data processing devices with spy Determine in the computer-readable memory that mode works so that the instruction being stored in the computer-readable memory, which produces, to be included referring to Make the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one square frame of block diagram or The function of being specified in multiple square frames.

These computer program instructions can be also loaded into computer or other programmable data processing devices so that counted Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented processing, so as in computer or The instruction performed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one The step of function of being specified in individual square frame or multiple square frames.

Although preferred embodiments of the present invention have been described, but those skilled in the art once know basic creation Property concept, then can make other change and modification to these embodiments.So appended claims be intended to be construed to include it is excellent Select embodiment and fall into having altered and changing for the scope of the invention.

Obviously, those skilled in the art can carry out the essence of various changes and modification without departing from the present invention to the present invention God and scope.So, if these modifications and variations of the present invention belong to the scope of the claims in the present invention and its equivalent technologies Within, then the present invention is also intended to comprising including these changes and modification.

Claims

1. a kind of remote control method, it is characterised in that methods described includes：

According to the detection parameters information of terminal, presentation screen picture, the presentation screen picture are detected in the demonstration graph picture For the screen-picture of terminal described in the demonstration graph picture, including, being detected in the demonstration graph picture all has quadrangle wheel Wide object, in all objects with quadrangular configuration detected, finds out with terminal quadrangular configuration The presentation screen picture, the terminal quadrangular configuration and the detection parameters information matches of the terminal；

According to the parameter information of the presentation screen picture and the screen parameter information of the terminal, determine that the presentation screen is drawn Mapping relations between pixel on face and the pixel on the display screen of the terminal；

2. the method as described in claim 1, it is characterised in that the operation information includes operational motion information and the first operation Positional information, the first operating position information include the first pixel point set operated on the presentation screen picture Information；

Operation information according to user to the presentation screen picture, and the mapping relations, manipulate the terminal, including：

According to the information of the first pixel point set, and the mapping relations, it is corresponding to obtain the first pixel point set The information of the second pixel point set on the display screen of the terminal；

3. the method as described in claim 1, it is characterised in that the operation information includes the first operating position information, described First operating position information includes the information of the first pixel point set operated on the presentation screen picture；

4. the method as described in any one of claims 1 to 3, it is characterised in that the parameter information bag of the presentation screen picture Include at least one in following information：The width value of presentation screen picture, the height value of the presentation screen picture and described Coordinate value of the reference image vegetarian refreshments in the demonstration graph picture on presentation screen picture, the screen parameter information include the end The screen resolution at end；

According to the parameter information of the presentation screen picture and the screen parameter information of the terminal, determine that the presentation screen is drawn Mapping relations between pixel on face and the pixel on the display screen of the terminal, including：

According on the width value of the presentation screen picture, the height value of the presentation screen picture, the presentation screen picture Coordinate value and the terminal of the reference image vegetarian refreshments in the demonstration graph picture screen resolution, determine the demonstration screen The mapping relations between the pixel on pixel and the display screen on curtain picture.

5. method as claimed in claim 4, it is characterised in that the picture on the presentation screen picture is determined according to below equation The mapping relations between pixel on vegetarian refreshments and the display screen of the terminal：

Wherein, (X, Y) is the coordinate value of the pixel on the presentation screen picture, (Gx, Gy) be with (X, Y) corresponding to it is described The coordinate value of pixel on display screen, TrX are the width value of the display screen, and TrY is the height of the display screen Value, CW are the width value of the presentation screen picture, and CH is the height value of the presentation screen picture, and (PtX, PtY) is described Coordinate value of the reference image vegetarian refreshments in the demonstration graph picture, the coordinate that the reference image vegetarian refreshments corresponds on the display screen are former Point.

6. the method as described in any one of claims 1 to 3, it is characterised in that the screen parameter information of the terminal includes institute Laying state information, the screen resolution of the terminal of terminal are stated, the parameter information of the presentation screen picture includes demonstration The width value of screen-picture, the presentation screen picture height value and the presentation screen picture on reference image vegetarian refreshments exist Coordinate value in the demonstration graph picture；

According to the width value of the presentation screen picture, the height value of the presentation screen picture, the reference image vegetarian refreshments in institute Coordinate value, the screen resolution of the terminal and the laying state information of the terminal in demonstration graph picture are stated, it is determined that described The mapping relations between the pixel on pixel and the display screen on presentation screen picture.

7. method as claimed in claim 6, it is characterised in that the laying state information includes the laying state of the terminal With respect to the inclination angle of reference state, wherein, the inclination angle is that the laying state of the terminal is relative on the demonstration graph picture The angle that reference state rotates in the counterclockwise direction；When the laying state of the terminal is the reference state, the display Direction where the top margin of screen is horizontal direction, and the direction where the side of the display screen is vertical direction；According to Lower formula determines the mapping between the pixel on the display screen of the pixel and the terminal on the presentation screen picture Relation：

<mrow> <mi>G</mi> <mi>x</mi> <mo>=</mo> <mfrac> <mrow> <mi>C</mi> <mi>W</mi> <msqrt> <mrow> <msup> <mrow> <mo>(</mo> <mi>X</mi> <mo>-</mo> <mi>P</mi> <mi>t</mi> <mi>X</mi> <mo>)</mo> </mrow> <mn>2</mn> </msup> <msup> <mrow> <mo>(</mo> <mi>Y</mi> <mo>-</mo> <mi>P</mi> <mi>t</mi> <mi>Y</mi> <mo>)</mo> </mrow> <mn>2</mn> </msup> </mrow> </msqrt> <mi>c</mi> <mi>o</mi> <mi>s</mi> <mi>&theta;</mi> </mrow> <mrow> <mi>T</mi> <mi>r</mi> <mi>X</mi> </mrow> </mfrac> <mo>;</mo> </mrow>

<mrow> <mi>G</mi> <mi>y</mi> <mo>=</mo> <mfrac> <mrow> <mi>C</mi> <mi>H</mi> <msqrt> <mrow> <msup> <mrow> <mo>(</mo> <mi>X</mi> <mo>-</mo> <mi>P</mi> <mi>t</mi> <mi>X</mi> <mo>)</mo> </mrow> <mn>2</mn> </msup> <msup> <mrow> <mo>(</mo> <mi>Y</mi> <mo>-</mo> <mi>P</mi> <mi>t</mi> <mi>Y</mi> <mo>)</mo> </mrow> <mn>2</mn> </msup> </mrow> </msqrt> <mi>s</mi> <mi>i</mi> <mi>n</mi> <mi>&theta;</mi> </mrow> <mrow> <mi>T</mi> <mi>r</mi> <mi>Y</mi> </mrow> </mfrac> <mo>;</mo> </mrow>

Wherein, (X, Y) is the coordinate value of the pixel on the presentation screen picture, (Gx, Gy) be with (X, Y) corresponding to it is described The coordinate value of pixel on display screen, the width value of display screen described in TrX, TrY are the height value of the display screen, CW is the width value of the presentation screen picture, and CH is the height value of the presentation screen picture, and (PtX, PtY) is the reference Coordinate value of the pixel in the demonstration graph picture, the origin of coordinates that the reference image vegetarian refreshments corresponds on the display screen, θ The angle value determined for the angle value β based on the inclination angle, θ is between pixel (X, Y) and reference image vegetarian refreshments (PtX, PtY) Line and the display screen top margin where direction between the first acute angle angle value.

8. method as claimed in claim 7, it is characterised in that the angle value β based on the inclination angle determines that described first is sharp The angle value θ of angle angle, including：

When the angle value β at the inclination angle is 0, the angle value θ for determining first acute angle is α, wherein, α is pixel The angle value of second acute angle of line and horizontal direction between point (X, Y) and reference image vegetarian refreshments (PtX, PtY)；Or

When the angle value β at the inclination angle, which is more than the 0 and angle value β at the inclination angle, is less than π, first acute angle is determined The angle value θ of angle is that the angle value β at the inclination angle subtracts the difference after angle value α；Or

When the angle value β at the inclination angle is less than 3 pi/2 more than or equal to π and the inclination angle angle value β, it is determined that described The angle value θ of first acute angle is that the angle value β at the inclination angle is added with angle value α, subtracts the value after π again；Or

When the angle value β at the inclination angle is less than 2 π more than or equal to the angle value β of 3 pi/2s and the inclination angle, institute is determined The angle value θ for stating the first acute angle is that the angle value β at the inclination angle subtracts angle value α, subtracts the value after π again.

9. the method as described in claim 1, it is characterised in that being detected in the demonstration graph picture all has quadrangular configuration Object, including：

10. method as claimed in claim 9, it is characterised in that according to being determined including at least one of following information Image detection scope：The mobile message in the demonstration graph picture of the presentation screen picture, drilling in the demonstration graph picture Show demonstrator's hand position information of the terminal operation, default regional extent information.

11. a kind of remote operation controller, it is characterised in that the device includes：

Detection module, for the detection parameters information according to terminal, in the demonstration graph picture that the acquisition module is got Presentation screen picture is detected, the presentation screen picture is the screen-picture of terminal described in the demonstration graph picture, including, in institute State and all objects with quadrangular configuration are detected in demonstration graph picture, in all things with quadrangular configuration detected In body, the presentation screen picture with terminal quadrangular configuration, the terminal quadrangular configuration and the terminal are found out Detection parameters information matches；

Determining module, for the parameter information of the presentation screen picture detected according to the detection module and the terminal Screen parameter information, determine pixel on the display screen of the pixel and the terminal on the presentation screen picture it Between mapping relations；

Operational module, for the operation information according to user to the presentation screen picture, and the determining module determines The mapping relations, manipulate the terminal.

12. device as claimed in claim 11, it is characterised in that the operation information includes operational motion information and the first behaviour Make positional information, the first operating position information includes the first pixel point set operated on the presentation screen picture Information；

The operational module is specifically used for：

According to the information of the first pixel point set, and the mapping relations, it is corresponding to obtain the first pixel point set The information of the second pixel point set on the display screen of the terminal；According to the information of the second pixel point set and described Operational motion information, manipulate the terminal.

13. device as claimed in claim 11, it is characterised in that the operation information includes the first operating position information, institute Stating the first operating position information includes the information of the first pixel point set operated on the presentation screen picture；

The operational module is specifically used for：

According to the information of the first pixel point set, and the mapping relations, it is corresponding to obtain the first pixel point set The information of the second pixel point set on the display screen of the terminal；On the display screen of the terminal, by track icon It is moved to the opening position of the second pixel point set.

14. the device as described in any one of claim 11 to 13, it is characterised in that the parameter information of the presentation screen picture Including at least one in following information：The width value of presentation screen picture, the height value of the presentation screen picture and institute Coordinate value of the reference image vegetarian refreshments in the demonstration graph picture on presentation screen picture is stated, the screen parameter information includes described The screen resolution of terminal；

The determining module is specifically used for：

15. device as claimed in claim 14, it is characterised in that the determining module is specifically used for being determined according to below equation Mapping relations between pixel on the presentation screen picture and the pixel on the display screen of the terminal：

Wherein, (X, Y) is the coordinate value of the pixel on the presentation screen picture, (Gx, Gy) be with (X, Y) corresponding to it is described The coordinate value of pixel on display screen, the width value of display screen described in TrX, TrY are the height value of the display screen, CW is the width value of the presentation screen picture, and CH is the height value of the presentation screen picture, and (PtX, PtY) is the reference Coordinate value of the pixel in the demonstration graph picture, the origin of coordinates that the reference image vegetarian refreshments corresponds on the display screen.

16. the device as described in any one of claim 11 to 13, it is characterised in that the screen parameter information of the terminal includes The laying state information of the terminal, the screen resolution of the terminal, the parameter information of the presentation screen picture include drilling Show the width value of screen-picture, the reference image vegetarian refreshments on the height value and the presentation screen picture of the presentation screen picture Coordinate value in the demonstration graph picture；

The determining module is specifically used for：

17. device as claimed in claim 16, it is characterised in that the laying state information includes the placement shape of the terminal State with respect to reference state inclination angle, wherein, the inclination angle be on the demonstration graph picture, the laying state phase of the terminal The angle rotated in the counterclockwise direction to reference state；It is described aobvious when the laying state of the terminal is the reference state Direction where the top margin of display screen curtain is horizontal direction, and the direction where the side of the display screen is vertical direction；It is described Determining module is specifically used for the display screen that pixel and the terminal on the presentation screen picture are determined according to below equation The mapping relations between pixel on curtain：

18. device as claimed in claim 17, it is characterised in that the determining module is specifically used for：

19. device as claimed in claim 11, it is characterised in that the detection module is specifically used for：

20. device as claimed in claim 19, it is characterised in that the detection module, which is specifically used for basis, includes following information At least one of determine described image detection range：The mobile letter in the demonstration graph picture of the presentation screen picture Demonstrator's hand position information, the default regional extent information of the demonstration terminal operation in breath, the demonstration graph picture.