Summary of the invention
Based on this, it is provided that a kind of easy to operate image interaction method and system and the display device of correspondence.
A kind of image interaction method, comprises the steps:
Obtain, by controlling device, the image frame that display device shows, obtain gathering picture;
The control received on the described collection picture operation of described control device, and obtain described control operation
Positional information in described collection picture;
The positional information that described collection picture and described control operate is sent to described aobvious by described control device
Showing device;
The described collection picture of described display device reception described control device transmission and described control operate
Positional information, and the image frame of described collection picture and self display is carried out image recognition, obtain self
The mapping picture identical with described collection picture and positional information thereof in the image frame of display;
The positional information that described display device operates according to described collection picture and described control, described mapping
Picture and positional information thereof, obtain the image that the described position correspondence controlling operation shows in described display device
Position in picture;
Described display device is transplanted in image frame of its display described control operation and described control operates
Position correspondence position, and perform to operate corresponding preset function with described control.
A kind of image interactive system, including the control device for controlling and controlled display device, described
Control device to include: image collection module, display screen, control module, processing module and transmitter module,
Described image collection module, for obtaining the image frame that described display device shows, obtains gathering picture,
And shown by described display screen;
Described control module is used for responding user's input, is controlled operation on described collection picture;
Described processing module operates the positional information in described collection picture for obtaining described control;
Described transmitter module is for the described collection picture described image collection module obtained and described control
The positional information of operation is sent to described display device;
Described display device includes: display module, receiver module, identification module, parsing module and execution mould
Block,
Described display module is used for showing image frame;
Described collection picture and described control that described receiver module sends for receiving described control device are grasped
The positional information made;
Described identification module shows for the described collection picture received by receiver module and described display module
Image frame carries out image recognition, obtains in the image frame that described display module shows and described collection picture
Identical mapping picture and positional information thereof;
Described parsing module is for the positional information operated according to described collection picture and described control, described
Map picture and positional information thereof, obtain what the described position correspondence controlling operation showed at described display module
Position in image frame;
Described execution module is for being transplanted to described control operation in the image frame that described display module shows
The position correspondence position operated with described control, and perform to control, with described, the preset function that operation is corresponding.
A kind of display device, including: display module, receiver module, identification module, parsing module and execution
Module,
Described display module is used for showing image frame;
Collection picture that described receiver module sends for receiving control device and enterprising at described collection picture
Row controls the positional information of operation;
Described identification module shows for the described collection picture received by receiver module and described display module
Image frame carries out image recognition, obtains in the image frame that described display module shows and described collection picture
Identical mapping picture and positional information thereof;
Described parsing module is for the positional information operated according to described collection picture and described control, described
Map picture and positional information thereof, obtain what the described position correspondence controlling operation showed at described display module
Position in image frame;
Described execution module is for being transplanted to described control operation in the image frame that described display module shows
The position correspondence position operated with described control, and perform to control, with described, the preset function that operation is corresponding.
The display device of above-mentioned image interaction method and system and correspondence utilizes and controls device acquisition display device
Picture, and by image recognition, picture in being converted to display device in the operation controlled on device is entered
Row controls, and compares traditional remote control mode, and operation is directly perceived, convenient.
Detailed description of the invention
Hereinafter image interactive method and system relate to controlling device and display device, in order to make it easy to understand, first
First using mobile phone as controlling device, this programme thought is simply introduced by television set as a example by display device:
Mobile phone possesses image collection module (rearmounted) and touch screen.As it is shown in figure 1, television screen 100
For showing image frame, handset touch panel 120 for display from television screen 100 photographs to quilt
Clap picture 140.Black bars 121 and rhombus in Fig. 1 represent picture material.User touches handset touch panel
After A point on 120, image and A dot position information that touch screen 120 is shown by mobile phone are sent to TV
Machine.TV by image recognition find correspondence by clap picture 140, and be calculated A point correspondence at TV
Position a point in the picture of machine screen 100 display.Then television set performs similar click/touch TV
The function that on machine, at a point, picture is corresponding.
So just reach to realize the effect to TV images touch control, operation side by handset touch panel
Just and user experience is high.In other embodiments, control the touch screen on device and control can also be mouse
Other control modes such as operation.
Please refer to Fig. 2, it is the flow chart of steps of image interaction method of an embodiment, including walking as follows
Rapid:
Step S201, obtains, by controlling device, the image frame that display device shows, obtains gathering picture.
In one embodiment, described control device has photographic head, and it obtains display device by photographic head and shows
Some or all of image.Controlling device can be that mobile phone, notebook computer or panel computer etc. have bat
The electronic equipment of camera shooting function, in order to the formula of taking is preferred.Display device can be television set or other have image show
Show the electronic equipment of function.
Step S202, controls the control received on the described collection picture operation of device, and obtains controlling behaviour
Make the positional information in gathering picture.
Described control operation can be mouse action or have the touch controlled on the device behaviour of touch screen
Make.
Step S203, controls device and the positional information gathering picture and control operation is sent to display device.
In present embodiment, transmission mode is for being wirelessly transferred, such as wifi, bluetooth etc..
Step S204, what display device receiving control device sent gathers picture and the position of described control operation
Confidence ceases, and the image frame gathering picture and self display is carried out image recognition, obtains self display
Mapping picture identical with gathering picture in image frame and positional information thereof.
The picture of handset touch panel 120 display in described collection picture corresponding diagram 1, described mapping picture is corresponding
Shown in Fig. 1 by clap picture 140.The positional information mapping picture (by clapping picture 140) can be left with it
(x y) represents upper angular vertex coordinate.
Step S205, display device, according to gathering picture and the positional information of described control operation, maps picture
Face and positional information thereof, obtain controlling the corresponding position in the image frame that display device shows, position of operation
Put.
In one embodiment, display device first obtains the size gathering picture and the size mapping picture, further according to
The size gathering picture, the positional information controlling operation and the positional information of mapping picture and size, obtain
Control the corresponding position in the image frame that display device shows, position of operation.
Assume that the size gathering picture obtained is wide high w1,h1, the size mapping picture is wide high w2, h2。
Assuming to control the positional information of operation, i.e. in corresponding diagram 1, A point coordinates is that (m, n), that just can calculate
Obtain the coordinate (m', n') of a point in Fig. 1,
The representation of above-mentioned positional information can also be adopted in other ways, as by the distance angle on summit and length
Degree, or carry out positional representation by the mode of stress and strain model, will not enumerate here.
Step S206, described control operation is transplanted in image frame of its display and controls behaviour by display device
The position correspondence position made, and perform to control, with described, the preset function that operation is corresponding.
Will user in virtual being transplanted in display device of operation controlled on device, reach the effect of remote control
Really.
As a example by user's touch operation on the touch screen controlling device, touch control operation can resolve into three kinds
Behavior, all operations can be expressed by these three behavior permutation and combination.
These three behavior is: touches down, slide, lift.It is abstracted into data structure and expresses as follows:
Touch down event: the coordinate (m of the point comprising the event of touching down0,n0), if any referring to touch, then attach more
I(i > 0 altogether) individual other coordinate (m put already at touch condition1,n1)~(mi,ni)。
(mj,nj)∈(X,Y)(0≤j≤i)。
Slip event: comprise the coordinate (m of the point triggering slip event0,n0), if any referring to touch, the most attached more
I(i > 0 it is total on band) individual other coordinate (m put already at touch condition1,n1)~(mi,ni)。
(mj,nj)∈(X,Y)(0≤j≤i)。
Lift event: comprise the coordinate (m touching the point lifting start event0,n0), if any referring to touch, the most attached more
I(i > 0 it is total on band) individual other coordinate (m put already at touch condition1,n1)~(mi,ni)。
(mj,nj)∈(X,Y)(0≤j≤i)。
One event can only have a trigger point, if change the state of two points, this behavior simultaneously
Two continuous print correspondence events can be split as.
User clicks on the touchscreen and can be expressed as: touch down (m0,n0), lift (m0,n0) two behaviors.
User pulls on the touchscreen, frame choosing can be expressed as: touch down (m0,n0), mobile (m0,n0), mobile
(m0,n0) ... mobile (m0,n0), lift (m0,n0)。
The most double gesture referring to amplify of user can be expressed as: touches down (m0,n0), touch down
(m0,n0)(m1,n1), mobile (m0,n0)(m1,n1), mobile (m0,n0)(m1,n1) ... mobile (m0,n0)(m1,n1), lift
(m0,n0)(m1,n1), lift (m0,n0)。
The above behavior is all to control on the touch screen of device, and the data that these are operated pass through wireless network
Network is sent to display device, during the event that display device is changed into it on giant-screen, it is only necessary to event is related to
And the coordinate of point, by conversion formula described previously:
Simple conversion i.e. becomes the coordinate in display device.And the gesture identification of display device can be according to this
A little data change into corresponding gesture.
Above-mentioned image interactive method obtains the picture of display device by controlling device, and by image recognition,
Picture in being converted to display device in the operation controlled on device is controlled controlling, compares traditional distant
Prosecutor formula, not only operation is directly perceived, convenient, and Consumer's Experience sense is higher.
As it is shown on figure 3, the functional block diagram of the image interactive system 30 that it is an embodiment, including for controlling
The control device 300 of system and controlled display device 400.
Control device 300 to include: image collection module 301, display screen 302, control module 303, process
Module 304 and transmitter module 305.
Image collection module 301, for obtaining the image frame of display device 400 display, obtains gathering picture,
And shown by display screen 302.
In one embodiment, described image collection module 301 is photographic head, obtains described collection picture for shooting
Face.
Control module 303 is used for responding user's input, is controlled operation on described collection picture.
Described control operation can be mouse action or other operations, in an embodiment, described display screen 302
Being combined into touch screen with control module 303, controlling operation is then touch control operation on the touchscreen.
Processing module 304 is for obtaining the positional information controlling operation in gathering picture.
Transmitter module 305 gathers picture for image collection module 301 obtained and controls the position of operation
Confidence breath is sent to display device 400.
Display device 400 includes: display module 402, receiver module 403, identification module 404, parsing mould
Block 405 and execution module 406.
Display module 402 is used for showing image frame.
Receiver module 403 gathers picture for what receiving control device 300 sent and controls the position of operation
Information.
Identification module 404 shows for the collection picture received by receiver module 403 and display module 402
Image frame carries out image recognition, obtains in the image frame of display module 402 display identical with gathering picture
Mapping picture and positional information.
Parsing module 406 for according to gathering picture and controlling the positional information of operation, map picture and
Positional information, obtains the described position correspondence of operation that controls in the image frame of display module 402 display
Position.
In one embodiment, parsing module 406 gathers picture for obtaining and maps the size of picture, and according to
The size gathering picture, the positional information controlling operation, the positional information mapping picture and size, controlled
The corresponding position in the image frame of display module 402 display, position of system operation.
Assume that the size gathering picture is wide high w1,h1;The size mapping picture is wide high w2, h2;Map picture
With top left corner apex coordinate, (x y) represents in the position in face;The false positional information controlling operation, i.e. corresponding diagram 1
Middle A point coordinates be (m, n);The most just the coordinate (m', n') of a point in Fig. 1 can be calculated.
The representation of above-mentioned positional information can also be adopted in other ways, as by the distance angle on summit and length
Degree, or carry out positional representation by the mode of stress and strain model.
Perform module 406 for described control operation being transplanted in the image frame that display module 402 shows
The position correspondence position operated with control, and perform to control, with described, the preset function that operation is corresponding.
A lot of mode that is mapped with about various operations realizes, the mapping method of the touch screen operation as front enumerated,
Here repeat no more.
Above-mentioned image interactive system obtains the image frame of display device 400 display by controlling device 300,
Be converted to the image frame of display device 400 is controlled behaviour in the operation controlling device 300 by user
Make.Comparing traditional mode, not only operation is directly perceived, convenient, and Consumer's Experience sense is higher.
If above-mentioned control device obtains collection picture by the way of shooting, at the image of shooting display device
During picture, it is understood that there may be filming frequency and the inconsistent situation of display frequency, such as the display frequency of display device
Being 60 frames/s, filming frequency is 24 frames/s.(LCDs and plasma panel then do not have such
Problem)
In one embodiment, the some two field picture pictures overlap that can display device be shown/merge and store, with weight
The image frame of poststack carries out image recognition.Such as overlapping 3 two field picture pictures, due to continuous 3 two field picture pictures
Content gap is not too large, and the image frame that overlap goes out also is can be identified with image recognition algorithm.
Embodiment described above only have expressed the several embodiments of the present invention, and it describes more concrete and detailed,
But therefore can not be interpreted as the restriction to the scope of the claims of the present invention.It should be pointed out that, for this area
Those of ordinary skill for, without departing from the inventive concept of the premise, it is also possible to make some deformation and
Improving, these broadly fall into protection scope of the present invention.Therefore, the protection domain of patent of the present invention should be with appended
Claim is as the criterion.