CN103179359A

CN103179359A - Method and device for controlling video terminal and video terminal

Info

Publication number: CN103179359A
Application number: CN 201110431720
Authority: CN
Inventors: 郭鹏; 程懿远; 王嘉; 鲍东山
Original assignee: Beijing Nufront Mobile Multimedia Technology Co Ltd
Current assignee: Beijing Nufront Mobile Multimedia Technology Co Ltd
Priority date: 2011-12-21
Filing date: 2011-12-21
Publication date: 2013-06-26

Abstract

The invention discloses a method and a device for controlling a video terminal and the video terminal. The method includes acquiring track images of moving objects according to a set acquisition frequency; determining position information of selected feature points on the moving objects in the acquired track images and obtaining track feature information of the selected feature points according to the determined position information; matching the obtained track feature information of the selected feature points with a preset track model of input information to obtain input information of the moving objects; and controlling the video terminal to execute corresponding operations according to operational orders corresponding to the input information. By means of the method and the device for controlling the video terminal, all kinds of information input by gestures can be recognized, control over the video terminal is achieved, the operation is flexible, and the generality is high.

Description

The method of control video terminal and device, video terminal

Technical field

The present invention relates to video control technology field, be specifically related to control the method for video terminal and device, video terminal.

Background technology

At present, the user will realize the control to TV, comprise the various operations controls such as the switching of channel and the switching of function, generally need to realize by remote controller, the user presses channel digit buttons different on remote controller or different function buttons, realizes controlling TV and shows different channels or switch to different functions.

Along with increasing of TV functions and increasing of channel quantity, the number of keys that need to arrange on remote controller is also constantly increasing, like this when having new function to increase, new button need to be set on remote controller, also increase the complexity of remote controller design and the cost of update, be unfavorable for the realization of man-machine interaction.

Therefore, control the mode of TV is paid close attention to gradually by gesture, this mode is without any need for medium, directly realize by space gesture, the implementation of gesture control at present is all generally the identification to the two-dimensional signal of user's gesture, need the user in the position of appointment, realize by the action input of standard, its interactive mode is single.

And existing gesture control mode, only can realize the identification of simple track, can not understand the semanteme of gesture, can not satisfy the demand for control of the multiple operating function of TV.For example: due to the semanteme that can not understand gesture, can only carry out the simple identification of gesture track, realize turning over before simple change platform, after turn over and change platform, can not realize that numeral input changes platform, for the channel switching controls that relates to a lot of numeral inputs, be irrealizable, therefore, its operation is controlled dumb, and versatility is poor.

Summary of the invention

In view of this, an object of the present invention is to solve the problem that in prior art, video terminal control interactive mode is single, operation control is dumb, versatility is poor.For there is a basic understanding some aspects to the embodiment that discloses, the below has provided simple summary.This summary part is not to comment general, neither determine the key/critical component or describe the protection range of these embodiment.Its sole purpose is to present some concepts with simple form, with this preamble as following detailed description.

The embodiment of the present invention provides a kind of method of controlling video terminal, comprising:

Gather the trace image of mobile object according to the frequency acquisition of setting;

Determine the selected positional information of characteristic point in each trace image that collects on described mobile object, obtain the track characteristic information of described selected characteristic point according to the positional information of determining;

The locus model of the track characteristic information of the described selected characteristic point that obtains with the input message of setting up in advance mated, obtain the input message of described mobile object;

The operational order corresponding according to described input message controlled video terminal and carried out corresponding operating.

In some optional embodiments, select the positional information of characteristic point in each trace image that collects on described definite described mobile object, obtain the track characteristic information of described selected characteristic point according to the positional information of determining; Specifically comprise:

Determine on described mobile object the selected three dimensional local information of characteristic point in each trace image that collects, the three dimensional local information of the described selected characteristic point that obtains is fitted to two-dimensional position information in selected plane;

According to the two-dimensional position information of described selected characteristic point corresponding to each trace image, determine that described selected characteristic point corresponding to the track characteristic value of each trace image, obtains the track characteristic information of described selected characteristic point.

In some optional embodiments, the described process that obtains the track characteristic information of described selected characteristic point specifically comprises:

For the selected two-dimensional position information of characteristic point in each trace image, obtain the two-dimensional position information of described selected characteristic point in current trace image and the fit line of two-dimensional position information in described selected plane in the previous frame trace image, determine the angle of the selected axis in described fit line and described selected plane, obtain described selected characteristic point corresponding to the track characteristic value of current trace image;

Form described track characteristic information by the selected characteristic point that obtains corresponding to the track characteristic value of each trace image.

In some optional embodiments, the locus model of the track characteristic information of the described selected characteristic point that will obtain and the input message of setting up in advance mates, and obtains the input message of described mobile object; Specifically comprise:

The selected characteristic point that comprises in track characteristic information according to the selected characteristic point that obtains determines corresponding to the track characteristic value of each trace image the variable condition that the described selected characteristic point on mobile object experiences;

The variable condition that each input message that comprises in locus model with described selected the characteristic point variable condition that experiences and each input message of setting up in advance experiences is mated;

When matching the locus model of Satisfying Matching Conditions, obtain the input message corresponding to locus model of described Satisfying Matching Conditions, obtain the input message of described mobile object.

In some optional embodiments, the described operational order corresponding according to described input message controlled video terminal and carried out corresponding operating, specifically comprises:

According to the described input message that obtains, the semantic model that comprises input message and operational order corresponding relation that inquiry is set up is in advance determined the operational order that described input message is corresponding, controls video terminal according to the operational order of determining and carries out corresponding operating.

In some optional embodiments, the track characteristic information of the described selected characteristic point that will obtain also comprises before mating with the locus model of setting up in advance:

The present mode that judgement arranges is mode control terminal or locus model training mode;

When being mode control terminal, carry out the step that the track characteristic information of the described selected characteristic point that will obtain and the locus model of setting up in advance mate;

When being the track training mode, the track characteristic information of selected characteristic point on the described mobile object that record obtains; According to the track characteristic information of the selected quantity that records, obtain the locus model of the input message of current training.

In some optional embodiments, the described track characteristic information of selected quantity according to record obtains the locus model of the input message of current training, specifically comprises:

According to the described selected characteristic point that comprises of each track characteristic information of the record track characteristic value corresponding to each trace image, determine the variable condition that the input message of the current training that the locus model of the input message of current training comprises experiences.

In some optional embodiments, after determining to select the positional information of characteristic point in each trace image that collects on described mobile object, also comprise:

The positional information of selected characteristic point in each trace image that collects of determining screened, determine the track characteristic information of selected characteristic point according to the selected positional information of characteristic point in the trace image that filters out; Wherein, the positional information that filters out satisfies following condition: the distance between the selected characteristic point in adjacent two trace images is greater than the first distance threshold of setting, less than the second distance threshold value of setting.

In some optional embodiments, after the trace image of described collection mobile object, also comprise:

The trace image that collects is carried out the data preliminary treatment, remove the interfere information in the trace image data.

It is a kind of for video terminal being operated the device of control that the embodiment of the present invention provides, and comprising:

Image capture module is used for the trace image according to the frequency acquisition collection mobile object of setting;

Data processing module is used for determining that on described mobile object, selected characteristic point in the positional information of each trace image that collects, obtains the track characteristic information of described selected characteristic point according to the positional information of determining;

Semantic identification module mates for the locus model of the track characteristic information of the described selected characteristic point that will obtain with the input message of setting up in advance, obtains the input message of described mobile object;

Command execution module is used for the operational order corresponding according to described input message and controls video terminal execution corresponding operating.

In some optional embodiments, described data processing module specifically is used for:

In some optional embodiments, described semantic identification module specifically is used for:

In some optional embodiments, described command execution module specifically is used for:

In some optional embodiments, said apparatus also comprises: mode decision module and model training module;

Described mode decision module, before mating for the track characteristic information of the selected characteristic point that will obtain and the locus model of setting up in advance, the present mode that judgement arranges is mode control terminal or locus model training mode; When being mode control terminal, the step that the locus model of notifying described semantic identification module to carry out the track characteristic information of the described selected characteristic point that will obtain and foundation in advance mates; When being the track training mode, notify described model training module;

Described model training module is used for recording the track characteristic information of selecting characteristic point on the described mobile object that obtains; According to the track characteristic information of the selected quantity that records, obtain the locus model of the input message of current training.

In some optional embodiments, described model training module specifically is used for:

In some optional embodiments, said apparatus also comprises:

The information sifting module, be used for determining that on described mobile object, selected characteristic point is after the positional information of each trace image that collects, the positional information of selected characteristic point in each trace image that collects of determining screened, and notify described data processing module to determine the track characteristic information of selected characteristic point according to the selected positional information of characteristic point in the trace image that filters out; Wherein, the positional information that filters out satisfies following condition: the distance between the selected characteristic point in adjacent two trace images is greater than the first distance threshold of setting, less than the second distance threshold value of setting.

In some optional embodiments, said apparatus also comprises:

Pretreatment module after being used for the trace image of described image capture module collection mobile object, is carried out the data preliminary treatment to the trace image that collects, and removes the interfere information in the trace image data.

The embodiment of the present invention provides a kind of video terminal, comprises the above-mentioned device that is used for video terminal is operated control.

The method of the above-mentioned control video terminal that the embodiment of the present invention provides and device, video terminal, by gathering the trace image of mobile object, determine the selected positional information of characteristic point in each trace image that collects on mobile object, obtain the track characteristic information of selected characteristic point according to the positional information of determining; The locus model of the track characteristic information of the selected characteristic point that then will obtain and the input message of setting up in advance mates, and determines the input message of mobile object; The operational order corresponding according to input message controlled video terminal and carried out corresponding operating.Thereby the track of having realized the movement-based object is realized the control to video terminal, does not need by media implementation such as remote controllers, therefore no longer is subject to the number of keys of remote controller, has satisfied video terminal and has controlled the increasing demand of function.And can realize semantic analysis by the track of understanding mobile object, and be not only the identification of simple instruction, therefore can identify various possible semantic meaning representations and input information, make the interactive information of user and video terminal more diversified, interactive mode is also more diversified, make the user can select any possible gesture input to control, and no longer be subject to the restriction of button and also no longer be subject to the restriction that can only identify simple information, be that man-machine interaction is more simple, also be convenient to user's operation.The method operation is controlled flexibly, and highly versatile is applicable to the various possible control operation demand of various video terminals.

For above-mentioned and relevant purpose, one or more embodiment comprise the feature that the back will describe in detail and particularly point out in the claims.Below explanation and accompanying drawing describe some illustrative aspects in detail, and its indication is only some modes in the utilizable variety of way of principle of each embodiment.Other benefit and novel features will consider by reference to the accompanying drawings and become obviously along with following detailed description, and the disclosed embodiments are to comprise being equal to of all these aspects and they.

Figure of description

Fig. 1 is the flow chart of video terminal control method in the embodiment of the present invention one;

Fig. 2 is a kind of optional particular flow sheet of video terminal control method in the embodiment of the present invention two;

Fig. 3 be in the embodiment of the present invention two binocular vision apart from the three-dimensional view of power sensor model;

Fig. 4 be in the embodiment of the present invention two binocular vision apart from the vertical view of power sensor model;

Fig. 5 is HMM model schematic diagram in the embodiment of the present invention two;

Fig. 6 is the schematic diagram of semantic model in the embodiment of the present invention two;

Fig. 7 is a kind of optional particular flow sheet of video terminal control method in the embodiment of the present invention three;

Fig. 8 is the structural representation of video terminal control device in the embodiment of the present invention.

Embodiment

The following description and drawings illustrate specific embodiments of the present invention fully, to enable those skilled in the art to put into practice them.Other embodiments can comprise structure, logic, electric, process and other change.Embodiment only represents possible variation.Unless explicitly call for, otherwise independent assembly and function are optional, and the order of operation can change.The part of some embodiments and feature can be included in or replace part and the feature of other embodiments.The scope of embodiment of the present invention comprises the gamut of claims, and all obtainable equivalents of claims.In this article, these embodiments of the present invention can be represented with term " invention " individually or always, this is only for convenient, surpasses the invention of and if in fact disclose, and is not that the scope that will automatically limit this application is any single invention or inventive concept.

Control the implementation operation and control dumb in order to solve in prior art gesture, the problem that versatility is poor, the embodiment of the present invention provides a kind of video terminal control method, the semanteme of the track representatives such as gesture that can Recognition and Acquisition arrive, thereby can identify various gesture, can only identify the problem of simple gesture when having avoided gesture identification in the prior art.The below has provided several optional realization flows in some optional embodiments.

Embodiment one

The embodiment of the present invention one provides a kind of video terminal control method, by gathering the trace image of mobile object, determine the track characteristic information of characteristic point selected on mobile object, thereby know corresponding input message, and then operational order corresponding to acquisition input message, deciphering and identification to the mobile object track have been realized, definite various information of inputting, and correspondence goes out operational order, thereby can satisfy the input control of various possible operational orders, its flow process comprises the steps: as shown in Figure 1

S11: the trace image that gathers mobile object according to the frequency acquisition of setting.

When capturing the mobile object of input instruction, gather the trace image of mobile object according to the frequency acquisition of setting.For example, when the user passes through the gesture input message, mobile object is user's hand, when having captured the user and made a gesture input message, gather the motion track information of user's hand according to the frequency acquisition of setting, specifically can utilize the information extraction hand shapes such as the colour of skin, parallax, gather the trace image of a series of successive frame according to the hand shape of extracting, get the trace information of shape in one's hands.

Can adopt the harvesters such as camera, infrared sensor, radar to realize to the collection of mobile object trace image.For example: the user need to control video terminal and carry out the channel switching, the numeral of the channel that can will be transferred to by gesture input, the Chinese character information of the channel that also can will be transferred to by gesture input, namely as long as this operation can be carried out in the semanteme that the user that video terminal can be identified expresses, general video terminal can be identified the various information relevant to the operation that will carry out.For example to be transferred to the channel 1 at 1 place of central authorities, can input the various information such as " CCTV1 ", " central authorities one ".

S12: determine the selected positional information of characteristic point in each trace image that collects on mobile object, obtain the track characteristic information of selected characteristic point on mobile object according to the positional information of determining.

This step need to select some to be used for characterizing the selected characteristic point of moving-object trace on mobile object, for example: when the user passes through the gesture input message, the characteristic point of selecting can be the center of gravity of user's hand, this moment is for each trace image, first according to the hand shape of the information extraction user hands such as the colour of skin, aberration, determine the positional information of center of gravity in trace image of user's hand according to hand shape.Again for example: when the user passed through the gesture input message, the characteristic point of selection can be also center or the fingertip location of user's hand.

After determining the selected positional information of characteristic point in trace image, according to the selected positional information of characteristic point in each trace image, can obtain characterizing the track characteristic information of the movement locus of selecting characteristic point.Wherein track characteristic information can be to select characteristic point corresponding to the track characteristic value of each trace image, can determine according to the selected positional information of characteristic point in trace image.

Preferably, after determining to select the positional information of characteristic point in each trace image that collects on mobile object, also comprise: the positional information of selected characteristic point in each trace image that collects of determining screened, determine the track characteristic information of selected characteristic point according to the selected positional information of characteristic point in the trace image that filters out; Wherein, the positional information that filters out satisfies following condition: the distance between the selected characteristic point in adjacent two trace images is greater than the first distance threshold of setting, preferred, the distance between the selected characteristic point in adjacent two trace images is less than the second distance threshold value of setting.

S13: the locus model of the track characteristic information of the selected characteristic point that will obtain and the input message of setting up in advance mates, and obtains the input message of mobile object.

According to the track characteristic value of the selected characteristic point that comprises in the selected characteristic point track characteristic information that obtains corresponding to each trace image, determine the variable condition that the selected characteristic point on mobile object experiences; The variable condition that each input message that comprises in locus model with selected the characteristic point variable condition that experiences and each input message of setting up in advance experiences is mated; When matching the locus model of Satisfying Matching Conditions, obtain the input message corresponding to locus model of Satisfying Matching Conditions, obtain the input message of mobile object.

This step realizes the user is identified by the information of the mobile objects such as gesture input, resolves its semanteme, such as parsing the information such as character that the user inputs by gesture or character combination.

When carrying out input message identification, need to use the locus model of setting up in advance, for example: for each possible input message, determine by a large amount of training datas the locus model that it is corresponding in advance, the variable condition that experiences of the input message of the current training that comprises of locus model wherein, wherein variable condition can characterize by a series of track characteristic value.When according to the selected characteristic point that the collects track characteristic value corresponding to each trace image, after having determined variable condition that selected characteristic point experiences, just can mate variable condition that selected characteristic point experiences whether with certain locus model in the variable condition of the input message that comprises be complementary.For example: the difference that characterizes track characteristic value corresponding in each track characteristic value of the variable condition experience and locus model when each that collects satisfies when imposing a condition, when the difference that for example characterizes track characteristic value corresponding in each track characteristic value of the variable condition experience and locus model when each that collects is not more than the threshold value of setting, think to mate.

S14: the operational order corresponding according to the input message that obtains controlled video terminal and carried out corresponding operating.

After the input message that obtains user's input, further can determine the operational order that input message is corresponding, control video terminal according to the operational order of determining and carry out corresponding operating.

Can set up in advance the corresponding relation of input message and operational order, and be stored as semantic model, after obtaining input message, the semantic model that comprises input message and operational order corresponding relation that inquiry is set up in advance, determine the operational order that input message is corresponding, control video terminal according to the operational order of determining and carry out corresponding operating.Such as: input message " CCTV1 ", " central authorities one " etc., its corresponding operational order is channel 1 this channel that switches to the place, CCTV, controls video terminal and carries out the operation that switches to channel 1.

Embodiment two

The video terminal control method that the embodiment of the present invention two provides, gather the three dimensional local information of selected characteristic point on mobile object, be converted into again two-dimensional position information, to realize the accurate collection of multi-faceted input message, make the control of video terminal more flexible, more convenient operation, the method flow process comprises the steps: as shown in Figure 2

S21: when capturing the mobile object of input message, gather the trace image of mobile object according to the frequency acquisition of setting.

Specifically referring to step S11.

In the present embodiment take the binocular camera shooting head as example, the binocular camera shooting head can obtain depth of view information, so just can remove other interfere informations, and can collect the three dimensional local informations such as three dimensional space coordinate of mobile object by depth of view information, the position of user's input does not need to face the collection direction of binocular camera shooting head like this, thereby can user friendlyly operate.

S22: the trace image that collects is carried out the data preliminary treatment, remove the interfere information in the trace image data.

This step is optional step, and main being responsible for processed the data such as trace image that collect, and comprises removing interfere information, removal noise etc.

The mode that different semantic analysis algorithms, data are processed also has larger difference, in following step take binocular camera shooting head and semantic RM as HMM as the example explanation.

S23: determine the selected three dimensional local information of characteristic point in each trace image that collects on mobile object.

Generally need to select a selected characteristic point to characterize the motion track of mobile object, therefore, need to determine the selected three dimensional local information of characteristic point in each trace image that collects on mobile object.Selected characteristic point can be the characteristic point of center of gravity, center or other appointment, the below is that center of gravity is as example take selected characteristic point, need first to determine the center of gravity of mobile object, as long as the shape of mobile object does not produce larger variation, the position of centre of gravity of object is exactly certain, select center of gravity as the sign point of mobile object track, general effect is reasonable.

After collecting a frame track image, wherein the computing formula of mobile object center of gravity is as follows:

Figure 201110431720X100002DEST_PATH_IMAGE002

，

Wherein,

Figure 201110431720X100002DEST_PATH_IMAGE006

Wherein,

Figure 201110431720X100002DEST_PATH_IMAGE008

,

Figure 201110431720X100002DEST_PATH_IMAGE010

Barycentric coodinates for mobile object;

Figure 201110431720X100002DEST_PATH_IMAGE012

Be (p+q) rank two-dimensional geometry square,

Figure 201110431720X100002DEST_PATH_IMAGE014

Be 0 rank square,

Figure 201110431720X100002DEST_PATH_IMAGE016

Be the first moment of x direction,

Figure 201110431720X100002DEST_PATH_IMAGE018

First moment for the y direction;

X, y are the coordinate in trace image, f(x, y) be the color value that in trace image, coordinate points (x, y) is located.

Therefore, before the barycentric coodinates of calculating mobile object, the actual coordinate of each point on the mobile object in the trace image that generally can first determine to collect is such as three-dimensional location coordinates, when using binocular camera shooting head acquisition trajectories image, it is the world coordinates that the binocular camera shooting head obtains.Be illustrated in figure 3 as the three-dimensional view of binocular vision model, Fig. 4 is the vertical view of Fig. 3.

According to the principle of as shown in Figure 3 and Figure 4 binocular vision range sensor measuring distance, two parallel placements of camera that focal length is f, the distance between the optical axis of two cameras is T.Two rectangles in Fig. 3 represent respectively the imaging plane of two cameras,

Figure 201110431720X100002DEST_PATH_IMAGE020

With

Figure 201110431720X100002DEST_PATH_IMAGE022

Be the focus of two cameras for any point P in scene shown in Figure 3, the imaging point on the imaging plane of two cameras is respectively

Figure 201110431720X100002DEST_PATH_IMAGE024

With

Figure 201110431720X100002DEST_PATH_IMAGE026

, the imager coordinate (image coordinate) of these two imaging points on imaging plane is

Figure 201110431720X100002DEST_PATH_IMAGE028

With

, parallax is

(vertical view shown in Figure 4).

Suppose the focus of left camera shown in Figure 3

Be initial point,

The straight line at place is X-axis, and the optical axis of left camera is Z axis, is Y-axis perpendicular to the direction of XZ axle, and the position coordinates of P point in this reference frame satisfies following formula:

，

，

Can calculate the three dimensional local information of P point in coordinate system, i.e. actual coordinate (X, Y, Z) by above-mentioned formula.Therefore, also can adopt this formula to calculate its actual coordinate in take the focal length of a camera as the reference frame of initial point for selected characteristic point, obtain the three dimensional local information of selected characteristic point.

In Fig. 3 ( ) be the intersection point of optical axis and imaging plane.

S24: the three dimensional local information of the selected characteristic point that will obtain fits to the two-dimensional position information in selected plane.

Due to the track of mobile object generally in a two dimensional surface, and the HMM model training in two dimensional surface discrimination than higher in three dimensions.So be the coordinate transformation of three-dimensional preferably coordinate in two dimensional surface.

Still to select characteristic point as center of gravity as example, known the three dimensional local information of center of gravity by above-mentioned steps S23, be its coordinate (X in world coordinate system, Y, Z), three-dimensional location coordinates with the center of gravity in the trace image that does not collect in the same time is fitted in a selected plane so, for example can carry out match with least square method, and three-dimensional coordinate just can be converted into the two-dimensional coordinate in certain plane that is fitted to like this.

If the general equation on plane is:

If treat that the three-dimensional coordinate of match point (as selected characteristic point-center of gravity) comprising:

Wherein,

The selected three-dimensional coordinate of characteristic point in i trace image of expression.

If the three-dimensional coordinate of the match point that remains satisfies the general equation on above-mentioned plane, think that these treat match point at same plane, otherwise think that these treat that match point is not at same plane.At this moment need these points are carried out match to obtain a best fit plane.The below carries out fit Plane take least square method and describes as example, can obtain coordinate z according to the general equation on above-mentioned plane and establish an equation under satisfying:

Wherein, if order

Can obtain new plane equation is:

Will be with treating match point

The above-mentioned plane equation of the Fitting Calculation makes the represented S value of following formula reach minimum:

Wherein, it is minimum making S, should satisfy:

Be:

Separate above-mentioned system of linear equations, obtain

Three parameters have so just obtained by least square fitting plane equation out, and the plane that this equation represents is selected plane.Plane equation has been arranged, also needed not projecting on selected plane at the tracing point on plane equation.

The coordinate of supposing the some P of projection to be treated is

, the method for concrete projection can be some P perpendicular to the linear equation on plane, got linear equation according to the general equation on above-mentioned plane to be:

Solving equations:

Namely can obtain the coordinate of its subpoint on selected plane, so far just can obtain being fitted to the tracing point of the selected characteristic point on same plane, then pass through to select x axle and y axle in this selected plane, three-dimensional coordinate just is converted into two-dimensional coordinate.

By above-mentioned Coordinate Conversion process, three-dimensional coordinate is converted into the two-dimensional coordinate of selecting in the plane, has realized the match of three dimensional local information to two-dimensional position information.

S25: according to the two-dimensional position information of selected characteristic point corresponding to each trace image, determine that selected characteristic point corresponding to the track characteristic value of each trace image, obtains the track characteristic information of selected characteristic point.

For the selected two-dimensional position information of characteristic point in each trace image, obtain the two-dimensional position information of described selected characteristic point in current trace image and the two-dimensional position information in the previous frame trace image fit line in selected plane, determine the angle of the selected axis in this fit line and selected plane, obtain selected characteristic point corresponding to the track characteristic value of current trace image; Form track characteristic information by the selected characteristic point that obtains corresponding to the track characteristic value of each trace image.

Above-mentioned be with the angle of the selected axis in fit line and selected plane as the track characteristic value of selected characteristic point, in actual applications, it is also conceivable that other parameters such as the length of selecting fit line, curvature are as the track characteristic value.

Preferably, due in the actual acquisition process, may there be certain deviation in the position of the selected characteristic point of determining and the ideal position of selected characteristic point, simultaneously, due to the trace image that obtains all take pixel as unit, angular relationship between points is not very accurate, so need to screen the positional information of the selected characteristic point (as center of gravity) that obtains.

Supposed to select the selected characteristic point in a trace image positional information C1, whether select for the positional information C2 of the selected characteristic point in the next frame trace image, can weigh by following formula:

, namely the position of the selected characteristic point that characterizes of alternative positional information and the distance between the position that characterizes of the positional information of selected selected characteristic point greater than the first distance threshold of setting.This be because two positional informations of the selected characteristic point chosen from the words close to too, can cause the deviation range of the track characteristic value that subsequent calculations obtains to enlarge, do not judge requirement and do not satisfy, make result of determination can not react really the variable condition of selected characteristic point reality.

Preferred, in order to improve the variable condition of the selected characteristic point reality of better reaction, the distance between the position that the positional information of the position that the selected characteristic point that alternative positional information characterizes can be set and selected selected characteristic point characterizes is less than the second distance threshold value of setting; Namely

By determining that selected characteristic point corresponding to the track characteristic value of each trace image, can obtain the track characteristic information of selected characteristic point.The track characteristic information that is to say selected characteristic point comprises that selected characteristic point is corresponding to the track characteristic value of each trace image at place.Wherein when screening positional information, in fact also be equivalent to screen trace image when above-mentioned, the trace image that collects is accepted or rejected, change so that the track characteristic value that obtains of subsequent calculations can reflect the virtual condition of selected characteristic point better.

Wherein, selected characteristic point can the positional information in trace image be determined according to it corresponding to the track characteristic value of each trace image, according to the selected positional information of characteristic point in the current trace image that will calculate and its positional information in the previous frame trace image, can obtain fit line between these two positional informations and the angular relationship of selected axis, with the angular relationship that obtains as the track characteristic value.

The main of HMM algorithm is exactly to determine the relation of point between point, here be to get angular relationship between the tracing point of selected characteristic point as its track characteristic value, can certainly select other relations between points, when selecting angular relationship, suppose that the selected positional information of characteristic point in current trace image is

, the positional information in the previous frame trace image is , the angular relationship between these two positional informations of so selected characteristic value is:

This angular relationship has characterized fit line between these two positional informations of selected characteristic point and the angular relationship of selected axis.

when selection angular relationship between points characterizes the track characteristic value, owing to may have shake in input process, therefore, when the distance between two positions is too near, the angular deviation that may occur calculating is very large, for example, when drawing straight line, due to shake, may make the angle of fit line of the very near point of two distances relatively with selected reference axis deviation more than 45, distance between the position that the positional information of the position that therefore limits the selected characteristic point that alternative positional information characterizes and selected selected characteristic point characterizes is greater than the first distance threshold of setting, the error of avoiding this shake to bring.

Above-mentioned steps S23-step S25 has realized the selected positional information of characteristic point in each trace image that collects on definite mobile object, obtains the process of the track characteristic information of selected characteristic point according to the positional information of determining.

S26: according to the track characteristic value of the selected characteristic point that comprises in the selected characteristic point track characteristic information that obtains corresponding to each trace image, determine the variable condition that the selected characteristic point on mobile object experiences.

Above-mentionedly obtain selected characteristic point and can realize by a data processor corresponding to the operation of the track characteristic value of each trace image, after data processor obtains the track characteristic value, can provide an input message corresponding to semantic identifier judgement identification.For example: the user has inputted character " central authorities one ", the character of inputting according to the trace information identification of the input character that obtains so, the last channel that is jumped to again 1 place of central authorities by the command executer fill order.Optional recognition methods has the method for SVMs (Support Vector Machine, SVM), neural net, the identification of HMM isotype.

For example: the track characteristic value of the selected characteristic point corresponding according to each frame track image of determining, can obtain a state change process as shown in Figure 5, wherein from initial condition S through

intermediate change state

1,2 ..., N, arrive state of termination F.Wherein

intermediate change state

1,2 ..., N can think a kind of hidden state, each hidden state correspondence a stroke of a character or character.Character can be formed by one or several stroke, input message can be formed by one or several character.

S27: will select the variable condition that each input message of comprising in the locus model of the variable condition that characteristic point experiences and each input message of setting up in advance experiences and mate.

In the HMM mode as example, identify the information of user's input, generally need first input message to be carried out a large amount of training of respective action, obtain the HMM model of the corresponding specific action of input message, i.e. the locus model of each input message.For example: can first build each input message, such as letter, after the HMM initial model of numeral or Chinese character, then it be trained in a large number.Concrete training process can be with reference to the BW algorithm in HMM.Corresponding initial model has been arranged, and the track characteristic value that so data acquisition unit is obtained in the initial model that substitution defines, obtains the probability P of track characteristic value under this model according to the forward direction algorithm in HMM, gets maximum probability as the result of last identification.

For example: a HMM locus model that obtains can be as shown in Figure 5.In this HMM model, S is initial condition, and F is state of termination, 1,2 ..., N be this model distinctive and successively the experience the intermediate change state, also claim hidden state.So, the hidden state in this HMM model is to comprise the character of letter, numeral or Chinese character etc. or certain unicursal in character, can form this by stroke and comprise the character of letter, numeral or Chinese character.

Certainly the HMM model also has aobvious state, corresponding one or several the aobvious state of each hidden state, and one or several aobvious state can obtain a hidden state in other words.The simplest Chinese character " one " for example, hidden state is a horizontal line, its aobvious state is the angular relationship between the tracing point of selected characteristic point, namely determined by the track characteristic value of the selected characteristic point of determining, because the user generally can not write a straight horizontal line fully, so aobvious state generally has-10 degree to the angle of inclination of+10 degree.

when the variable condition that each input message that comprises in the locus model that will select the characteristic point variable condition that experiences and each input message of setting up in advance experiences is mated, variable condition corresponding in the variable condition that selected characteristic point experiences and locus model does not require in full accord yet, but Satisfying Matching Conditions gets final product, when for example in the variable condition that experiences of selected characteristic point and locus model, corresponding variable condition characterizes by the track characteristic value, the threshold value of setting can think to mate as long as the difference of the angular relationship that its track characteristic value reflects is not more than.

S28: when matching the locus model of Satisfying Matching Conditions, obtain the input message corresponding to locus model of Satisfying Matching Conditions, obtain the input message of mobile object.

After the locus model that finds Satisfying Matching Conditions, namely can identify the information of user's input, if the information of the user who identifies input is a plurality of characters, also need so each character group synthetic is formed word after identifying each character, form by this input message that word representative of consumer finally will be expressed.

Above-mentioned steps S26-step S28 has realized that the locus model of the track characteristic information of the selected characteristic point that will obtain and the input message of setting up in advance mates, and obtains the process of the input message of mobile object.

S29: determine operational order corresponding to input message that obtain, control video terminal according to the operational order of determining and carry out corresponding operating.

According to the input message that obtains, the input message that the semantic model that inquiry is set up in advance comprises and the corresponding relation of operational order are determined the operational order that input message is corresponding, control video terminal according to the operational order of determining and carry out corresponding operating.For example can be carried out by command executer the operational order of required response.For example: when definite input message is " CCTV1 " or " central authorities one ", carry out the operation of the channel 1 that switches to 1 place of central authorities.

Due to the user in the process of written information, inevitably can produce some interfere informations, such as: in the process of the Chinese word of user writing " central authorities ten ", particularly " ten " are if for connecting pen, the track that forms may be identified as " central P " or " central authorities ten " two, actual conditions and the user's semanteme to be expressed controlled according to video terminal so should be easy to it is judged to be " central authorities ten " but not " central P ".Therefore, the actual conditions of using according to the video terminal that comprises TV, the semantic model that can pre-defined user may express.The schematic diagram that is a predefined semantic model shown in Figure 5 for example, its mid band 1 is central authorities one, possible input message comprises numeral " 1 ", Chinese " one ", word " one ", " central authorities one ", " central authorities 1 ", " CCTV1 ", " CCTV one " etc., therefore set up in advance these input messages and channel 1(central authorities one) corresponding relation, thereby the operational order of setting up these input messages and will carrying out---switch to channel 1(central authorities one) corresponding relation.Wherein, for channel 2(central authorities two) and channel 3 be also similarly, will not enumerate.

in above-mentioned flow process shown in Figure 2, when obtaining the track characteristic information of mobile object, by first obtaining the three dimensional local information of selected characteristic point on mobile object, and then fit to again two-dimensional position information, thereby make the user by the gesture input message time, can face controlled video terminal, it is the direction that mobile object can be not orthogonal to harvester, namely regardless of the position relationship of mobile object and harvester, all can get track characteristic information accurately, thereby improved the accuracy of track characteristic information identification, also reduced the status requirement when user profile is inputted, be convenient for users to operate.

Embodiment three

The video terminal control method that the embodiment of the present invention three provides is carried out different operating processes according to the present mode that arranges, and such as terminal control or locus model training, its flow process comprises the steps: as shown in Figure 7

S31: when capturing the mobile object of input message, gather the trace image of mobile object according to the frequency acquisition of setting.

Referring to step S11 and step S21, repeat no more herein.

S32: determine the selected positional information of characteristic point in each trace image that collects on mobile object, obtain the track characteristic information of the selected characteristic point of mobile object according to the positional information of determining.

Referring to step S11 and step S23-step S25, repeat no more herein.Need to prove, also can carry out in advance process of data preprocessing as described in step S22 before carrying out this step.

S33: the present mode that judgement arranges is mode control terminal or locus model training mode.If mode control terminal, execution in step S34; If the locus model training mode, execution in step S36.

By operating personnel, present mode being set is mode control terminal or locus model training mode, generally before dispatching from the factory, equipment is set to the locus model training mode, be convenient to the designer and train locus model, be set to mode control terminal after dispatching from the factory, user-friendly.If the locus model training mode, the track characteristic information that repeatedly obtains of follow-up basis is trained, and extracts general character, obtains the locus model of the input message of current training, the coupling when being used for the subsequent user input message.If mode control terminal shows that the importer is final user, can realize semantic judgement and deciphering according to the information that obtains locus model confirmation user input, so that the operation that should carry out definite according to the semanteme of user's input information.

S34: the locus model of the track characteristic information of the selected characteristic point that will obtain and the input message of setting up in advance mates, and obtains the input message of mobile object.

Referring to step S13 and step S26-step S28, repeat no more herein.

S35: the operational order corresponding according to input message controlled video terminal and carried out corresponding operating.

Referring to step S14 and step S29, repeat no more herein.

S36: the track characteristic information of selected characteristic point on the mobile object that record obtains.

If the locus model training mode, the track characteristic information recording/ with Chosen Point on mobile object gets off, and carries out the training of locus model by a large amount of track characteristic information of record.

S37: according to the track characteristic information of the selected quantity that records, obtain the locus model of the input message of current training.

Specifically can be according to each track characteristic information selected characteristic point that comprise of record track characteristic value corresponding to each trace image, determine the variable condition that the input message of the current training that the locus model of the input message of current training comprises experiences.The specific implementation process can be referring to the description in step S26-step S28.

Based on same inventive concept, it is a kind of for video terminal being operated the device of control that the embodiment of the present invention also provides, this device can be arranged in video terminal, the structure of this device comprises as shown in Figure 8: image capture module 10, data processing module 20, semantic identification module 30 and command execution module 40.

Image capture module 10 is used for the trace image according to the frequency acquisition collection mobile object of setting.

Data processing module 20 is used for determining that on mobile object, selected characteristic point in the positional information of each trace image that collects, obtains the track characteristic information of selected characteristic point according to the positional information of determining.

Semantic identification module 30 mates for the locus model of the track characteristic information of the selected characteristic point that will obtain with the input message of setting up in advance, obtains the input message of mobile object.

Command execution module 40 is used for the operational order corresponding according to input message and controls video terminal execution corresponding operating.

Preferably, above-mentioned data processing module 20, concrete be used for determining on mobile object selected characteristic point at the three dimensional local information of each trace image that collects, the three dimensional local information of the selected characteristic point that obtains is fitted to two-dimensional position information in selected plane; According to the two-dimensional position information of selected characteristic point corresponding to each trace image, determine that selected characteristic point corresponding to the track characteristic value of each trace image, obtains the track characteristic information of selected characteristic point.

Preferably, above-mentioned data processing module 20, concrete being used for for the two-dimensional position information of selected characteristic point at each trace image, obtain the selected two-dimensional position information of characteristic point in current trace image and the fit line of two-dimensional position information in described selected plane in the previous frame trace image, determine the angle of the selected axis in fit line and selected plane, obtain selected characteristic point corresponding to the track characteristic value of current trace image; Form track characteristic information by the selected characteristic point that obtains corresponding to the track characteristic value of each trace image.

Preferably, above-mentioned semantic identification module 30, the selected characteristic point that the concrete track characteristic information that is used for according to the selected characteristic point that obtains comprises determines corresponding to the track characteristic value of each trace image the variable condition that the selected characteristic point on mobile object experiences; The variable condition that each input message that comprises in locus model with selected the characteristic point variable condition that experiences and each input message of setting up in advance experiences is mated; When matching the locus model of Satisfying Matching Conditions, obtain the input message corresponding to locus model of Satisfying Matching Conditions, obtain the input message of mobile object.

Preferably, mentioned order Executive Module 40, concrete being used for according to the input message that obtains, the semantic model that comprises input message and operational order corresponding relation that inquiry is set up in advance, determine operational order corresponding to input message that obtain, control video terminal according to the operational order of determining and carry out corresponding operating.

Preferably, above-mentioned device for video terminal being operated control also comprises: mode decision module 50 and model training module 60.

Mode decision module 50, before mating for the track characteristic information of the selected characteristic point that will obtain and the locus model of setting up in advance, the present mode that judgement arranges is mode control terminal or locus model training mode; When being mode control terminal, the step that the locus model of notifying semantic identification module 30 to carry out the track characteristic information of the selected characteristic point that will obtain and foundation in advance mates; When being the track training mode, notification model training module 60.

Model training module 60 is used for recording the track characteristic information of selecting characteristic point on the mobile object that obtains; According to the track characteristic information of the selected quantity that records, obtain the locus model of the input message of current training.

Preferably, above-mentioned model training module 60, the selected characteristic point that concrete each track characteristic information that is used for according to record comprises is corresponding to the track characteristic value of each trace image, determines the variable condition that the input message of the current training that the locus model of the input message of current training comprises experiences.

Preferably, above-mentioned device for video terminal being operated control also comprises:

Information sifting module 70, be used for determining that on mobile object, selected characteristic point is after the positional information of each trace image that collects, the positional information of selected characteristic point in each trace image that collects of determining screened, and notification data processing module 20 is determined the track characteristic information of selected characteristic point according to the selected positional information of characteristic point in the trace image that filters out; Wherein, the positional information that filters out satisfies following condition: the distance between the selected characteristic point in adjacent two trace images is greater than the first distance threshold of setting, less than the second distance threshold value of setting.

Pretreatment module 80 after being used for the trace image of image capture module 10 collection mobile objects, is carried out the data preliminary treatment to the trace image that collects, and removes the interfere information in the trace image data.

In said apparatus, image capture module gathers the information such as trace image of user's input, data processing equipment is processed the track characteristic information that obtains selected characteristic point to the data such as trace image that collect, semantic recognition device identifies according to the information such as trace image of track characteristic information with user's input, is determined operation and the execution of this execution according to the input message that identifies by command execution module.Realize control to video terminals such as TVs based on the semantic analysis to the gesture of user input etc., can realize man-machine interaction without any need for medium, can identify the various information of user's input, no longer be subjected to the restriction in any medium and space, the interactive mode of user and TV is simple, is convenient to user's operation.This mode is applicable to various possible semantic meaning representations and input information, but the diversity of interactive mode and the diversity of identifying information have been realized, make the user can select any possible gesture input to control, and no longer be subject to the restriction of button and can only identify the restriction of simple information, make man-machine interaction become and be more prone to.

When especially adopting the binocular camera shooting head, can allow the user in any position that can collect by mode input messages such as gestures, and no longer require the user must face the camera input of video terminal, thereby be more convenient for the control of user's video terminal.

Unless otherwise specific statement, term such as processing, calculating, computing, determine etc. to refer to action and/or the process of one or more processing or computing system or similar devices, described action and/or process will be expressed as the register for the treatment of system or the data manipulation of the amount of the physics (as electronics) in memory.

Should be understood that the particular order of the step in disclosed process or the example that level is illustrative methods.Based on design preference, should be appreciated that, the particular order of the step in process or level can be in the situation that do not break away from protection range of the present disclosure and rearranged.Appended claim to a method has provided the key element of various steps with exemplary order, and is not to be limited to described particular order or level.

In above-mentioned detailed description, various features are combined in single embodiment together, to simplify the disclosure.This open method should be interpreted as reflecting such intention, that is, the embodiment of theme required for protection need to be than the more feature of the feature of clearly stating in each claim.On the contrary, as appending claims reflected, the present invention was in the state that lacks than whole features of disclosed single embodiment.Therefore, appending claims clearly is merged in detailed description hereby, and wherein every claim is alone as the independent preferred embodiment of the present invention.

Those skilled in the art it is also understood that various illustrative box, module and the algorithm steps in conjunction with the embodiment description of this paper all can be embodied as electronic hardware, computer software or its combination.For the interchangeability between hardware and software clearly is described, the above has all carried out usually describing around its function to various illustrative parts, frame, module, circuit and step.Be embodied as hardware or be embodied as software as for this function, depend on specific application and the design constraint that whole system is applied.Those skilled in the art can be for each application-specific, realizes described function in the mode of accommodation, still, thisly realizes that decision-making should not be construed as and deviates from protection range of the present disclosure.

The general processor that is used for the described function of execution the application can be microprocessor, and perhaps, this processor can be also processor, controller, microcontroller or the state machine of any routine.Processor also may be embodied as the combination of computing equipment, for example, and the combination of multi-microprocessor, perhaps any other this kind structure.

Software module or its combination that can directly be presented as hardware, be carried out by processor in conjunction with the step of the described method of the embodiment of this paper or algorithm.

For making any technical staff in this area realize or to use the present invention, the above is described disclosed embodiment.To those skilled in the art; The various alter modes of these embodiment are all apparent, and the General Principle of this paper definition also can be applicable to other embodiment on the basis that does not break away from spirit of the present disclosure and protection range.Therefore, the disclosure is not limited to the embodiment that this paper provides, but consistent with the widest scope of the disclosed principle of the application and novel features.

Realize for software, the technology of describing in the application can realize with the module (for example, process, function etc.) of carrying out the described function of the application.These software codes can be stored in memory cell and be carried out by processor.Memory cell can be realized in processor, also can realize outside processor, and under latter event, it is coupled to processor by correspondence via various means, and these are all well known in the art.

Description above comprises giving an example of one or more embodiment.Certainly, all possible combination of describing parts or method in order to describe above-described embodiment is impossible, but those of ordinary skills should be realized that, each embodiment can do further combinations and permutations.Therefore, the embodiment that describes herein is intended to contain all the such changes, modifications and variations in the protection range that falls into appended claims.In addition, " comprise " with regard to the term that uses in specification or claims, the mode that contains of this word is similar to term and " comprises ", just as " comprising, " in the claims as link word explain like that.In addition, using any one term " perhaps " in the specification of claims is to represent " non-exclusionism or ".

Claims

1. a method of controlling video terminal, is characterized in that, comprising:

2. the method for claim 1, is characterized in that, selectes the positional information of characteristic point in each trace image that collects on described definite described mobile object, obtains the track characteristic information of described selected characteristic point according to the positional information of determining; Specifically comprise:

3. method as claimed in claim 2, is characterized in that, the described process that obtains the track characteristic information of described selected characteristic point specifically comprises:

4. the method for claim 1, is characterized in that, the locus model of the track characteristic information of the described selected characteristic point that will obtain and the input message of setting up in advance mates, and obtains the input message of described mobile object; Specifically comprise:

5. the method for claim 1, is characterized in that, the described operational order corresponding according to described input message controlled video terminal and carried out corresponding operating, specifically comprises:

6. described method as arbitrary in claim 1-5, is characterized in that, the track characteristic information of the described selected characteristic point that will obtain also comprises before mating with the locus model of setting up in advance:

7. method as claimed in claim 6, is characterized in that, the described track characteristic information of selected quantity according to record obtains the locus model of the input message of current training, specifically comprises:

8. described method as arbitrary in claim 1-5, is characterized in that, after determining to select the positional information of characteristic point in each trace image that collects on described mobile object, also comprises:

9. described method as arbitrary in claim 1-5, is characterized in that, after the trace image of described collection mobile object, also comprises:

10. a device that is used for video terminal is operated control, is characterized in that, comprising:

11. device as claimed in claim 10 is characterized in that, described data processing module specifically is used for:

12. device as claimed in claim 11 is characterized in that, described data processing module specifically is used for:

13. device as claimed in claim 10 is characterized in that, described semantic identification module specifically is used for:

14. device as claimed in claim 10 is characterized in that, described command execution module specifically is used for:

15. described device as arbitrary in claim 10-14 is characterized in that, also comprises: mode decision module and model training module;

16. device as claimed in claim 15 is characterized in that, described model training module specifically is used for:

17. described device as arbitrary in claim 10-14 is characterized in that, also comprises:

18. described device as arbitrary in claim 10-14 is characterized in that, also comprises:

19. a video terminal is characterized in that, comprises as arbitrary in claim 10-18 described for video terminal being operated the device of control.