CN103268589A - Somatosensory interaction method based on front-facing camera of mobile terminal - Google Patents

Abstract

The invention discloses a somatosensory interaction method based on a front-facing camera of a mobile terminal. The method includes the following steps: (1) acquiring a current scene image, namely a video sequence, through the front-facing camera, (2) judging whether the acquired current scene image is a first-frame image, and if yes, returning the process to carry out the step (1), (3) carrying out optical flow field analysis on image data of a current frame and image data of a previous frame to acquire optical flow variation between the two frames and coordinates of corresponding points, (4) selecting a coordinate set of the points with obvious optical flow variation between the two frames, and calculating the variation tendency and the variation amplitude of scenes between the frames as the tendency and the amplitude of translation, rotation and scaling correspondingly, and (5) carrying out the step (1) to the step (4) circularly, calculating tendency data and amplitude data of the translation, the rotation and the scaling, and providing the data to an application program. The somatosensory interaction method based on the front-facing camera of the mobile terminal converts the variation of the scenes into the motions of translation, rotation, scaling and the like, and is more universal.

Description

A kind of body sense exchange method based on the preposition camera of portable terminal

Technical field

The present invention relates to intelligent movable equipment application, particularly a kind of body sense exchange method based on the preposition camera of mobile phone.

Background technology

On mobile terminal devices such as mobile phone, with mobile phone alternately mainly by touch-screen and physical keyboard.More and more higher along with handset capability, application program is more and more abundanter, and the diversity of interactive mode is also had higher requirement.

The preposition camera of portable terminals such as mobile phone can obtain abundant scene information, but existing application to preposition camera is main still in purposes such as video calling, auto heterodyne, recognitions of face.

Utilize methods such as the detection of people's face, gestures detection to follow the trail of the variation of scene, though have higher accuracy, real-time and applicability are very poor, lose efficacy easily under complex scene.

Under general common scene, the video of preposition camera is carried out the detection and tracking of specific objective, have very high algorithm complex, verification and measurement ratio is also lower, can't satisfy the demand of real-time, interactive.

In most cases, do not need point-device detection and tracking precision, but real-time and applicability there is very high requirement, patent of the present invention has provided a kind of method and has addressed this problem, the variation of scene is converted to actions such as translation, rotation and convergent-divergent, realizes more general body sense interactive mode.

Summary of the invention

In view of problems of the prior art, the object of the invention is to provide a kind of body sense exchange method based on the preposition camera of mobile phone, may further comprise the steps:

(1) by preposition camera, obtains the current scene image, i.e. video sequence;

(2) judge whether the current scene image that obtains is first two field picture, if return execution in step (1);

(3) to the view data of present frame and former frame, carry out the optical flow field analysis, obtain light stream variation between two frames and the coordinate of corresponding point;

(4) choose the point coordinate set that two interframe light streams are changed significantly, calculate variation tendency and the amplitude of interframe scene, correspond to trend and the amplitude of translation, rotation and convergent-divergent;

(5) circulation execution in step (1)～(4), and simultaneously trend and the amplitude data of the translation, rotation and the convergent-divergent that calculate offered application program.

Preferably, between step (2) and (3), also comprise step: the scene image that gets access to is carried out down-sampling, obtain the view data of small in resolution.

Preferably, the Downsapling method of employing is the nearest neighbor pixels method, makes the gray-scale value of pixel after the conversion equal gray-scale value apart from its nearest input pixel.

Preferably, use the KLT algorithm to calculate optical flow field, obtain the corresponding point between consecutive frame.

Description of drawings

Fig. 1 illustration a kind of body sense exchange method process flow diagram based on the preposition camera of mobile phone of the embodiment of the invention;

Fig. 2 illustration corresponding point set graph of a relation between embodiment of the invention image.

Embodiment

For above-mentioned purpose of the present invention, feature and advantage are become apparent more, the present invention is further detailed explanation below in conjunction with the drawings and specific embodiments.

Fig. 1 illustration a kind of body sense exchange method process flow diagram based on the preposition camera of mobile phone of providing of the embodiment of the invention.

As shown in Figure 1, said method may further comprise the steps:

(1) by preposition camera, obtains the current scene image, i.e. video sequence;

(2) judge whether the current scene image that obtains is first two field picture, if return execution in step (1);

(3) scene image that gets access to is carried out down-sampling, obtain the view data of small in resolution;

(4) to the view data of present frame and former frame, carry out the optical flow field analysis, obtain light stream variation between two frames and the coordinate of corresponding point;

(5) choose the point coordinate set that two interframe light streams are changed significantly, calculate variation tendency and the amplitude of interframe scene, correspond to trend and the amplitude of translation, rotation and convergent-divergent;

(6) circulation execution in step (1)～(5), and simultaneously trend and the amplitude data of the translation, rotation and the convergent-divergent that calculate offered application program.

Pass through said method, use the optical flow tracking algorithm, obtain mean change trend and the amplitude of moving region in the scene, and be converted to actions such as translation, rotation and convergent-divergent, thereby realize a kind of real-time, general body sense interactive mode, application program can be utilized trend and the corresponding various processing operations of amplitude data of the translation, rotation and the convergent-divergent that obtain.

For more detailed and the present invention will be described best and the protection, below the illustrated method of Fig. 1 is carried out the more explanation of details, but it will be appreciated by those skilled in the art that it is not construed as limiting the invention.

In step shown in Figure 1 (3), the video data that gets access to is carried out down-sampling, obtain the image of small in resolution, purpose is to reduce the computational data amount.In the method, do not need to obtain very high-precision gesture, so not very high requirement of the quality of data, on the image of little resolution, still can effectively calculate, but can reduce computation complexity significantly.Therefore, one skilled in the art can appreciate that step (3) is the preferred steps that makes that the present invention more optimizes, is not steps necessary, and under the situation of not carrying out the down-sampling processing, the present invention still can realize, only is that computation complexity is higher.

Among the present invention, the Downsapling method of employing can be the nearest neighbor pixels method for example, makes the gray-scale value of pixel after the conversion equal gray-scale value apart from its nearest input pixel.

In step shown in Figure 1 (4), use the KLT algorithm to calculate optical flow field, obtain the corresponding point between consecutive frame.

In the KLT algorithm, following prerequisite hypothesis is arranged: (a) brightness constancy, (b) time continuous, (c) space unanimity.On a window W of image, all points (x, y) all to a direction translation (dx dy), thereby obtains (x ', y '), i.e. t ₀Constantly (x is y) at t ₁Constantly be (x ', y '), be converted into following formula is sought minimum value so seek the problem of corresponding point:

$\∈\left(d\right)=\∈(d_x,d_y)={\mathrm{\Σ}}_{x=u_x-w_x}^{u_x+w_x}{\mathrm{\Σ}}_{y=y_y-w_y}^{u_y+w_y}{(I(x,y)-J(x+d_x,y+d_y))}^2---\left(1\right)$

Use the KLT algorithm, obtain the corresponding point set of adjacent two interframe respectively, for { Pi}, { Pi ' }, the distribution on image is similar to shown in Figure 2.

In step (2) and (3), the corresponding point set by step (1) obtains calculates its central point

,

, wherein: try to achieve each unique point to the mean distance of central point Di}, { Di ' }, wherein:

$\mathrm{Di}=\mathrm{Pi}-{\stackrel{\→}{C}}_0---\left(4\right)$

${\mathrm{Di}}^{\′}={\mathrm{Pi}}^{\′}-{\stackrel{\→}{C}}_1---\left(5\right)$

Try to achieve each unique point to the angle { α of central point _i, { α _i', wherein:

${\mathrm{\α}}_i=\arctan (\mathrm{Pi}-{\stackrel{\→}{C}}_0)---\left(6\right)$

${{\mathrm{\α}}_i}^{\′}=\arctan ({\mathrm{Pi}}^{\′}-\stackrel{\→}{C_1})---\left(7\right)$

By above result calculated, can obtain the amplitude of translation, convergent-divergent, rotation respectively:

$D(t_0,t_1)={\stackrel{\→}{C}}_1-{\stackrel{\→}{C}}_0---\left(8\right)$

$Z(t_0,t_1)=\frac{\mathrm{\Σ}{D}_i}{\mathrm{\Σ}{D_i}^{\′}}---\left(9\right)$

$R(t_0,t_1)=\mathrm{\Σ}({\mathrm{\α}}_i^{\′}-{\mathrm{\α}}_i)/n---\left(10\right)$

It more than is the detailed description that the preferred embodiments of the present invention are carried out, but those of ordinary skill in the art is to be appreciated that, within the scope of the present invention, and guided by the spirit, various improvement, interpolation and replacement all are possible, for example use that the different programming language (as C, C++, Java etc.) of algorithm, use that can realize functional purpose of the same race is realized etc.These are all in the protection domain that claim of the present invention limits.

Claims (4)

1. body sense exchange method based on the preposition camera of mobile phone may further comprise the steps:

(1) by preposition camera, obtains the current scene image, i.e. video sequence;

(2) judge whether the current scene image that obtains is first two field picture, if return execution in step (1);

(3) to the view data of present frame and former frame, carry out the optical flow field analysis, obtain light stream variation between two frames and the coordinate of corresponding point;

(4) choose the point coordinate set that two interframe light streams are changed significantly, calculate variation tendency and the amplitude of interframe scene, correspond to trend and the amplitude of translation, rotation and convergent-divergent;

(5) circulation execution in step (1)～(4), and simultaneously trend and the amplitude data of the translation, rotation and the convergent-divergent that calculate offered application program.

2. the method for claim 1 is characterized in that also comprising step between step (2) and (3): the scene image that gets access to is carried out down-sampling, obtain the view data of small in resolution.

3. method as claimed in claim 2 is characterized in that, the Downsapling method of employing is the nearest neighbor pixels method, makes the gray-scale value of pixel after the conversion equal gray-scale value apart from its nearest input pixel.

4. as arbitrary described method among the claim 1-3, it is characterized in that, use the KLT algorithm to calculate optical flow field, obtain the corresponding point between consecutive frame.

Images