WO2013075466A1

WO2013075466A1 - Character input method, device and terminal based on image sensing module

Info

Publication number: WO2013075466A1
Application number: PCT/CN2012/075103
Authority: WO
Inventors: 辛静
Original assignee: 中兴通讯股份有限公司
Priority date: 2011-11-23
Filing date: 2012-05-04
Publication date: 2013-05-30
Also published as: CN102520790A

Abstract

Disclosed are a character input method, device and terminal based on an image sensing module. The method includes: acquiring motion trace information about an object fingertip; and querying a pre-trained character model library according to the motion trace information, and converting the motion trace information into a corresponding character. The character input method, device and terminal based on an image sensing module provided in the present invention can meet the personalized demand of a user for a handheld terminal, and at the same time can realize rapid and accurate input of characters.

Description

Character input method, device and terminal based on image sensing module

The present invention relates to the field of mobile terminals, and in particular, to a character input method, apparatus, and terminal based on an image sensing module. Background technique

At present, mobile communication is undergoing rapid development, and thus, the functional requirements for human-computer interaction functions of mobile terminals (such as mobile phones) are becoming higher and higher. As an indispensable communication tool in people's daily life and work, mobile phones are increasingly being upgraded and developed intelligently and ergonomically to meet the increasingly demanding individual needs of people.

Text editing and input technology has dramatically changed the way people communicate since its birth. With this technology, people can edit text messages, enter characters, and more. At present, there are many ways to input characters, but in view of the fact that many character input methods have more or less limitations or defects.

Initially, the input of characters (or text) is edited by means of a button. In the process of editing characters, it is necessary to use a button that does not stop with a thumb. With this method, the user needs to repeatedly repeat the button action, and the input efficiency is low, and Due to long-term button presses, even when it is serious, it may cause health problems similar to "threbitis", which greatly affects the health of users.

In addition, since most of the current handheld terminals are equipped with a resistive screen or a capacitive screen, when a character is input, writing is performed on the screen of the terminal by means of a finger or a stylus, by pressure or capacitive sensing between the finger or the stylus and the screen. The recognition of characters and the generation of characters, thereby inputting corresponding characters. However, the method requires the user's terminal to configure a resistive screen or a capacitive screen, and a mobile phone without a resistive screen or a capacitive screen cannot use the method for character input, so the method has certain limitations. In addition, some handheld terminals can control the sensor to sense various actions made by the user to achieve the purpose of character input. For example, when the user wears special gloves, the target tracking is performed by the sensor, and the user is detected. The character input is performed by the action; or the target tracking is performed in the case where the user uses the highlighter of the special device, and the action made by the user is detected to achieve the same purpose of character input. However, when these methods are used to implement character input, it is necessary to make these portable terminals additionally configure corresponding hardware, which makes the cost higher, and requires additional equipment when performing character input, thereby making the user's use extremely inconvenient. .

To this end, how to provide a personalized, input-driven character input method has gradually received widespread attention. Summary of the invention

In view of this, the main object of the present invention is to provide a character input method, device and terminal based on an image sensing module, which can meet the personalized requirements of the user for the handheld terminal, and at the same time realize the fast and accurate character. Input.

In order to achieve the object of the present invention, the present invention adopts the following technical solutions:

A character input method based on an image sensing module, comprising:

Obtaining the running track information of the fingertip of the object;

The pre-trained character model library is queried according to the running track information, and the running track information is converted into corresponding characters.

Preferably, before the acquiring the running track information of the fingertip, the character input method based on the image sensing module further includes:

Acquiring object gesture contour image information by using an image sensing module;

Querying the pre-trained gesture contour model library according to the object gesture contour image information, and acquiring a matching gesture contour model;

Tracking of the fingertip of the object is performed according to the gesture contour model.

Preferably, the querying the pre-trained gesture according to the object gesture contour image information Before contouring the model library, the method also includes:

Binarizing the acquired object gesture contour image information; and/or,

Performing image sharpening processing on the collected object gesture contour image information; and/or performing image smoothing processing on the collected object gesture contour image information.

Preferably, the tracking of the fingertip of the object according to the gesture contour model is: mapping a rectangular region of the fingertip coarsely positioned of the reference gesture contour model to the extracted object gesture contour image, and obtaining the finger of the object gesture contour image The rectangular area where the pointed position is located;

Equally dividing the contours of the fingers in the rectangular area, and calculating the edge bending rate of each segment of the finger contour;

The finger contour segment with the largest edge curvature is obtained as the fingertip of the object.

Preferably, the running track information of the fingertip of the acquiring object is:

Tracking prediction of the fingertip of the object according to the Kalman filter;

Obtaining the starting coordinate information of the fingertip of the object from the video frame collected by the image sensing module, and then collecting the real-time coordinate information of the fingertip of the object at least once every at least one frame;

Calculating a tangential angle ^ of the real-time coordinate information and the previous coordinate information, and acquiring trajectory information of the fingertip of the object according to the change of the tangential angle, wherein the tangential angle ^ is calculated as: CJ ¹ ) refers to the coordinates of the fingertip of the object at the last ti time.

Information, the ( ) refers to the real-time coordinate information of the fingertip of the object at the time t in real time.

Preferably, the querying the pre-trained character model library according to the running track information, and converting the running track information into corresponding characters:

Querying the pre-trained character model library according to the running track information, and matching the running track information with all character models;

The character model with the largest likelihood value is obtained as the target character model, wherein the likelihood value of each character model is calculated by using the Verbit algorithm;

The running track information is converted into corresponding characters according to the target character model. A character input device based on an image sensing module, the device comprising:

An image sensing module, configured to collect a video frame including a fingertip of the object;

The object fingertip running track information acquiring module is configured to acquire the running track information of the fingertip of the object through the video frame collected by the image sensing module;

The character conversion module is configured to query the pre-trained character model library according to the running track information, and convert the running track information into corresponding characters.

Preferably, the character input module based on the image sensing module further includes:

An object gesture contour image information collecting module, configured to collect object hand contour image information by using an image sensing module;

a gesture contour model obtaining module, configured to query a pre-trained gesture contour model library according to the object gesture contour image information, and obtain a matching gesture contour model, so that the object fingertip running track information acquiring module performs the object according to the gesture contour model Tracking of the fingertips, obtaining the running track information of the fingertips of the object.

The image processing module is configured to process the object gesture contour image information collected by the object gesture contour image information collection module as follows:

Performing binarization processing on the collected object gesture contour image information; and/or performing image sharpening processing on the collected object gesture contour image information; and/or, performing mapping on the collected object gesture contour image information Like smoothing.

Preferably, the object fingertip running track information acquiring module performs tracking of the fingertip of the object according to the gesture contour model:

Mapping a rectangular area of the fingertip coarsely positioned of the reference gesture contour model to the extracted object hand contour image, and obtaining a rectangular area where the fingertip position of the object gesture contour image is located;

Equally dividing the contours of the fingers in the rectangular area, and calculating the edge bending rate of each segment of the finger contour; The finger contour segment with the largest edge curvature is obtained as the fingertip of the object. Preferably, the information about the running track of the fingertip of the object obtained by the object fingertip trajectory information acquiring module by the image sensing module is:

Calculating a tangential angle ^ of the real-time coordinate information and the previous coordinate information, and acquiring running trajectory information of the fingertip of the object according to the change of the tangential angle, wherein the tangential angle ^ is calculated as: 0 = 4^; The ¹ , }^ ¹ ) refers to the coordinate information of the fingertip of the object at the last time ti, and the ( ) refers to the real-time coordinate information of the fingertip of the object at the time t in real time.

Preferably, the character conversion module queries the pre-trained character model library according to the running track information, so as to convert the running track information into corresponding characters:

The running track information is converted into corresponding characters according to the target character model.

A terminal comprising the image sensor module based character input device, the device comprising:

It can be seen from the above technical solution of the present invention that the image sensing module provided by the present invention is based on The character input method, device and terminal of the block rely on the image sensing module (such as the camera device) provided by the terminal to collect images, extract the gesture contour, match the hand contour with the existing gesture contour model, and recognize the gesture model to the object. The fingertip is coarsely positioned, and then the fingertip of the finger is accurately positioned according to the bending rate of the contour of the finger. Predict the approximate position of the fingertip at the next moment, capture the movement trajectory of the fingertip, calculate the tangential angle at different moments, and accumulate the tangential angle changes over a period of time to obtain the trajectory of the fingertip of the object during the time period. The obtained object fingertip trajectory is matched with the pre-stored character model library to generate corresponding characters. The image input module-based character input method, device and terminal provided by the invention can meet the personalized requirements of the user for the handheld terminal, and at the same time realize fast and accurate input of characters. DRAWINGS

The drawings are intended to provide a further understanding of the present invention, and are intended to be a part of the invention. In the drawing:

1 is a schematic flow chart of a character input method based on an image sensing module according to an embodiment of the present invention;

2 is a detailed flow chart of a character input method based on an image sensing module according to a preferred embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a character input device based on an image sensing module according to an embodiment of the present invention. detailed description

The basic idea of the present invention is: acquiring the running track information of the fingertip of the object; querying the pre-trained character model library according to the running track information, and converting the running track information into corresponding characters.

In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer, It is to be understood that the present invention will be further described in detail below with reference to the drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

As shown in FIG. 1, a character input method based on an image sensing module according to an embodiment of the present invention includes the following steps:

S101. Acquire, by using an image sensing module, information about a running track of an object fingertip;

In this step, the image sensing module is configured to collect fingertip screen information of a user's finger, and extract user's fingertip running track information according to the collected fingertip screen information, for example, the image sensing module. A camera device that is configured for both regular mobile phones and smartphones.

In the specific implementation, in this step, the following specific steps are included:

a. acquiring an object gesture contour image through the image sensing module, and performing joint segmentation and binarization processing on the object gesture contour image in time domain and spatial domain;

b. matching the gesture contour with the stored gesture contour reference model by panning, zooming, rotating, etc., selecting the gesture contour reference model with the largest similarity, and mapping the fingertip region of the reference model to the fingertip region of the object model;

C. Calculate the bending rate of the finger contour at equal intervals in the fingertip region, and the bending position is the precise position of the fingertip;

d. Calculate the tangent angle at different moments, accumulate the tangent angle changes over a period of time, and obtain the running track information of the fingertip of the object.

S102. Query a pre-trained character model library according to the running track information, so as to convert the running track information into corresponding characters.

In this step, according to the collected fingertip running track information of the user, matching with the pre-established character model library, when the corresponding character model is found, the running track information is translated into corresponding character information, This provides a basis for character input.

In a preferred embodiment, before performing all the steps, the method further includes the following steps: S1001: Acquire an object gesture contour image information, such as the object, by using an image sensing module. Gesture contour image information includes but is not limited to the following: Only the index finger is straight, other fingers are clenched; only the middle finger is straight, other fingers are clenched; the thumb and forefinger are straight at the same time, other fingers are clenched.

S1002: Query a pre-trained gesture contour model library according to the object gesture contour image information, and obtain a matching gesture contour model.

S1003. Track the object fingertip according to the gesture contour model.

In a preferred embodiment, before querying the pre-trained gesture contour model library according to the object gesture contour image information to obtain a matching gesture contour model, the method further includes:

(1) performing binarization processing on the acquired object gesture contour image information; and/or,

(2) performing image sharpening processing on the collected object gesture contour image information; and/or, (3) performing image smoothing processing on the collected object gesture contour image information.

Taking the binarization processing, the image sharpening processing, and the image smoothing processing of the collected object gesture contour image information at the same time as an example, since the grayscale image has only the luminance information of the spatial sampling point, it can be represented by a numerical value. For color images, the content is richer, but the presentation is more complicated. However, in the present invention, it is not necessary to maintain the color information of the image, so that the object gesture contour image information of the color image can be binarized and converted into a grayscale image, which reduces complexity and reduces memory resources. At the same time, in order to highlight the edge of the gesture outline, image sharpening processing can be performed on the object gesture contour image information, and the present invention uses the Roberts gradient sharpening method to enhance the image edge. In addition to this, in order to reduce noise, it is also possible to perform image smoothing processing on the subject gesture profile image information.

In a preferred embodiment, in the step S1003, the step of tracking the fingertip of the object according to the gesture contour model includes:

S10031: Map a rectangular area of the fingertip coarsely positioned of the reference gesture contour model to the extracted object gesture contour image, and obtain a rectangular area where the fingertip position of the target gesture contour image is located.

S10032, equally dividing a finger contour in the rectangular area, and calculating each segment The edge bend rate of the finger contour segment.

S 10033. Obtain a finger contour segment with the largest edge bending rate as the object fingertip.

In the step S101, the method for acquiring the running track information of the fingertip of the object by the image sensing module specifically includes the following steps:

S 1011: Tracking prediction of an object fingertip according to a Kalman filter.

51012. Acquire a starting coordinate information of an object fingertip from a video frame collected by the image sensing module, and then collect real-time coordinate information of the fingertip of the object once every at least one frame.

51013. Calculate a tangential angle of the real-time coordinate information and the previous coordinate information, and obtain operation trajectory information of the fingertip of the object according to the change of the tangential angle, where the calculation formula of the tangential angle ø is as follows: The (e - ¹ ) refers to the coordinates of the fingertip of the object at the last ti time.

Information, the ( ί, refers to the real-time coordinate information of the fingertip of the object at real time t.

In the step S102, the method for querying the pre-trained character model library according to the running track information to convert the running track information into corresponding characters includes:

51021. Query a pre-trained character model library according to the running track information, and match the running track information with all character models.

51022. The character model with the largest likelihood value is the target character model, wherein the likelihood value of each character model is calculated by the Verbit algorithm.

51023. Convert the running track information into a corresponding word payment according to the target character model.

As shown in FIG. 2, FIG. 2 is a schematic flowchart of a character input method based on an image sensing module according to a preferred embodiment of the present invention. The process specifically includes the following steps:

S301, gesture model selection, and fingertip coarse positioning;

The selection of the gesture model directly affects the recognition effect. The present invention selects a common gesture model as a reference model, for example, but not limited to the following: Only the index finger is straight, and other fingers are clenched; Only the middle finger is straight, the other fingers are clenched; the thumb and forefinger are straight at the same time, and other fingers are clenched. For different gesture models, the initial position of the fingertip is predefined. For example, in the case where only the index finger is straight, the initial position of the fingertip is defined at the top of the index finger, and the fingertip position is included with a rectangular frame of a certain size, which is the area where the subsequent fingertip is accurately positioned.

5302, gesture contour extraction;

Video images are typically analyzed using inter-frame or intra-frame information. However, due to the presence of noise in the scene, a single video segmentation method does not accurately approximate the edge of the object. Therefore, to accurately extract the gesture outline, the spatial information such as the color and brightness of the object should be combined for video segmentation. For example, the embodiment of the present invention adopts a time-space joint segmentation method, and comprehensively utilizes inter-frame motion information in a time domain and spatial skin color and luminance information, and simultaneously performs time and space segmentation methods to extract an accurate gesture edge. Through the spatial domain segmentation, the initial segmentation region with accurate semantics is obtained, and the motion region of the image is obtained by time domain segmentation, and the discontinuous edge is connected to obtain the gesture contour.

5303, image preprocessing;

The grayscale image has only the luminance information of the spatial sampling point, and thus can be represented by a numerical value. For color images, the content is richer, but the representation is more complicated. However, in the embodiment of the invention, it is not necessary to preserve the color information of the image. Therefore, the color image needs to be binarized and converted into a grayscale image, which reduces complexity and reduces memory resources. To highlight the edges of the hand and sharpen the image, the present invention uses the Roberts gradient sharpening method to enhance the edges of the image. To reduce noise, the binary image can be smoothed.

S304. The gesture contour is matched with the gesture contour model.

The extracted gesture contour is matched with the gesture contour model, and the matching process is: performing panning matching on the extracted gesture contour image with each reference gesture contour model, calculating a matching value, and selecting a gesture contour model with the largest matching value as the target to be identified. . Then, the rectangular area of the fingertip coarsely positioned by the gesture contour model is mapped onto the extracted gesture contour, and the rectangular area where the fingertip position of the gesture contour is located is obtained. In order to improve the matching degree, the extracted gesture outline can be scaled S305, precise positioning of the fingertips;

The contours of the fingers in the rectangular area are equally spaced, and the bending rate of each curve is calculated separately, and the maximum curve of the bending rate is taken as the precise position of the fingertip.

S306, moving target prediction tracking

In video images, the time interval of adjacent frames is small and can be considered as uniform motion, so the change of motion state can be described as a dynamic linear system. In view of real-time requirements, the embodiment of the present invention selects a Kalman filter to track the position of the fingertip. The two main stages of the Kalman filter are prediction and updating. The equations are:

S (n) = AS (n - \)

D{n) = AD{n - \)A

5307, the starting point of the moving target is obtained;

In the specific implementation process of the embodiment of the present invention, the user generally has a medium-quality frame in the middle of the writing process, and the starting point has many meaningless frames, which affects the target detection. Therefore, according to the empirical value, the fourth frame is used as the starting point. The starting point of the track.

5308, the target of the moving target is obtained;

In the embodiment of the present invention, when the moving target (i.e., the fingertip) does not have any motion within 2 seconds, the character input operation is considered to be ended.

5309, acquisition of motion trajectory features;

After obtaining the coordinates of two adjacent frames, calculate the tangent angle, set the coordinates of the fingertip point at time t-1 to be -^ό- ¹ , and the coordinate of the fingertip point at time t is (, there is a tangent angle ^f^). In order to simplify the calculation, the present invention quantifies the tangent angle, for example, every 15 is quantized into one direction, that is, a uniform quantization method using 24 feature vectors. The change of the tangent angle at different times constitutes a finger Sharp trajectory.

S310, dynamic track recognition.

The obtained trajectory is matched with the trained character model, and the model with the greatest likelihood is selected as the target character model. Among them, the probability problem is involved. The present invention uses the Verbit algorithm to find the likelihood value of each model, and determines the maximum likelihood as the final target. The running track information is finally converted into corresponding characters according to the target character model.

As shown in FIG. 3, an embodiment of the present invention further provides a character input device based on an image sensing module, where the device includes:

The image sensing module 10 is configured to collect a video frame including a fingertip of the object;

The object fingertip trajectory information acquiring module 20 is configured to acquire the trajectory information of the fingertip of the object by using the video frame acquired by the image sensing module. In the embodiment of the present invention, the object fingertip trajectory information acquiring module 20 acquires the object. The trajectory information of the fingertip includes the following specific steps: acquiring an object gesture contour image through the image sensing module, performing time domain and spatial domain joint segmentation and binarization processing on the object gesture contour image; and performing gesture by panning, zooming, rotating, etc. The contour is matched with the stored gesture contour reference model, and the gesture contour reference model with the highest similarity is selected, and the fingertip region of the reference model is mapped to the fingertip region of the object model; the finger contour bending rate is calculated at equal intervals in the fingertip region, The maximum bending rate is used as the precise position of the fingertip; the tangential angle at different times is calculated, and the tangential angle changes over a period of time are accumulated to obtain the running track information of the fingertip of the object.

The character conversion module 30 is configured to query the pre-trained character model library according to the running track information to convert the running track information into corresponding characters.

In a preferred embodiment, the character input device based on the image sensing module further includes: an object gesture contour image information collecting module 40, configured to collect object gesture contour image information through the image sensing module;

a gesture contour model obtaining module 50, configured to query according to the object gesture contour image information The pre-trained gesture contour model library obtains a matching gesture contour model, so that the object fingertip running track information acquiring module can track the object fingertip according to the gesture contour model, and acquire the running track information of the object fingertip.

In a preferred embodiment, the image sensor module-based character input device further includes: an image processing module 60, configured to process the object gesture contour image information collected by the object gesture contour image information collection module 40 as follows:

The step of the object fingertip trajectory information acquiring module 20 performing tracking of the fingertip of the object according to the gesture contour model includes:

(1) mapping a rectangular area of the fingertip coarsely positioned of the reference gesture contour model to the extracted object gesture contour image, and obtaining a rectangular area where the fingertip position of the object gesture contour image is located;

(2) equally dividing the contours of the fingers in the rectangular area, and calculating the edge bending rate of each of the finger contour segments;

(3) Obtain the finger contour segment with the largest edge bending rate as the fingertip of the object.

In addition, the method for acquiring the trajectory information of the fingertip of the object by the target finger trajectory information acquiring module 20 through the video frame collected by the image sensing module includes:

(1) Tracking prediction of the fingertip of the object according to the Kalman filter;

(2) acquiring the starting coordinate information of the fingertip of the object from the video frame collected by the image sensing module, and then collecting the real-time coordinate information of the fingertip of the object at least once every at least one frame;

(3) calculating a tangential angle of the real-time coordinate information and the previous coordinate information, and acquiring trajectory information of the fingertip of the object according to the change of the tangential angle, wherein the calculation formula of the tangential angle ^ is as follows:

^= f ^ ' where ¹ , }^ ¹ ) refers to the coordinates of the object's fingertip at the last t-i time Information, the () refers to real-time coordinate information of the fingertip of the object at real time t.

The method for the character conversion module 30 to query the pre-trained character model library according to the running track information to convert the running track information into corresponding characters includes:

(1) querying the pre-trained character model library according to the running track information, and matching the running track information with all character models;

(2) obtaining the character model with the largest likelihood value as the target character model, wherein the Verbit algorithm is used to calculate the likelihood value of each character model;

(3) converting the running track information into corresponding characters according to the target character model. Correspondingly, the embodiment of the present invention further provides a terminal, which includes the image input module based character input device as described above. Referring to FIG. 3, the device includes:

The object fingertip running track information acquiring module 20 is configured to acquire the running track information of the fingertip of the object through the video frame collected by the image sensing module;

In the terminal, the image sensing module can be a common camera device. Therefore, for the terminal provided by the present invention, since the image can be acquired by the camera device provided by the terminal, the gesture contour of the user is extracted, and the hand contour and the existing hand are saved. The gesture contour model is matched, and the gesture model is recognized to perform coarse positioning on the fingertip of the object, and then the fingertip of the finger is accurately positioned according to the bending rate of the contour of the finger. Predict the approximate position of the fingertip at the next moment, capture the movement trajectory of the fingertip, calculate the tangential angle at different moments, and accumulate the tangential angle changes over a period of time to obtain the trajectory of the fingertip of the object during the time period. The obtained object fingertip trajectory is matched with the pre-stored character model library to generate corresponding characters. The image input module-based character input method, device and terminal provided by the invention can meet the personalized requirements of the user for the handheld terminal, and at the same time realize fast and accurate input of characters. The above description shows and describes a preferred embodiment of the present invention, but as described above, it should be understood that the present invention is not limited to the forms disclosed herein, and should not be construed as Other combinations, modifications, and environments are possible and can be modified by the teachings or related art or knowledge within the scope of the inventive concept described herein. All changes and modifications made by those skilled in the art are intended to be within the scope of the appended claims.

Claims

Claim

A character input method based on an image sensing module, the method comprising: acquiring operation track information of a fingertip of an object;

2. The image input module-based character input method according to claim 1, wherein before the acquiring the trajectory information of the fingertip of the object, the method further comprises:

The image sensor module-based character input method according to claim 2, wherein the method further comprises: before querying the pre-trained gesture contour model library according to the object gesture contour image information, the method further comprises:

Binarizing the acquired object gesture contour image information; and/or,

4. The image input module-based character input method according to claim 2, wherein the tracking of the object fingertip according to the gesture contour model is:

5. The image sensing module based character input method according to claim 1, wherein the character input method is The information about the running track of the fingertip of the acquiring object is:

Calculating a tangential angle of the real-time coordinate information and the previous coordinate information, and acquiring trajectory information of the fingertip of the object according to the change of the tangential angle, wherein the tangential angle 0 is calculated as: where (CJ ¹ ) refers to the coordinates of the fingertip of the object at the last ti time

Information, said, refers to the real-time coordinate information of the fingertip of the object at real time t.

The image input module-based character input method according to claim 1, wherein the querying the pre-trained character model library according to the running track information, converting the running track information into corresponding The characters are:

A character input device based on an image sensing module, the device comprising: an image sensing module, configured to collect a video frame including a fingertip of the object;

8. The image sensor module-based character input device of claim 7, wherein the device further comprises:

Object gesture contour image information acquisition module, used for collecting object hands through image sensing module Potential contour image information;

9. The image sensor module-based character input device of claim 8, wherein the device further comprises:

Binarizing the acquired object gesture contour image information; and/or,

The image sensor module-based character input device according to claim 8, wherein the object fingertip trajectory information acquisition module performs tracking of the fingertip according to the gesture contour model as:

The image sensor module-based character input device according to claim 8, wherein the object fingertip trajectory information acquisition module acquires the trajectory information of the fingertip of the object through the video frame acquired by the image sensing module. For:

Obtain the starting coordinate information of the fingertip of the object from the video frame captured by the image sensing module, and then Real-time coordinate information of the fingertip of the object is collected once every at least one frame;

The character input module of the image sensing module according to claim 8, wherein the character conversion module queries the pre-trained character model library according to the running track information to use the running track information Convert to the corresponding characters as:

A terminal, characterized in that the terminal comprises the image sensing module based character input device according to any one of claims 7 to 12.