TW201512996A - Method for recognizing images in real time in mobile terminal and mobile terminal thereof - Google Patents

Abstract

The present invention provides a method for recognizing images in real time in a mobile terminal and a mobile terminal thereof, in which said method comprises: utilizing a camera of the mobile terminal to collect data in real time for acquiring a video frame; proceeding a motion estimation on the video frame so as to determine a video frame motion state; determining whether the video frame motion state is "a motion state to a static state" or not, if yes, determining the video frame as a frame of clear image and uploading the frame of clear image to a cloud server; and receiving a recognition result retrieved from the cloud server and presenting the recognition result. The present invention can save network flow and retrieve the recognition result efficiently.

Description

Method for real-time image recognition in mobile terminal and its shift Mobile terminal

The present invention relates to techniques for image processing and recognition, and more particularly to a method for performing instant image recognition on a mobile terminal and a mobile terminal thereof.

The method for performing real-time image recognition on the mobile terminal includes: acquiring a video frame about the target by using the photographic lens of the mobile terminal, and transmitting the video frame to the cloud server, and the cloud server identifies the received video frame, and determines the After the corresponding description message, the feedback is sent to the mobile terminal for display.

For example, data collection can be performed on various objects such as book covers, CD covers, movie posters, barcodes, 2D barcodes, and product logos. After receiving the video frame, the cloud server feeds back a description message, and the description message may include information about the purchase of the related item, a comment message, and the like. In this way, you can shoot instantly, very fast.

There are two ways for the conventional mobile terminal to collect and transmit data, which are specifically described below.

Method 1: Use the camera lens of the mobile terminal to take a picture with the target, and send the obtained video frame to the cloud server.

This method has the following disadvantages: it is inconvenient to perform manual operation after alignment. Moreover, if there is no alignment or shaking, the cloud server will not be able to perform image recognition, and the mobile terminal will not be able to successfully obtain a description message about the target.

Method 2: Instead of taking a photo, the data is collected on the entire screen captured by the photographic lens, and the collected image data is sent to the cloud server.

Although this method does not require manual shooting, the operation is convenient, but the following disadvantages exist: since the collected video frame is sent to the cloud server in real time, the traffic is large. Moreover, some of the captured data frames are not clear, the cloud server cannot recognize, and the recognition result cannot be effectively fed back.

It can be seen that the conventional method for real-time image recognition in a mobile terminal has a large data flow and cannot effectively feedback the recognition result.

The invention provides a method for real-time image recognition in a mobile terminal and a mobile terminal thereof, so as to save network traffic and effectively feedback the recognition result.

The present invention provides a method for performing instant image recognition on a mobile terminal, comprising the steps of: performing data acquisition by using a photographic lens of the mobile terminal to obtain a video frame; performing motion estimation on the video frame to determine a motion state of the video frame Determining whether the motion state of the video frame is motion to standstill, if yes, determining to be a clear frame image, uploading the clear frame image to the cloud server; and receiving the recognition result fed back by the cloud server, displaying the identification result.

Another aspect of the present invention provides a mobile terminal for performing instant image recognition, comprising: a data collection unit, a motion estimation unit, a clear frame determination unit, and a recognition result display unit, wherein: the data collection unit utilizes the mobile terminal The photographic lens acquires the data frame to obtain the video frame and sends the video frame to the motion estimation unit. The motion estimation unit performs motion estimation on the video frame to determine the motion state of the video frame, and sends the video frame to the clear frame judgment unit. The clear frame judging unit judges whether the motion state of the video frame is motion to standstill, and if yes, determines the clear frame image, uploads the clear frame image to the cloud server; and the recognition result display unit receives The recognition result fed back by the cloud server displays the recognition result.

As can be seen from the above technical solution, in the present invention, motion estimation is performed on the collected video frame, and the motion state of the video frame is determined. When it is determined that the motion state of the video frame is motion to standstill, it is determined as a clear frame image, and The clear frame image is uploaded to the cloud server. The invention adopts the method of actively collecting data by the photographic lens, and does not require the user to manually take a photo, and the operation is simple. Moreover, only the clear frame image is sent to the cloud server, instead of instantly transmitting the captured video frame to the cloud server, saving traffic. Moreover, since the cloud server is based on the clear frame image feedback recognition result, the recognition result is more effective.

101‧‧‧ Instantly collect data using the camera lens of the mobile terminal to obtain the video frame

102‧‧‧A motion estimation of the video frame to determine the motion state of the video frame

103‧‧‧ Determine whether the motion state of the video frame is motion to standstill, if yes, determine the clear frame image, upload the clear frame image to the cloud server

104‧‧‧ Receive the recognition result of the cloud server feedback, and display the recognition result

201‧‧‧ Instantly collect data using the camera lens of the mobile terminal to obtain the video frame

202‧‧‧A motion estimation of the video frame to determine the motion state of the video frame

203‧‧‧ Determine whether the motion state of the video frame is motion to standstill, if yes, execute step 204; otherwise, end the flow

204‧‧‧ Calculate the number of corner features of the video frame to be detected

205‧‧‧ Is the number of features of the corner point greater than the threshold of the number of corner points? If yes, it is determined to be a clear frame image, and the clear frame image is uploaded to the cloud server; otherwise, it is determined to be a blurred frame image.

206‧‧‧ Receive the recognition result of the cloud server feedback, and display the recognition result

301‧‧‧Get the central area pixel of the frame to be processed and store it

302‧‧‧Get the central area pixel of the previous video frame of the pending video frame

303‧‧‧ Using the central area of the to-be-processed video frame as a starting point, searching for an area similar to the pixel of the central area of the previous video frame to determine a matching block

304‧‧‧ Calculate the position vector between the central area of the to-be-processed video frame and the matching block as a motion vector

305‧‧‧Determining the motion state of the video frame from the motion vector

306‧‧ Does the exercise estimate continue?

501‧‧‧ data collection unit

502‧‧‧Sports Estimation Unit

503‧‧‧clear box judgment unit

504‧‧‧Recognition result display unit

505‧‧‧Motion vector calculation subunit

506‧‧‧Status determination subunit

507‧‧‧Status determination module

508‧‧‧Motion vector determination module

509‧‧‧Moving to stationary determination module

510‧‧‧ corner detection module

FIG. 1 is a schematic flow chart showing a method for performing instant image recognition on a mobile terminal according to the present invention.

FIG. 2 shows an example of a flow chart of a method for performing instant image recognition on a mobile terminal according to the present invention.

Figure 3 shows an example of a flow chart of the motion estimation method of the present invention.

Figure 4 shows an example of a schematic diagram of data block matching in accordance with the present invention.

Figure 5 is a diagram showing the structure of a mobile terminal for performing instant image recognition according to the present invention.

The following description of the various embodiments is intended to illustrate the specific embodiments

In the process of carrying out the invention, the inventor found that in the process of data collection, in actual application, the user first opens the photographic lens, then moves to the aiming target, and the photographic lens performs data collection, which is a motion to a stationary state. process. Based on this, the present invention determines the motion state of the captured video frame, and when it is known that the motion state of the video frame is motion to standstill, it is determined as a clear frame image, and the clear frame image is uploaded to the cloud server. In this way, only the clear frame image is sent to the cloud server, which saves traffic. Moreover, since the cloud server is based on the clear frame image feedback recognition result, the recognition result is more effective.

Referring to FIG. 1, there is shown a schematic flow chart of a method for performing instant image recognition on a mobile terminal of the present invention, which includes the following steps.

Step 101: Instantly collect data by using a photographic lens of the mobile terminal to obtain a video frame.

Step 102: Perform motion estimation on the video frame to determine a motion state of the video frame.

The moving photographic lens captures the frame frame by frame, and performs motion estimation on a video frame that is acquired immediately to determine the motion state of the certain video frame.

The English name for motion estimation is Motion Estimation, which is used in video coding technology. The present invention applies motion estimation to the processing of the video frame captured by the photographic lens of the mobile terminal to determine the motion state of the video frame. Specifically, the motion vector can be used to determine the motion state of the video frame, including: calculating a motion vector between the video frame and the previous video frame, the motion vector including the motion amplitude and the motion direction; determining the motion state of the video frame by the motion vector .

The motion vector is used to calculate the motion vector between the video frame and the previous video frame. The method may be as follows: acquiring a central region pixel of the previous video frame; searching for the surrounding area of the video frame as a starting point An area similar to the central area pixel of the previous video frame to determine a matching block; and a position vector between the central area of the video frame and the matching block as a motion vector.

The motion state includes motion, rest, motion to rest, rest to motion, and there are many ways to determine the motion state of the video frame by the motion vector, which can be set according to actual needs. Determining the motion state of the video frame by the motion vector includes: reading the stored background motion state; If the background motion state is stationary, and the motion amplitude of the continuous N frame from the current frame is greater than a first motion threshold, N is a natural number, and the current frame is the first frame, then the motion of the first to N+1 frames The state is stationary, the background motion state is still stationary, the N+1th frame motion state is determined to be stationary to motion, and the background motion state is modified to motion; if the background motion state is stationary, and the current frame motion amplitude is less than The first motion threshold, the current frame motion state is still still, the background motion state is still; and if the background motion state is motion, and the continuous N frame motion amplitude from the current frame is less than a second motion threshold , N is a natural number, the current frame is the first frame, then the motion state of the first to N+1 frames is motion, the background motion state is still motion, and the N+1th frame motion state is determined to be motion to rest, and Modifying the background motion state to be stationary; if the background motion state is motion, and the current frame motion amplitude is greater than the second motion threshold, the current frame motion state is still motion, and the background motion state is still move.

Further, after determining that the background motion state is stationary, and the motion amplitude of the current frame is less than the first motion threshold, the method further includes: determining whether the motion amplitude is greater than a third motion threshold, and if yes, the current frame motion For the micro motion, the background motion state is still stationary. If the motion of the continuous M frame from the current frame is the micro motion in the same direction, and the current frame is the first frame, the motion state of the M frame is determined to be stationary to Exercise and modify the background motion state to motion, M is a natural number.

In the case where the background motion state is stationary, if the motion amplitude is known, the motion amplitude of the two consecutive frames after the last video frame is greater than S1, and When it is known from the direction of motion that the moving directions of the two consecutive frames are opposite, it is determined that the motion is still, and the motion state of the two consecutive frames is still determined to be stationary.

If it is known by the motion amplitude that the motion amplitude of the two consecutive frames after the last video frame is greater than S1, and the motion direction is known to be the same in the motion direction of the two consecutive frames, the latest frame in the two consecutive frames is determined to be stationary to motion. status.

Step 103: Determine whether the motion state of the video frame is motion-to-station, and if yes, determine the clear frame image, and upload the clear frame image to the cloud server.

If it is determined that the motion state of the video frame is not moving to a standstill, the video frame is not uploaded to the cloud server.

Further, in order to improve the accuracy of the judgment clear frame, after determining that the motion state of the video frame is motion to standstill, corner point detection may also be performed, as follows: calculating the number of corner feature of the video frame; and determining the corner point Whether the number of features is greater than a threshold number of corner points, and if so, is determined to be a clear frame image; otherwise, it is determined to be a blurred frame image.

Generally, after determining that the motion state of the video frame is moving to a stationary state, and determining the clear frame image, the clear frame image is uploaded to the cloud server. In some application environments, the timing of uploading a clear frame image may be determined based on the motion state of consecutive multiple video frames. Specifically, the current frame is the first frame, and the first frame is determined. When the N+1 frame is in a static state, it is determined that the N+1 frame is a clear frame, and the clear frame image is uploaded to the cloud server, where N is a natural number.

Step 104: Receive a recognition result fed back by the cloud server, and display the recognition result.

After receiving the video frame, the cloud server will feed back related description messages, including information about the purchase of related items, comment messages, and the like.

In the present invention, motion estimation is performed on the collected video frame, and the motion state of the video frame is determined. When it is determined that the motion state of the video frame is motion to standstill, it is determined as a clear frame image, and the clear frame image is uploaded to the cloud. server. The invention adopts the method of actively collecting data by the photographic lens, and does not require the user to manually take a photo, and the operation is simple. Moreover, only the clear frame image is sent to the cloud server, instead of instantly transmitting the captured video frame to the cloud server, saving traffic. Moreover, since the cloud server is based on the clear frame image feedback recognition result, the recognition result is more effective.

The following is a description of a method for performing instant image recognition on a mobile terminal according to the present invention through FIG. 2, which includes the following steps.

Step 201: Instantly collect data by using a photographic lens of the mobile terminal to obtain a video frame.

Step 202: Perform motion estimation on the video frame to determine a motion state of the video frame.

For convenience of explanation, the video frame for performing motion estimation is hereinafter referred to as a video frame to be processed.

In the present invention, the conventional motion estimation concept for video coding is transplanted to the processing of the image by the photographic lens of the mobile terminal, and the image sequence acquired by the photographic lens of the mobile terminal has a common continuous image correlation. Therefore, the motion estimation algorithm is universal. However, there are differences between the two, For example, the image resolution obtained by the photographic lens of the mobile terminal tends to be low, and the mobile terminal does not have too much motion when the user actually uses it; more importantly, the motion estimation algorithm for the global domain is adopted in the video coding. This calculation method is very slow, and even on a PC, it is often impossible to achieve immediate results. Therefore, for the difference between the two, the present invention improves the motion estimation algorithm for video coding, so that it can achieve very efficient performance on various mobile terminals, while consuming less CPU resources and even consuming CPU. Resources are basically negligible.

Referring to Figure 3, there is shown an example of a flow chart of a motion estimation method of the present invention, which includes the following steps.

Step 301: Acquire a central area pixel of the to-be-processed video frame, and store it.

Step 302: Acquire a central area pixel of a previous video frame of the to-be-processed video frame.

Each time the mobile terminal collects the video frame, the central area pixel of the video frame is stored. Specifically, the pixel gray value of the central area is stored. In this step, the stored central region pixel grayscale value of the previous video frame immediately adjacent to the video frame to be processed is extracted.

Step 303: Starting from a central area of the to-be-processed video frame, searching for an area similar to a pixel of a central area of the previous video frame to determine a matching block.

The method of determining the matching block will be described in detail below with reference to FIG. In Figure 4, the square area marked with the grid in the previous video frame is the center area of the previous video frame, and the dotted area in the to-be-processed video frame is waiting. The central area of the video frame is processed, and a finite neighborhood around the dotted frame is searched from the inside to the outside to find an area similar to the gray value of the pixel in the central area of the previous video frame. This area is called a matching block. The square area marked with the grid in the pending video frame is the searched matching block.

In this example, the pixel gray level of the central area (x, y) of the previous video frame is represented as I ( x, y ), and the to-be-processed video frame is used to match the central area of the previous video frame. The search block is expressed as I '( x, y ), and the sum of squares of the differences between the two is used as an index of block similarity. Assuming that the block size is N times N pixels, the sum of squared errors S is:

According to this formula, the block with the smallest S is calculated as a matching block. The motion vector between the two frames is determined according to the position of the matching block to the central region of the previous video frame, and the arrow indicates the direction of motion as shown in FIG. The above search process uses an approximation algorithm. Specifically, the large step movement is first performed to find a region where the similarity difference is relatively small; then the step size is reduced in the region, and the final search result is gradually approached. In order to ensure the speed of the algorithm, if the pixel of the video frame is too large and exceeds a certain threshold, the downsampling process may be performed first. For example, the video frame multiplied by 2000 by 2000 is downsampled to 400 times 400. In Fig. 4, a rectangular area is used to indicate a matching block, but in practical applications, other shapes of blocks such as diamond matching, circular matching, or the like may be used for matching.

In the motion estimation, in addition to the similarity determination method using the sum of squares of the above errors, a mean square error, an absolute error sum, an average error sum, and other similarity determination methods may be employed. In addition to using an approximation search algorithm In addition, in practical applications, other search algorithms such as a three-step method and a diamond search method can also be used.

Step 304: Calculate a position vector between the central area of the to-be-processed video frame and the matching block as a motion vector.

The calculated motion vector includes the direction of motion and the magnitude of motion.

In step 305, the motion state of the video frame is determined by the motion vector.

In the present invention, the video frame motion state includes, but is not limited to, the following four states: motion, still, motion to still, still to motion, wherein the motion to the stationary state is determined as the timing of uploading an image.

In practical applications, different amplitude thresholds are required for motion to rest and rest to motion. In image recognition applications, the amplitude threshold for motion to rest is generally higher, and the amplitude threshold is represented by a second motion threshold. The amplitude threshold for rest to motion is lower, the amplitude threshold at this time is represented by a first motion threshold, and the first motion threshold is less than the second motion threshold.

The mobile terminal stores a background motion state that can be extracted from the stored state. Then, combining the background motion state, the first motion threshold, and the second motion threshold, the motion state of the to-be-processed video frame can be determined, as follows: reading the stored background motion state; if the background motion state is stationary, and The motion amplitude of the continuous N frame from the current frame is greater than a first motion threshold, N is a natural number, and the current frame is the first frame, and the motion state of the first to N+1 frames is still, and the background motion state is still For stationary, the N+1th frame motion state is determined to be stationary to motion, and the background motion state is modified to motion; if the background motion state is static And if the current frame motion amplitude is less than the first motion threshold, the current frame motion state is still still, the background motion state is still still; and if the background motion state is motion, and the continuous N frame motion is started from the current frame The amplitude is less than a second motion threshold, N is a natural number, and the current frame is the first frame, then the motion state of the first to N+1 frames is motion, the background motion state is still motion, and the N+1th frame motion is performed. The state is determined to be moving to rest, and the background motion state is modified to be stationary; if the background motion state is motion, and the current frame motion magnitude is greater than the second motion threshold, the current frame motion state is still motion, and the background motion state is Still sporty.

After determining that the background motion state is stationary, and the motion amplitude of the current frame is less than the first motion threshold, the method further includes: determining whether the motion amplitude is greater than a third motion threshold, and if yes, the current frame motion is micro motion The background motion state is still stationary. If the motion of the continuous M frame is the micro motion in the same direction from the current frame, and the current frame is the first frame, the motion state of the M frame is determined to be stationary to motion, and The background motion state is modified to motion, and M is a natural number.

In this example, the "state hold" strategy is adopted, and state switching is not performed for sporadic single-stationary or motion states, and state switching is performed only when more than two state changes are accumulated, and the state can be achieved through this strategy. Stability. The first motion threshold is represented by S1, the second motion threshold is represented by S2, the third motion threshold is represented by S3, and S represents the magnitude of motion of the video frame to be processed. It is assumed that it is usually necessary to accumulate two state changes before performing state switching, and for micromotions, it is necessary to accumulate five state changes before performing state switching. The corresponding "state keeping" strategy specifically includes the following.

(1) When the background motion state is stationary:

(1) When S>S1, it is determined that the to-be-processed video frame (indicated by the Y-frame) is in a stationary state, the background motion state is still still, and then it is determined whether the motion amplitude of the Y+1 frame is still greater than S1, if Then, it is determined that the Y+1 frame is stationary to the motion state, and the background motion state is modified to the motion.

(2) When S<S1, it is determined that the to-be-processed video frame is in a stationary state, and the background motion state is still stationary.

(3) When S3 < S < S1, it is determined that the to-be-processed video frame (represented by the Z-th frame) is micro-motion, and the Z-th to Z+3 frames are judged to be micro-motions in the same direction, but the Z-th to Z+ The 3 frame is still determined to be in a stationary state. If the Z+4 frame is also a micro motion in the same direction, the Z+4 frame is determined to be stationary to the motion state, and the background motion state is modified to the motion. The cumulative number of times can be set as needed.

(2) The situation in which the background motion state is exercise:

(1) When S<S2, it is determined that the to-be-processed video frame (indicated by the Y-th frame) is in motion, the background motion state is still motion, and then it is judged whether the motion amplitude of the Y+1 frame is still less than S2, if Then, it is determined that the Y+1 frame is moved to a stationary state, and the background motion state is changed to still.

(2) When S>S2, it is determined that the to-be-processed video frame is in motion, and the background motion state is still motion.

Further, it is also possible to make a determination by the situation of the opponent's shaking. If there is a situation of "sudden left and right", that is, the opposite direction of motion, it is judged to be "hand shake". In this case, if the background is stationary, the motion state is not modified until the same direction is generated. Until now.

At step 306, it is determined whether to continue the motion estimation. If yes, return to step 301 to continue the execution, otherwise the flow ends.

If the video frame is continuously acquired in step 201, then to step 301, motion estimation of the acquired video frame is continued.

Step 203: Determine whether the motion state of the video frame is motion to standstill. If yes, execute step 204; otherwise, end the process.

When the photographic lens is just opened, the status can be set to rest by default; after that, the user moves the photographic lens to the target, and the process will experience rest to motion, motion, and motion to rest.

When it is determined that the motion state of the video frame is motion to standstill, the corresponding video frame is used as the video frame to be detected.

Step 204: Calculate the number of corner feature of the video frame to be detected.

There are many kinds of corner detection algorithms, such as FAST corner detection algorithm, Harris corner detection algorithm, CHOG corner detection algorithm, FREAK corner detection algorithm, etc., one of which can be selected. These algorithms have better corners. Test your ability. According to the definition of the effective picture, the first requirement is clear, and the second requirement has a rich texture. Based on these two points, the FAST corner detection algorithm can be used. When the picture is not clear, there are often fewer FAST corner points. For example, in a large blank or single color picture, the FAST corner point is very small, so it is only necessary to judge the number of FAST corner points of the picture to determine whether it is valid. image.

Moreover, in addition to the corner point detection algorithm for image validity discrimination, in practical applications, an algorithm for discriminating image validity based on gradient features, edge features, and the like can also be employed.

Step 205: Determine whether the number of corner feature numbers is greater than a corner number threshold. If yes, determine the clear frame image, and upload the clear frame image to the cloud server; otherwise, determine the blur frame image.

Step 206: Receive a recognition result fed back by the cloud server, and display the recognition result.

Referring to FIG. 5, it is a schematic structural diagram of a mobile terminal for performing instant image recognition according to the present invention. The mobile terminal includes a data collection unit 501, a motion estimation unit 502, a clear frame determination unit 503, and a recognition result display unit 504. .

The data collection unit 501 performs data collection by using the photographic lens of the mobile terminal to acquire a video frame and send it to the motion estimation unit 502.

The motion estimation unit 502 performs motion estimation on the video frame to determine the motion state of the video frame, and sends it to the clear frame determining unit 503.

The clear frame determining unit 503 determines whether the motion state of the video frame is motion-to-station, and if so, determines the clear frame image, and uploads the clear frame image to the cloud server.

The recognition result display unit 504 receives the recognition result fed back by the cloud server, and displays the recognition result.

Preferably, motion estimation unit 502 includes a motion vector calculation sub-unit 505 and a state determination sub-unit 506.

The motion vector calculation sub-unit 505 calculates the motion vector between the video frame and its previous video frame and sends it to the state determination sub-unit 506, which includes the motion amplitude and the motion direction.

The state determination sub-unit 506 determines the motion state of the video frame from the motion vector.

Preferably, the state determining sub-unit 506 includes a state determining module 507 that reads the stored background motion state; if the background motion state is stationary, and the continuous N-frame motion amplitude from the current frame is greater than a first Motion threshold, N is a natural number, the current frame is the first frame, then the motion state of the first to N+1th frames is stationary, the background motion state is still still, and the N+1th frame motion state is determined to be stationary to Moving, and modifying the background motion state to motion; if the background motion state is stationary, and the current frame motion magnitude is less than the first motion threshold, the current frame motion state is still still, and the background motion state is still stationary.

If the background motion state is motion, and the motion range of the continuous N frame from the current frame is less than a second motion threshold, N is a natural number, and the current frame is the first frame, then the motion state of the first to N+1 frames For motion, the background motion state is still motion, determining the motion state of the N+1th frame to move to rest, and modifying the background motion state to be stationary; if the background motion state is motion, and the current frame motion amplitude is greater than the The second motion threshold is that the current frame motion state is still motion, and the background motion state is still motion.

Preferably, after the state determination module 507 determines that the background motion state is stationary, and the motion amplitude of the current frame is less than the first motion threshold, it is further determined whether the motion amplitude is greater than a third motion threshold, and if so, the current The frame motion is micro motion, and the background motion state is still stationary. If the motion of the continuous M frame from the current frame is micro motion in the same direction, the current When the frame is the first frame, the motion state of the Mth frame is determined to be stationary to motion, and the background motion state is modified to motion, and M is a natural number.

Preferably, the motion vector calculation unit 505 includes a motion vector determination module 508 that acquires a central region pixel of the previous video frame; and searches for the previous video around the central region of the video frame as a starting point. A pixel-like area in the center of the frame is used to determine a matching block; a position vector between the central area of the video frame and the matching block is used as a motion vector.

Preferably, the clear frame determining unit 503 includes a motion to stationary determining module 509 and a corner detecting module 510.

The motion-to-station determination module 509 determines whether the motion state of the video frame is motion-to-station, and if so, sends a start command to the corner detection module 510.

The corner detection module 510 receives the start command from the motion to the stationary determination module 509, calculates the number of corner features of the video frame, determines whether the number of corner features is greater than a threshold number of corner points, and if so, determines to be clear. The frame image is uploaded to the cloud server; otherwise, it is determined to be a blurred frame image.

The mobile terminal described in the embodiments of the present invention may be implemented by using hardware, software, firmware, or any combination of the three. If implemented in software, these functions can be stored on a computer readable medium or transmitted as one or more instructions or code on a computer readable medium. Computer readable media includes computer storage media and communication media that facilitates transfer of a computer program from one place to another. The storage medium can be any available media that can be accessed by any general purpose or special computer. For example, such a computer readable medium This may include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other device that can be used for carrying or storing instructions or data structures and other general or special computers. Or a medium of program code in the form of a general or special processor read. In addition, any link may be appropriately defined as a computer readable medium, and the disk and disc include a compact disk, a laser disk, a compact disk, a DVD, a floppy disk, and a Blu-ray disk, and a disk. The data is usually reproduced magnetically, and the disc is usually optically replicated with a laser. Combinations of the above may also be included in a computer readable medium.

While the invention has been described above in terms of preferred embodiments, it is not intended to limit the invention. Various changes and modifications may be made without departing from the spirit and scope of the invention. Therefore, the scope of the invention is defined by the scope of the appended claims.

101‧‧‧ Instantly collect data using the camera lens of the mobile terminal to obtain the video frame

102‧‧‧A motion estimation of the video frame to determine the motion state of the video frame

103‧‧‧ Determine whether the motion state of the video frame is motion to standstill, if yes, determine the clear frame image, upload the clear frame image to the cloud server

104‧‧‧ Receive the recognition result of the cloud server feedback, and display the recognition result

Claims (13)

A method for real-time image recognition in a mobile terminal, comprising the steps of: performing data acquisition by using a photographic lens of the mobile terminal to obtain a video frame; performing motion estimation on the video frame to determine a motion state of the video frame; determining the video Whether the motion state of the frame is motion to standstill, if yes, it is determined as a clear frame image, and the clear frame image is uploaded to the cloud server; and the recognition result fed back by the cloud server is received, and the recognition result is displayed. The method for performing real-time image recognition on a mobile terminal according to claim 1, wherein the step of performing motion estimation on the video frame to determine the motion state of the video frame comprises: calculating the video frame and the previous one. a motion vector between the video frames, the motion vector including a motion amplitude and a motion direction; and the motion vector determines the motion state of the video frame. The method for performing instant image recognition on a mobile terminal as described in claim 2, wherein the step of determining the motion state of the video frame by the motion vector comprises: reading a stored background motion state; if the background motion state If it is stationary, and the motion range of the continuous N frame from the current frame is greater than a first motion threshold, N is a natural number, and the current frame is the first frame, then the motion state of the first to N+1 frames is static. Background movement The state is still stationary, the N+1th frame motion state is determined to be stationary to motion, and the background motion state is modified to motion; and if the background motion state is motion, and the continuous N frame motion range is from the current frame Less than a second motion threshold, N is a natural number, and the current frame is the first frame, then the motion state of the first to N+1 frames is motion, the background motion state is still motion, and the N+1th frame motion state is determined. To move to rest, and change the background motion state to still. The method for performing instant image recognition on a mobile terminal according to claim 3, wherein after determining that the background motion state is stationary and the motion amplitude of the current frame is less than the first motion threshold, the method further includes : determining whether the motion amplitude is greater than a third motion threshold, and if so, the current frame motion is micro motion, and the background motion state is still stationary. If the motion of the continuous M frame from the current frame is micro motion in the same direction, When the current frame is the first frame, the motion state of the Mth frame is determined to be stationary to motion, and the background motion state is modified to motion, and M is a natural number. The method for performing real-time image recognition on a mobile terminal according to claim 3, wherein after determining that the background motion state is stationary, the method includes: if the motion frame is known after the last video frame, two consecutive frames The motion amplitude is greater than the first motion threshold, and it is determined by the motion direction that the motion directions of the two consecutive frames are opposite, then the jitter condition is determined, and the motion state of the two consecutive frames is still determined to be stationary, and the motion state of the background is still still. The method for performing real-time image recognition on a mobile terminal as described in claim 2, wherein the step of calculating a motion vector between the video frame and the previous video frame comprises: acquiring a central area of the previous video frame a pixel; searching for a region similar to a pixel of a central region of the previous video frame from a central region of the video frame to determine a matching block; and matching the central region of the video frame with the matching block The position vector between the vectors as a motion vector. The method for performing instant image recognition on a mobile terminal according to any one of claims 1 to 6, wherein after determining that the motion state of the video frame is motion to standstill, the method further comprises: calculating the The number of corner feature of the video frame; and determining whether the number of corner feature is greater than a corner number threshold, and if so, determining to be a clear frame image; otherwise, determining to be a blurred frame image. A mobile terminal for performing real-time image recognition includes a data collection unit, a motion estimation unit, a clear frame determination unit, and a recognition result display unit, wherein: the data collection unit uses the photographic lens of the mobile terminal to perform data collection instantaneously Obtaining a video frame, and sending the video frame to the motion estimation unit; the motion estimation unit performs motion estimation on the video frame to determine a motion state of the video frame, and sends the video frame to the clear frame determining unit; The clear frame determining unit determines whether the motion state of the video frame is motion-to-station, and if yes, determines a clear frame image, uploads the clear frame image to the cloud server; and the recognition result display unit receives the The recognition result fed back by the cloud server displays the recognition result. The mobile terminal according to claim 8, wherein the motion estimation unit comprises a motion vector calculation subunit and a state determination subunit, wherein: the motion vector calculation subunit calculates the video frame and the previous video. a motion vector between the frames, and sent to the state determining subunit, the motion vector including a motion amplitude and a motion direction; and the state determining subunit, the motion frame vector determining the motion state of the video frame. The mobile terminal of claim 9, wherein the state determining subunit comprises a state determining module that reads the stored background motion state; if the background motion state is stationary, and continues from the current frame N The motion amplitude of the frame is greater than a first motion threshold, N is a natural number, and the current frame is the first frame, then the motion state of the first to N+1 frames is stationary, and the background motion state is still still, and the N+th The 1-frame motion state is determined to be stationary to motion, and the background motion state is modified to motion; if the background motion state is motion, and the motion amplitude of the continuous N-frame from the current frame is less than a second motion threshold, N is natural The current frame is the first frame, and the motion state of the first to N+1 frames is motion, the background motion state is still motion, and the N+1th frame motion state is determined to be motion to rest, and the background motion is The status is changed to standstill. The mobile terminal of claim 10, wherein after the state determining module determines that the background motion state is stationary, and the motion amplitude of the current frame is less than the first motion threshold, determining whether the motion amplitude is greater than one The third motion threshold, if yes, the current frame motion is micro motion, and the background motion state is still still. If the motion of the continuous M frame from the current frame is micro motion in the same direction, the current frame is the first frame. Then, the motion state of the Mth frame is determined to be stationary to motion, and the background motion state is modified to motion, and M is a natural number. The mobile terminal of claim 9, wherein the motion vector calculation unit includes a motion vector determining module that acquires a central region pixel of the previous video frame; starting from a central region of the video frame A region similar to the central region pixel of the previous video frame is searched to determine a matching block; a position vector between the central region of the video frame and the matching block is used as a motion vector. The mobile terminal according to any one of claims 8 to 12, wherein the clear frame determining unit comprises a motion to stationary determining module and a corner detecting module, wherein: the moving to the stationary determining module, Determining whether the motion state of the video frame is motion-to-station, if yes, sending a start command to the corner detection module; and the corner detection module receiving a start command from the motion to the stationary determination module, and calculating The number of corner features of the video frame; determining whether the number of corner features is greater than a threshold number of corner points, and if so, determining a clear block diagram For example, the clear frame image is uploaded to the cloud server; otherwise, it is determined to be a blurred frame image.