WO2013189279A1

WO2013189279A1 - Determination method and device for movement information about terminal

Info

Publication number: WO2013189279A1
Application number: PCT/CN2013/077397
Authority: WO
Inventors: 刘嫕; 刘春海
Original assignee: 中兴通讯股份有限公司
Priority date: 2012-06-19
Filing date: 2013-06-18
Publication date: 2013-12-27
Also published as: CN103516866A; KR20150023775A

Abstract

Disclosed are a determination method and device for movement information about a terminal. The method comprises: analyzing a plurality of images shot by a camera; and determining the movement information about a terminal according to the analysis result. Through the present invention, a plurality of images shot by a camera are analyzed first, and then the movement information about a terminal is determined according to the analysis result, and therefore the problems of relatively high cost, relatively large volume and relatively complicated design of a gravity sensor on an intelligent mobile terminal in the related art are solved. The function of a virtual gravity sensor is achieved by using the software technology; the demand of an upper APP for the function of the gravity sensor is met, the cost of the hardware is saved, and the area of the single board is reduced.

Description

The present invention relates to the field of communications, and in particular to a method and apparatus for determining terminal motion information. BACKGROUND OF THE INVENTION With the advancement of smart phone technology and the rapid development and popularity of smart phone applications, the number of users using smart phones is increasing, and mobile phones are more than just a simple communication tool. Due to the various needs of the application on the smartphone and the development of sensor technology, the sensors integrated on the smartphone are also becoming more and more abundant, such as gravity sensors, acceleration sensors, gyroscopes, proximity sensors, light sensors and the like. The gravity sensor on the smartphone can calculate the tilt angle of the mobile device relative to the horizontal plane by measuring the gravitational acceleration due to gravity. By analyzing the dynamic acceleration, you can analyze the direction of the mobile phone movement. Since so many sensors are integrated on the mobile phone, the hardware cost is increasing, the hardware board area is larger, and the board design is more complicated. Aiming at the problems of high cost, large area and complicated design of the gravity sensor on the intelligent mobile terminal in the related art, an effective solution has not been proposed yet. SUMMARY OF THE INVENTION In view of the problems of high cost, large area, and complicated design of a gravity sensor on a smart mobile terminal in the related art, the present invention provides a method and apparatus for determining terminal motion information to solve at least the above problems. According to an aspect of the present invention, a method for determining terminal motion information is provided, the method comprising: analyzing a plurality of images captured by a camera; and determining motion information of the terminal according to the analysis result. Before analyzing the plurality of images captured by the camera, the method may further include: selecting the camera for capturing the plurality of images; and determining an imaging mode of the camera according to the motion information of the terminal obtained. Determining the shooting mode of the camera according to the motion information of the terminal to be obtained may include: when the motion information is the rotation information of the terminal, the reference object in the image captured by the camera is compared with the reference image in the previous captured image. The rotation information of the terminal is determined by the angle at which the reference object deviates. In the case where the angle of the above deviation exceeds the threshold, the screen of the above terminal can be rotated. Determining the shooting mode of the camera according to the motion information of the terminal to be obtained may include: when the motion information is the moving direction information of the terminal, the reference object in the image captured by the camera is compared with the previous captured image The angle at which the reference object is deviated, and the time difference between the image captured by the camera and the previously captured image determine the moving direction information of the terminal. In the case where the angle of the above deviation exceeds the threshold and the time difference is less than the preset time difference, the screen of the terminal may be transformed. The above threshold may be between 0 and 90 degrees. According to another aspect of the present invention, a device for determining terminal motion information is provided, the device comprising: an image analysis module configured to analyze a plurality of images captured by a camera; and a motion information determining module configured to be according to the image analysis module The analysis results determine the motion information of the terminal. The device may further include: a camera selection module configured to select a camera for capturing the plurality of images; and a shooting mode determining module configured to determine a shooting mode of the camera according to the motion information of the terminal obtained as needed. The shooting mode determining module may include: a first imaging mode determining unit configured to: when the motion information is the rotation information of the terminal, the reference object in the image captured by the camera is compared with the reference in the previous captured image The angle of the object deviation determines the rotation information of the above terminal. The photographing mode determining module may include: a second photographing mode determining unit configured to: in a case where the motion information is the moving direction information of the terminal, the reference object in the image photographed by the camera is relative to the foregoing in the previously photographed image The moving direction information of the terminal is determined by the angle at which the reference object is deviated, and the time difference between the image captured by the camera and the previously captured image. Through the invention, the plurality of images captured by the camera are first analyzed, and then the motion information of the terminal is determined according to the analysis result, which solves the problems of high cost, large area and complicated design of the gravity sensor on the smart mobile terminal in the related art. The use of software technology to realize the function of the virtual gravity sensor not only satisfies the requirements of the upper layer application (APP) for the function of the gravity sensor, but also saves the hardware cost and reduces the board area. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are set to illustrate,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, In the drawing - 1 is a flowchart of a method for determining terminal motion information according to an embodiment of the present invention; FIG. 2 is a flowchart of an implementation process for using a Camera virtual G-sensor function according to an embodiment of the present invention; FIG. 3 is a flowchart according to an embodiment of the present invention. FIG. 4 is a block diagram showing a structure of a device for determining terminal motion information according to an embodiment of the present invention; FIG. 5 is a block diagram showing a specific structure of a device for determining motion information of a terminal according to an embodiment of the present invention; 6 is a schematic structural diagram of a virtual G-sensor function of a Camera according to an embodiment of the present invention; FIG. 7 is a flowchart of a structure implementation process of a virtual G-sensor function according to an embodiment of the present invention. BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict. In the related art, image processing and pattern recognition technologies are increasingly mature. For example, the depth information can be restored according to the reference scene image data acquired from the camera to determine the moving direction of the carrier where the camera is located, and the three-dimensional pose of the camera relative to the reference scene can be calculated in real time. . Therefore, image processing and pattern recognition technology provide algorithmic technical support for implementing a camera instead of a gravity sensor (G-sensor). Moreover, mobile phone chip technology is also developing rapidly, and multi-core processors are gradually becoming mainstream. The powerful processing capability provides hardware support for implementing complex algorithms on mobile phones. Based on this, an embodiment of the present invention provides a method and an apparatus for determining terminal motion information. The details will be described below by way of examples. The embodiment provides a method for determining terminal motion information, and a flowchart for determining a terminal motion information, as shown in FIG. 1. The method includes the following steps (step S102 - step S104): Step S102, analyzing the camera shooting a plurality of images; Step S104, determining motion information of the terminal according to the analysis result. Through the above method, the plurality of images captured by the camera are first analyzed, and then the motion information of the terminal is determined according to the analysis result, which solves the problems of high cost, large area and complicated design of the gravity sensor on the smart mobile terminal in the related art. The software technology realizes the function of the virtual gravity sensor, which not only satisfies the requirement of the upper layer APP for the gravity sensor function, but also saves the hardware cost and reduces the single board area. Before analyzing the image taken by the camera, you need to select the appropriate camera in the camera list of the mobile terminal. How to select the camera can be determined according to the actual situation. After selecting a suitable camera, it is necessary to determine the shooting mode, for example, it can be determined according to the motion information of the terminal that needs to be obtained. Therefore, the embodiment provides a preferred embodiment, that is, before analyzing the image captured by the camera, the method further includes: selecting a camera for capturing a plurality of images; determining the camera according to the motion information of the terminal obtained as needed Shooting method. For the motion information of the terminal that needs to be obtained, there may be multiple kinds of motion information, for example, when the mobile phone is rotated, a corresponding rotating mobile phone screen is needed to improve the user's experience. Or, when the terminal moves faster, the page can be flipped. For example, when the user scrolls through the mobile phone album, if the mobile phone is swung right, the next photo of the current photo can be flipped, if the mobile phone is swung to the left quickly , you can flip to the previous photo of the current photo, etc. Of course, there are other different situations, which are not listed here. Based on the foregoing, the method for determining the shooting mode of the camera is determined according to the motion information of the terminal, and the present embodiment provides a preferred implementation manner, that is, the motion information is the rotation information of the terminal. In this case, the rotation information of the terminal is determined based on the angle at which the reference object in the image captured by the camera deviates from the reference object in the previously captured image. And, when the angle of the above deviation exceeds the threshold, the screen of the terminal is rotated. The threshold is the calibration accuracy, and the threshold can be set between 0 and 90 degrees. Setting the accuracy of a calibration α° means that the screen of the terminal will be rotated only when the phone is rotated to α° relative to the reference object. Or, in a case where the motion information is the movement direction information of the terminal, an angle of deviation of the reference object in the image captured by the camera with respect to the reference object in the previous captured image, and a time difference between the image captured by the camera and the previously captured image, Determine the direction of movement of the terminal. And, in a case where the angle of the deviation exceeds the threshold and the time difference is less than the preset time difference, the screen of the terminal is transformed. The threshold is the calibration accuracy, and the threshold can be set between 0 and 90 degrees. The implementation process of the above embodiment will be described in detail below in conjunction with the preferred embodiments and the accompanying drawings. In this embodiment, the Camera (camera) on the mobile terminal is used instead of the G-sensor function. After the camera is opened, the image data is continuously captured, and the image data is analyzed to obtain the recognition result of the mobile phone's own mobile mode, and then the recognition is performed. The result is reported to the application layer that needs to use the G-sensor function, and the application layer performs the operation of rotating the display content accordingly. 2 is a flowchart of an implementation process of using the Camera virtual G-sensor function according to an embodiment of the present invention. As shown in FIG. 2, the method includes (Step S202 - Step S212): Step S202, a mobile phone user from a home interface (home Interface) Go to the phone settings menu and select Enable virtual G-sensor in the setup menu. All Camera columns currently integrated on the phone are listed in the settings menu. Table, select the Camera device to replace the G-sensor function. In this way, after the virtual G-sensor function is turned on, the Camera used to replace the G-sensor function is also automatically turned on. Step S204, after the virtual G-sensor function is turned on in the setting menu, click the virtual G-sensor calibration in the setting menu. The accuracy of the calibration can be set to generally range from 0° to 90°. Setting the accuracy of a calibration α° means that only when the mobile phone is rotated to α° relative to the reference object, a mobile recognition result is reported to the upper layer. Step S206, after setting the calibration accuracy, click Start Calibration in the menu. At this time, the camera that has been selected to replace the G-sensor function needs to be calibrated to the reference object. At the time of calibration, the identified reference image data is stored as a reference image for the virtual G-sensor to recognize the mobile phone movement mode. After the calibration is completed, the calibration will be prompted. Step S208, after the calibration is successful, the camera that replaces the G-sensor function can be aligned with the reference object to perform the virtual G-sensor. If you need to replace the Camera that replaces the G-sensor function or replace the reference object, you can reselect and calibrate. Step S210, after the virtual G-sensor function is turned on, select to turn on the automatic rotation screen in the setting menu, and the virtual G-sensor analyzes and compares the identified reference object with the stored standard reference object. In step S212, the analysis result is reported to the application layer that needs to use the G-sensor function, and the application layer rotates the display content according to the obtained mobile phone mobile mode information to achieve the result of automatically rotating the screen according to gravity sensing similar to G-sensor. . FIG. 3 is a schematic diagram of a virtual G-sensor setting window according to an embodiment of the present invention. As shown in FIG. 3, Cameral, Camera2, and the like are listed in the Camera list, and a suitable Camera can be selected therefrom. The virtual G- _SenSO r, calibration accuracy [0°-90°], auto-rotate screen options are listed on the left side of the window, and the options are set on the right side of the window: On/Off, Start Calibration, On/Off. Users can set related options on this setup window to implement the virtual G-sensor function instead of Camera. Corresponding to the method for determining the terminal motion information, the embodiment provides a device for determining terminal motion information, which is used to implement the above embodiment. 4 is a structural block diagram of a device for determining terminal motion information according to an embodiment of the present invention. As shown in FIG. 4, the device includes: an image analysis module 10 and a motion information determination module 20. The structure will be described below. The image analysis module 10 is configured to analyze a plurality of images captured by the camera; The motion information determining module 20 is coupled to the image analyzing module 10 and configured to be according to the image analyzing module described above.

The analysis result of 10 determines the motion information of the terminal. Through the above device, the image analysis module 10 first analyzes a plurality of images captured by the camera, and then the motion information determining module 20 determines the motion information of the terminal according to the analysis result, and solves the high cost and area of the gravity sensor on the smart mobile terminal in the related art. Larger, more complex design problems, using software technology to achieve the function of the virtual gravity sensor, not only meet the needs of the upper layer APP for gravity sensor function, but also save hardware costs and reduce the board area. Before analyzing the image taken by the camera, you need to select the appropriate camera in the camera list of the mobile terminal. How to select the camera can be determined according to the actual situation. After selecting a suitable camera, it is necessary to determine the shooting mode, for example, it can be determined according to the motion information of the terminal that needs to be obtained. Therefore, the present embodiment provides a specific structural diagram of the terminal motion information determining apparatus shown in FIG. 5, which includes, in addition to the modules in FIG. 4, the camera selection module. 30 and the shooting mode determining module 40. The structure will be described below. The camera selection module 30 is configured to select a camera for capturing a plurality of images; the shooting mode determining module 40 is coupled to the camera selecting module 30 and the image analyzing module 10, and is configured to determine the camera according to the motion information of the terminal obtained as needed. Shooting method. According to the motion information of the terminal, which is required to be determined, the image capturing mode of the camera is determined. The present embodiment provides a preferred embodiment, that is, the shooting mode determining module 40 may include: a first shooting mode determining unit. When the motion information is the rotation information of the terminal, the rotation information of the terminal is determined based on the angle at which the reference object in the image captured by the camera is deviated from the reference object in the previous captured image. And, when the angle of the above deviation exceeds the threshold, the screen of the terminal is rotated. The threshold is the calibration accuracy, and the threshold can be set between 0 and 90 degrees. Setting the accuracy of a calibration α° means that the screen of the terminal will be rotated only when the phone is rotated to α° relative to the reference object. Alternatively, the shooting mode determining module 40 may include: a second shooting mode determining unit configured to set the reference object in the image captured by the camera relative to the previous image in the previous captured image when the motion information is the moving direction information of the terminal The angle of deviation of the reference object, and the time difference between the image captured by the camera and the previously captured image, determine the moving direction information of the terminal. And, in a case where the angle of the deviation exceeds the threshold and the time difference is less than the preset time difference, the screen of the terminal is transformed. The threshold is the calibration accuracy, and the threshold can be set between 0 and 90 degrees. FIG. 6 is a schematic structural diagram of a virtual G-sensor function of a camera according to an embodiment of the present invention. As shown in FIG. 6, the function of the setting module is equivalent to the function of the shooting mode determining module in the above embodiment, and the setting module acquires the user setting. Parameters, including whether to turn on or off the virtual G- _SenSO r, Camera list, calibration accuracy, whether to turn on the auto-rotate screen, etc. After selecting the appropriate Camera from the Camera list and turning on the virtual G-sensor, the corresponding camera module (also called Camera module, which is the camera for capturing images) will automatically open, and the reference object will be obtained through the Camera module. The image data, the calibration module generates standard reference image data based on the collected reference image data, and stores the calibrated standard reference in the reference database. The function of the recognition algorithm module is equivalent to the functions of the image analysis module and the motion information determination module in the above embodiment, and the recognition algorithm module compares the image data collected from the Camera module with the reference object in the reference object database to calculate the acquisition. The data is offset from the position of the reference object, and then the movement direction of the mobile phone is determined, and the movement direction of the mobile phone is input into the movement direction management module of the mobile phone. Finally, the movement direction management module of the mobile phone reports the movement direction of the mobile phone to the registration use. Virtual G-sensor application. The application performs operations such as rotating the phone screen. Based on the structure of the virtual G-sensor function, this embodiment introduces the implementation process of the virtual G-sensor function of the Camera. FIG. 7 is a flowchart of the structure implementation process of the virtual G-sensor function according to the embodiment of the present invention, such as As shown in FIG. 7, the flow includes the following steps (step S702 - step S720): Step S702, first, the setting module is opened. Step S704, selecting a Camera for implementing the virtual G-sensor in the Camera list in the setting module. In step S706, the virtual G-sensor is turned on in the setting module, and the selected camera is automatically opened after the virtual G-sensor is turned on. In step S708, the calibration accuracy is set in the setting module, and the precision can be set between 0° and 90°. Step S710, click to start calibration, and generate a standard reference object image according to the image data collected by the Camera. Step S712, the standard reference object picture is stored in the reference object database. Step S714, the recognition algorithm analyzes and recognizes the moving angle of the current mobile phone according to the standard reference object picture in the reference object database and the image data collected from the Camera. Step S716, real-time management updates the movement angle of the mobile phone. Step S718, feeding back the motion angle update to the upper application. In step S720, the upper layer application performs a corresponding operation of rotating the mobile phone screen. For example, when the phone is turned from vertical to landscape, the screen orientation will rotate following the rotation of the phone. As can be seen from the above description, the present invention uses the Camera to realize the recognition of the mobile phone's own mobile mode without integrating the G-sensor, and feeds the recognition result to the application layer, and the application layer performs the corresponding operation. The invention utilizes software technology to realize the functions of other sensors, reduces the hardware cost, reduces the hardware board area, and simplifies the single board design. Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general-purpose computing device, which can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device, such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein. The steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software. The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims

Claim

A method for determining terminal motion information, comprising:

Analyzing multiple images taken by the camera;

The motion information of the terminal is determined according to the analysis result.

2. The method of claim 1, wherein before the analyzing the plurality of images captured by the camera, the method further comprises:

Selecting the camera for capturing the plurality of images;

Determining the shooting mode of the camera according to the motion information of the terminal that needs to be obtained.

The method according to claim 1, wherein, according to the motion information of the terminal that is required to be determined, determining a shooting manner of the camera comprises:

In the case where the motion information is the rotation information of the terminal, the rotation information of the terminal is determined based on an angle at which the reference object in the image captured by the camera deviates from the reference object in the previously captured image.

4. The method of claim 3, comprising: rotating the screen of the terminal if the angle of the deviation exceeds a threshold.

The method according to claim 1, wherein, according to the motion information of the terminal that is required to be determined, determining a shooting manner of the camera includes:

In the case where the motion information is the moving direction information of the terminal, an angle according to a reference object in the image captured by the camera with respect to the reference object in the previously captured image, and an image captured by the camera The moving direction information of the terminal is determined from the time difference of the previously taken image.

6. The method of claim 5, comprising: transforming a screen of the terminal if the angle of the deviation exceeds a threshold and the time difference is less than a preset time difference.

The method according to any one of claims 1 to 6, wherein the threshold is between 0 and 90 degrees.

8. A device for determining terminal motion information, comprising:

An image analysis module, configured to analyze a plurality of images captured by the camera; The motion information determining module is configured to determine motion information of the terminal according to the analysis result of the image analyzing module.

9. The apparatus according to claim 8, further comprising: a camera selection module configured to select a camera for capturing the plurality of images; a shooting mode determining module configured to obtain motion information of the terminal according to the need, Determine how the camera is shot.

The device according to claim 8, wherein the shooting mode determining module comprises:

a first photographing mode determining unit configured to set an angle at which a reference object in an image captured by the camera is deviated from the reference object in a previously captured image, in a case where the motion information is rotation information of the terminal , determining rotation information of the terminal.

a second photographing mode determining unit configured to, in a case where the motion information is moving direction information of the terminal, deviate from a reference object in an image captured by the camera with respect to the reference object in a previously captured image The angle, and the time difference between the image captured by the camera and the previously captured image, determine the moving direction information of the terminal.