EP2490104B1 - Method and mobile terminal for controlling application in the mobile terminal - Google Patents

Method and mobile terminal for controlling application in the mobile terminal Download PDF

Info

Publication number
EP2490104B1
EP2490104B1 EP12155829.0A EP12155829A EP2490104B1 EP 2490104 B1 EP2490104 B1 EP 2490104B1 EP 12155829 A EP12155829 A EP 12155829A EP 2490104 B1 EP2490104 B1 EP 2490104B1
Authority
EP
European Patent Office
Prior art keywords
axis
motion
values
subspaces
groups
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP12155829.0A
Other languages
German (de)
French (fr)
Other versions
EP2490104A1 (en
Inventor
Tangsuo Li
Jianfeng Wang
Feilong Bao
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Publication of EP2490104A1 publication Critical patent/EP2490104A1/en
Application granted granted Critical
Publication of EP2490104B1 publication Critical patent/EP2490104B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1633Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1684Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
    • G06F1/1694Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72469User interfaces specially adapted for cordless or mobile telephones for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W88/00Devices specially adapted for wireless communication networks, e.g. terminals, base stations or access point devices
    • H04W88/02Terminal devices
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M2250/00Details of telephonic subscriber devices
    • H04M2250/12Details of telephonic subscriber devices including a sensor for measuring a physical value, e.g. temperature or motion

Definitions

  • the present invention relates generally to mobile terminals and, more particularly, to a method and an apparatus for controlling applications in a mobile terminal.
  • Various mobile terminals such as a mobile terminal, a tablet computer, and a smart phone increasingly include more functions and, specifically, not only basic communications services, such as calling, but also various applications can be implemented.
  • an application When an application is to be started or exited, a user performs the operation of start or exit by selecting the menus provided by the mobile terminal level by level. However, such operation is inconvenient.
  • Some high-end mobile terminals also allow users to control special functions in applications in the mobile terminals. For example, a user may zoom into a picture or control a game by shaking the terminal.
  • all of these controls are simple. More and more demands on control of applications have been made. Thus, it is difficult for simple shaking of the terminal to meet more complex user demands.
  • US 2009/0303204 describes a technique for controlling and accessing content in a mobile device using motion processing.
  • WO 2005/103863 describes a mobile device that recognizes a movement gesture of the mobile device and performs a function corresponding to the recognized movement gesture.
  • the present invention provides a method capable of easily controlling various applications of mobile terminals and the mobiles terminals.
  • the present invention also provides a method for controlling applications of mobile terminal by user writing with the mobile terminal and the mobile terminal
  • a method of controlling applications in a mobile terminal comprising: collecting acceleration values continuously using an acceleration sensor, when the mobile terminal is being moved; identifying a motion trace, from the collected acceleration values, indicating that the mobile terminal has been moved; matching the identified motion trace with a preset motion trace; and executing a control command mapped to the preset motion trace to control an application in the mobile terminal, if the matching succeeds; wherein each of the acceleration values is a vector including a value on an X-axis, a value on an Y-axis, and a value on an Z-axis, and wherein collecting the acceleration values comprises: performing sampling at a preset time interval, by the acceleration sensor, when the mobile terminal is moved, and storing in a first storage space including M groups of subspaces, values on the X-axis obtained in the sampling, storing in a second storage space including M groups of subspaces, values on the Y-axis obtained in the sampling, and storing in
  • a mobile terminal comprising: an acceleration sensor for detecting an acceleration when the mobile terminal is being moved; a controller for collecting acceleration values using the acceleration sensor continuously when the mobile terminal is being moved, identifying a motion trace, from the collected acceleration values, indicating that the mobile terminal has been moved, matching the identified motion trace with a preset motion trace, and executing a control command mapped to the preset motion trace if the matching succeeds, to control applications in the mobile terminal; and a memory for providing first, second, and third storage spaces including respective M groups of subspaces, wherein the acceleration value is a vector including a value on the X-axis, a value on the Y-axis, and a value on the Z-axis and, when collecting the acceleration values, the controller is configured for performing sampling at a time interval preset by the acceleration sensor when the mobile terminal is being moved, storing values on the X-axis obtained during the sampling in the M groups of subspaces of the first storage space, storing values on the
  • FIG. 1 is a block diagram of a mobile terminal according to embodiments of the present invention.
  • a controller 100 executes a function of a mobile terminal and also performs an application control according to embodiments of the present invention.
  • the controller 100 identifies motion traces that the mobile terminal has been moved, from accelerations detected by an acceleration sensor 104.
  • the identified motion traces are matched with a preset motion trace, a corresponding application is controlled by executing a control command mapped to the preset motion trace.
  • a communication unit 102 performs communication for a data communication function and a phone call executed by the controller 100.
  • the acceleration sensor 104 detects the acceleration when the mobile terminal moves.
  • a display 106 displays a screen according to the movement of the mobile terminal by a control of the controller 100.
  • An input/output interface 108 provides a user interface for an input and an output by the user for the controller 100.
  • a memory 110 stores data according to an execution by the controller 100 and stores acceleration values detected by the acceleration sensor 104.
  • the memory 110 may further include an external memory or further include a storage device such as a Hard Disk Drive (HDD).
  • HDD Hard Disk Drive
  • FIG. 2 shows a flowchart according to an embodiment of the present invention.
  • the controller 100 continuously collects acceleration values using an acceleration sensor 104 when the mobile terminal is being moved in step 200.
  • accelerations occur in the mobile terminal in general when a user holds a mobile terminal and writes with the handheld mobile terminal in a space.
  • the acceleration sensor 104 detects an acceleration of the mobile terminal.
  • a detected acceleration value is a vector including scalar quantity values on the coordinate axes of a three-dimensional space, i.e., a value on the X-axis, the Y-axis and the Z axis.
  • step 202 the controller 100 identifies a motion trace indicating that the mobile terminal has been moved from the collected acceleration values.
  • the detected acceleration values are representative of the accelerations of the mobile terminal and cannot represent a motion trace. However, a plurality of continuous accelerations can represent a certain trend of the motion of the mobile terminal. Thus, in this step, a motion trace of the mobile terminal can be identified.
  • This method is described by taking the X-axis as an example. Assuming that an acceleration value on the X-axis of the mobile terminal is detected as 2 when the mobile terminal is moved to point A, an acceleration value on the X-axis of the mobile terminal is detected as 4 when the mobile terminal is moved to point B, and an acceleration value on the X-axis of the mobile terminal is detected as 6 when the mobile terminal is moved to point C during continuous detection, it can be derived that the mobile terminal is moved in the X-axis positive direction at all of points A, B and C and is moved in the positive direction from point A to point B, and then point C.
  • a motion trace described in the present invention means a shape of trace formed by motion trends, rather than a trace formed by positions in the three-dimensional space where the mobile terminal is. Description is given only by taking the X-axis as the sample. Processing on the Y-axis and the Z-axis is similar to that on the X-axis and will not be described herein.
  • a first storage space, a second storage space and a third storage space provided by the memory 110 are used to store respective sampled values.
  • the first storage space includes M groups of subspaces to store M groups of values on the X-axis
  • the second storage space includes M groups of subspaces to store M groups of values on the Y-axis
  • the third storage space includes M groups of subspaces to store M groups of values on the Z-axis.
  • M is a positive integer greater than or equal to 1.
  • the values on the X-axis, the Y-axis, and the Z-axis can be separately processed as follows to identify a motion trace.
  • M successive converted motion identifiers can represent a motion trend on the X-axis.
  • values on the Y-axis and Z-axis they can be converted into motion identifiers based on the above principle to represent a motion trend on the Y-axis/Z-axis.
  • a first method directly determining M motion identifiers based on values in M groups of subspaces, respectively. This method is the same as that in the above example.
  • the motion identifiers corresponding to the 8 groups of subspaces in the first storage space are 1, 1, 1, -1, -1, 1, 1, and 1, respectively.
  • Values in the second storage space or third storage space are processed in the same manner.
  • motion identifiers are determined directly based on values in M groups of subspaces.
  • M is assumed that M is equal to 8 and there is only one sample value in each subspace, while in practical applications, M and a number of sampled value in each subspace vary with sampling methods.
  • a second method determining M motion identifiers based on values in M groups of subspaces, a still value, and a threshold value.
  • an acceleration sensor 104 determines the mobile terminal's not being moved as being moved by mistake due to factors such as inaccurate operation of the user and/or an error of the sensor and thereby obtain a certain sampled value when a user writes with a handheld mobile terminal.
  • a still value representing the terminal not being moved and a threshold value are set.
  • a sampled value is compared with the still value. If the difference between the sampled value and the still value falls within a range of the threshold value, the mobile terminal is regarded as not having been moved. If the difference falls outside the range of the threshold value, the mobile terminal is determined as having been moved in a positive or negative direction based on the particular case.
  • each of the M groups of subspaces of a first storage space includes N data units for respectively storing N values on the X-axis that have been obtained in sampling; each of the M groups of subspaces of a second storage space includes N data units for respectively storing N values on the Y-axis that have been obtained in sampling; each of the M groups of subspaces of a third storage space includes N data units for respectively storing N values on the Z-axis that have been obtained in sampling.
  • N is an integer greater than or equal to 1.
  • the second method performs the following steps.
  • the method includes the following steps:
  • the processing methods for values on the X-axis, Y-axis, and Z-axis are similar. All of these methods subtract a corresponding still value from the N data in a group of subspace, to compare the differences with a corresponding threshold value to determine whether the differences fall within or outside a range of the threshold, and to determine the motion identifier of the group of subspace based on results of the comparison. Since each storage space has M groups of subspaces, M motion identifiers are obtained in all of the these methods after the above processing is performed. The M motion identifiers can represent a motion trace on a certain axis as in the first method. Of course, a motion trace identified by the second method should be more accurate than that by the first method since factors such as inaccurate operation of a user and an error of the sensor is eliminated in the second method.
  • first still value the second still value and the third still value represent no motion. They are the same or different.
  • each of the first threshold value, the second threshold value and the third threshold value represents a certain range.
  • Step 204 the controller 100 matches the identified motion trace with a preset motion trace, and executes a control command mapped to the preset motion trace to control an application in the mobile terminal if the matching succeeds.
  • This step compares the X-axis motion identifier, the Y-axis motion identifier and the Z-axis motion identifier and determines whether the matching has succeeded based on the results of the comparisons.
  • the control command mapped is automatically executed and thereby the application in the mobile terminal is controlled if the matching succeeds.
  • a sampling frequency of the acceleration sensor 104 in a mobile terminal is 400 Hz, i.e., the acceleration sensor collects data once every 0.0025 seconds. That is, every second, 400 pieces of data on the X-axis, 400 pieces of data on the Y-axis and 400 pieces data on the Z-axis are collected.
  • each of a first storage space, a second storage space and a third storage space to store sampled data includes M groups of subspaces, each of which includes N data units.
  • the first storage space, the second storage space and the third storage space are shown using queue 1, queue 2, queue 3, respectively.
  • Each of the queues has 8 nodes and each of the nodes includes 50 data units.
  • the sampled data is stored in an order of queue head to queue tail when sampled data are stored.
  • the data at the queue head is deleted and a node storing the new sampled data is inserted to the queue tail when data units of all of the nodes are occupied. That is, in this embodiment, each of the queues always stores the 400 sampled data that have been last obtained. Sampled data is stored in another data structure other than a queue.
  • controller 100 has an application control-setting module that operates in two modes, i.e., a motion trace-presetting mode and a motion trace-identifying mode.
  • FIG. 3 shows a flowchart of the method of this embodiment.
  • the controller 100 allows the acceleration sensor 104 to perform sampling at a rate of 0.0025 second and to store values on the X-axis obtained in the sampling in queue 1, store values on the Y-axis obtained in the sampling in queue 2, while store values on the Z-axis obtained in the sampling in queue 3, when the mobile terminal is being moved in step 300.
  • Queues 1, 2 and 3 in this step is shown as in Figs. 4A to 4C , respectively.
  • the sampled data is stored in the order of queue head to queue tail.
  • the first group of 50 data are placed at the head node of a queue when 50 pieces sampled data are obtained.
  • the second group of 50 sampled data is stored at the second node.
  • the third group of 50 sampled data is stored at the third node, and so on.
  • all of 8 nodes of a queue are full, data at the head node of the queue is deleted and the node storing the 50 sampled data that have been recently obtained is stored at the tail of the queue.
  • each of queues 1, 2 and 3 always stores the 400 pieces of data that have been obtained lately.
  • step 302 the controller 100 determines whether the application control-setting module is running, and if so, step 304 is performed. Otherwise, step 306 is performed.
  • step 304 the controller 100 determines the current mode for the application control-setting module. If the current mode is the motion trace-presetting mode, step 312 is performed. If the current mode is the motion trace-identifying mode, step 306 is performed.
  • the application control-setting module is provided in the mobile terminal and the function of motion trace presetting or motion trace identifying is started by the application control-setting module.
  • the application control-setting module is not provided in practical applications. Whether the application control-setting module is present does not affect the implementation of the solution, and steps 302 and 304 may be omitted.
  • step 306 8 X-axis motion identifiers are determined from the values in queue 1
  • 8 Y-axis motion identifiers are determined from the values in queue 2
  • 8 Z-axis motion identifiers are determined from the values in queue 3.
  • Determining the 8 X-axis motion identifiers can be done in the method of the above steps x1 to x4, determining of the 8 Y-axis motion identifiers can be done in the method of the above steps y1 to y4, and determining of the 8 Z-axis motion identifiers can be done in the method of the above steps z1 to z4, all of which will not be described herein.
  • All of the nodes of a queue do not store sampled data when sampling has just started. For example, only the head node of the queue stores sampled data while none of the other nodes stores data at 0.125 second. Thus, only the motion identifier corresponding to the head node of the queue can be determined. After 0.125 second, the second node of the queue also stored sampled data as described in step 300. Thus, the motion identifier corresponding to the second node can be determined.
  • the queue and the motion identifiers are not specified and can be for the X-axis, Y-axis or Z-axis.
  • a first still value, a second still value and a third still value are to be further updated in this step when the 8 X-axis motion identifiers, the 8 Y-axis motion identifiers and the 8 Z-axis motion identifiers are determined. That is, between steps x3 and x4, the method further includes the following step to be performed when the 8 X-axis motion identifiers are determined. Determining whether the X-axis motion identifier corresponding to the current group is 0 and if so, calculating a mean of the N values (N values on the X-axis) of the current group and updating the first still value as the calculated mean; otherwise, returning to step x4. A similar step can occur between steps y3 and y4 and z3 and z4.
  • step 308 the identified motion trace is matched with a preset motion trace, and step 310 is performed if the matching succeeds. Otherwise, the process returns to step 300.
  • Steps 300 to 308 form a large cycle, in which the acceleration sensor 104 in the mobile terminal performs sampling continuously and accesses the queues every 0.125 second. Accordingly, a motion trace is identified for the mobile terminal every 0.125 second. The identification may fail within the first one second from when sampling is started since all of the nodes in a queue do not store data and thus the determined 8 motion identifiers are inaccurate. However, all the nodes in the queue store sampled data after the one second. Thereby, effective 8 motion identifiers can be determined. As described in an embodiment above, X-axis motion identifiers, Y-axis motion identifiers and Z-axis motion identifiers represent a motion trace that a mobile terminal has been moved.
  • a preset motion trace is also represented using X-axis motion identifiers, Y-axis motion identifiers and Z-axis motion identifiers.
  • whether the matching succeeds is determined by comparing the 8 X-axis motion identifiers, the 8 Y-axis motion identifiers and the 8 Z-axis motion identifiers determined in step 306 with the 8 X-axis motion identifiers, the 8 Y-axis motion identifiers and the 8 Z-axis motion identifiers of the preset motion trace, respectively.
  • the matching does not necessarily succeed at once after the one second of sampling.
  • the acceleration sensor may continue with sampling, new motion identifiers may be determined from the sampled data that have been obtained lately for the mobile terminal, and matching with the preset motion trace is made in step 308 again. Once the matching succeeds, the next step 310 is performed.
  • the matching in step 308 is performed in a manner of exact match or a manner of approximate matching in practical applications.
  • the manner of exact matching means that the 8 X-axis motion identifiers, the 8 Y-axis motion identifiers and the 8 Z-axis motion identifiers of an identified motion trace are the same as the 8 X-axis motion identifiers, the 8 Y-axis motion identifiers and the 8 Z-axis motion identifiers of a preset motion trace, respectively.
  • the manner of approximate matching means that the motion identifiers on the three axes are not completely the same but a number of pairs of motion identifiers being the same is larger than a preset matching threshold value.
  • a matching threshold value is preset as 6 and 7 motion identifiers on each of the axes of an identified motion trace are the same as those of a preset motion trace, respectively.
  • the number, 7, is larger than the matching threshold value 6. This would be a case of matching succeeding.
  • a time threshold value also can be set. Identification is not performed anymore (i.e., no return to step 300) to avoid an infinite loop when the threshold value is exceeded.
  • step 310 a control command mapped to the preset motion trace is executed to control an application in the mobile terminal, and this procedure is completed.
  • step 312 acceleration values are continuously collecting using an acceleration sensor 104 when a mobile terminal is being moved.
  • the same method as that for step 300 can be used for this step and will not be described herein.
  • step 314 a motion trace that the mobile terminal has been moved is identified from the collected acceleration values and the identified motion trace is displayed to the user through the display 106.
  • step 306 The same method as that in step 306 is used to identify the motion trace. Moreover, after the motion trace is identified, the motion trace is visualized as a character to be displayed to the user through the display 106 for the convenience of confirmation by the user.
  • step 316 whether a confirmation instruction is received from the user is determined, and if the confirmation instruction is received through the input/output interface 108, step 318 is performed. Otherwise, return to step 312.
  • step 318 the identified motion trace is saved and this procedure is completed.
  • a preset motion trace occurs in steps 312 to 318. If there is an application control-setting module in a mobile terminal, this module also can provide functions of displaying, searching for, associating and deleting a preset motion trace for more exact operation by a user. Steps 312 to 318 make a preset motion trace and are not necessary in an identifying process. Thus, they can be omitted in identifying a motion trace and controlling an application.
  • a database is provided to store corresponding relationships between preset motion traces and control commands.
  • the motion trace whose matching succeeds is searched for in the database and a corresponding control command is executed.
  • Figs. 5A, 5B , 6A, and 6B each show examples illustrating controlling an application in accordance with an embodiment of the present invention.
  • a user writes the character, "L”, with a handheld mobile terminal in the air.
  • a motion trace is formed during motion of the mobile terminal. Supposing that this motion trace is preset in the mobile terminal and is mapped to the control command of "Opening" the application “Notepad”, “Notepad” is automatically opened once the matching succeeds.
  • the result is shown in Fig. 5B .
  • a user writes the character, "Z", with a handheld mobile terminal in the air.
  • a motion trace is formed during motion of the mobile terminal. Assuming that this motion trace is preset in the mobile terminal and is mapped to the control command of "Opening” the application “instant messenger”, “instant messenger” is automatically opened once the matching succeeds. The result is shown in Fig. 6B .
  • a motion trace formed during motion of the mobile terminal is identified by means of an acceleration sensor 104 when a user writes with a handheld mobile terminal.
  • a corresponding control command is started when the identified motion trace succeeds in matching with a preset motion trace.
  • the user can write any characters and pictures not only simple shakes with the handheld mobile terminal.
  • a lot of motion traces can be formed and a lot of control commands can thereby be mapped. Consequently, it is possible to easily control various applications in the mobile terminal by a user writing with a handheld mobile terminal.
  • Embodiments of the present invention can be implemented by hardware, software, and a combination of the hardware and the software.
  • the software can be stored, regardless of possibility of omission or rewrite, in volatile or non-volatile storage devices such as memories such as a Read Only Memory (ROM), a Random Access Memory (RAM), a memory chip, a device or an integrated circuit, and optically or magnetically recordable and machine-readable storage media such as a Compact Disc (CD), a Digital Versatile Disc (DVD), a magnetic disk, a magnetic tape, and the like.
  • ROM Read Only Memory
  • RAM Random Access Memory
  • CD Compact Disc
  • DVD Digital Versatile Disc
  • a memory that can be included in the mobile terminal is an example of a program including instructions for implementation of the embodiments of the present invention or the machine-readable readable storage media suitable for storing the programs.
  • the present invention includes a program including a code in order to implement the system or the method stated in certain claims of the specification, and the machine-readable storage media, which store such a program. Further, the program can be electronically transferred by certain media such as a communication signal transmitted through a wired or wireless connection.
  • the method of controlling applications in the mobile terminal may be manufactured in a form of a program to be stored in a storage medium, and the mobile terminal can provide an application control to a user by downloading a program (e.g., an application) for controlling applications from a program-providing server, installing the downloaded program, and executing the installed program. That is, when the program for controlling the application is installed in the mobile terminal, the program is executed and then the application control is provided to the user.
  • the program-providing server receives a transmission request for the program from the mobile terminal and responds to the transmission request to transmit the program to the mobile terminal.
  • the program-providing server can determine whether the mobile terminal having made the transmission request for the program is a subscriber terminal and further perform a user authentication and a payment information identification.
  • the program-providing server can include a storage medium storing the program. Further, the storage medium may be located inside the program-providing server or outside the program-providing server.

Description

    BACKGROUND OF THE INVENTION 1. Field of the Invention
  • The present invention relates generally to mobile terminals and, more particularly, to a method and an apparatus for controlling applications in a mobile terminal.
  • 2. Background of the invention
  • Various mobile terminals such as a mobile terminal, a tablet computer, and a smart phone increasingly include more functions and, specifically, not only basic communications services, such as calling, but also various applications can be implemented. When an application is to be started or exited, a user performs the operation of start or exit by selecting the menus provided by the mobile terminal level by level. However, such operation is inconvenient.
  • Some high-end mobile terminals also allow users to control special functions in applications in the mobile terminals. For example, a user may zoom into a picture or control a game by shaking the terminal. However, all of these controls are simple. More and more demands on control of applications have been made. Thus, it is difficult for simple shaking of the terminal to meet more complex user demands.
  • US 2009/0303204 describes a technique for controlling and accessing content in a mobile device using motion processing. WO 2005/103863 describes a mobile device that recognizes a movement gesture of the mobile device and performs a function corresponding to the recognized movement gesture.
  • SUMMARY OF THE INVENTION
  • Accordingly, the present invention provides a method capable of easily controlling various applications of mobile terminals and the mobiles terminals.
  • The present invention also provides a method for controlling applications of mobile terminal by user writing with the mobile terminal and the mobile terminal
  • In accordance with an aspect of the present invention, there is provided a method of controlling applications in a mobile terminal, the method comprising: collecting acceleration values continuously using an acceleration sensor, when the mobile terminal is being moved; identifying a motion trace, from the collected acceleration values, indicating that the mobile terminal has been moved; matching the identified motion trace with a preset motion trace; and executing a control command mapped to the preset motion trace to control an application in the mobile terminal, if the matching succeeds; wherein each of the acceleration values is a vector including a value on an X-axis, a value on an Y-axis, and a value on an Z-axis, and wherein collecting the acceleration values comprises: performing sampling at a preset time interval, by the acceleration sensor, when the mobile terminal is moved, and storing in a first storage space including M groups of subspaces, values on the X-axis obtained in the sampling, storing in a second storage space including M groups of subspaces, values on the Y-axis obtained in the sampling, and storing in a third storage space including M groups of subspaces, values on the Z-axis obtained in the sampling, wherein M is a positive integer greater than or equal to 1; wherein identifying of the motion trace comprises: determining M respective X-axis motion identifiers based on values in the M groups of subspaces for the values in the first storage space, wherein each of the X-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in an X-axis positive direction, -1 denotes motion in an X-axis negative direction, and 0 denotes no motion on the X-axis; determining M respective Y-axis motion identifiers based on values in the M groups of subspaces for the values in the second storage space, wherein each of the Y-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in a Y-axis positive direction, -1 denotes motion in a Y-axis negative direction, and 0 denotes no motion on the Y-axis; and determining M respective Z-axis motion identifiers based on values in the M groups of subspaces for the values in the third storage space, wherein each of the Z-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in a Z-axis positive direction, -1 denotes motion in a Z-axis negative direction, and 0 denotes no motion on the Z-axis; wherein the M X-axis motion identifiers, the M Y-axis motion identifiers and the M Z-axis motion identifiers are used to represent a motion trace of motion of the mobile terminal; wherein the M groups of subspaces of the first storage space include N data units for respectively storing N values on the X-axis that have been obtained in sampling most recently, the M groups of subspaces of the second storage space include N data units for respectively storing N values on the Y-axis that have been obtained in sampling most recently, the M groups of subspaces of the third storage space include N data units for respectively storing N values on the Z-axis that have been obtained in sampling most recently, and wherein N is an integer greater than or equal to 1; wherein determining the M respective X-axis motion identifiers based on values in the M groups of subspaces, determining of the M respective Y-axis motion identifiers based on values in the M groups of subspaces, and determining of the M respective Z-axis motion identifiers based on values in the M groups of subspaces comprises: selecting a first group of subspaces as the current group; subtracting a still value from the N values stored in the current group and comparing the differences between the N values and the still value with a preset threshold value to obtain N comparison results, wherein the N comparison results are obtained as follows: if a difference is larger than the preset threshold value, the corresponding comparison result is 1; if a difference is smaller than the negative preset threshold value, the corresponding comparison result is -1; and if the absolute value of a difference is smaller than the preset threshold value, the corresponding comparison result is 0; counting the N comparison results of the current group and taking one of 1, -1 and 0 whose number of the N comparison results is the maximum as a motion identifier of a corresponding axis among the X-axis motion identifier, the Y-axis motion identifier, and the Z-axis motion identifier corresponding to the current group of subspace; and selecting the next group as the current group and returns to the second step unless all of the M groups of subspaces are processed.
  • In accordance with another aspect of the present invention, there is provided a mobile terminal comprising: an acceleration sensor for detecting an acceleration when the mobile terminal is being moved; a controller for collecting acceleration values using the acceleration sensor continuously when the mobile terminal is being moved, identifying a motion trace, from the collected acceleration values, indicating that the mobile terminal has been moved, matching the identified motion trace with a preset motion trace, and executing a control command mapped to the preset motion trace if the matching succeeds, to control applications in the mobile terminal; and a memory for providing first, second, and third storage spaces including respective M groups of subspaces, wherein the acceleration value is a vector including a value on the X-axis, a value on the Y-axis, and a value on the Z-axis and, when collecting the acceleration values, the controller is configured for performing sampling at a time interval preset by the acceleration sensor when the mobile terminal is being moved, storing values on the X-axis obtained during the sampling in the M groups of subspaces of the first storage space, storing values on the Y-axis obtained in the sampling in the M groups of subspaces of the second storage space, and storing values on the Z-axis obtained in the sampling in the M groups of subspaces of the third storage space, wherein M is a positive integer greater than or equal to 1; wherein the controller is configured for: determining M respective X-axis motion identifiers based on values in the M groups of subspaces for the values in the first storage space, when the motion trace is identified, wherein each of the X-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in an X-axis positive direction, -1 denotes motion in an X-axis negative direction, and 0 denotes no motion on the X-axis, determining M respective Y-axis motion identifiers based on values in the M groups of subspaces for the values in the second storage space, wherein each of the Y-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in Y-axis positive direction, -1 denotes motion in a Y-axis negative direction, and 0 denotes no motion on the Y-axis, and determining M respective Z-axis motion identifiers based on values in the M groups of subspaces for the values in the third storage space, wherein each of the Z-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in a Z-axis positive direction, -1 denotes motion in a Z-axis negative direction, and 0 denotes no motion on the Z-axis, wherein the M X-axis motion identifiers, the M Y-axis motion identifiers and the M Z-axis motion identifiers are used to represent a motion trace of motion of the mobile terminal; wherein the M groups of subspaces of the first storage space each include N data units for respectively storing N values on the X-axis that have been obtained in sampling most recently, the M groups of subspaces of the second storage space each include N data units for respectively storing N values on the Y-axis that have been obtained in sampling most recently and the M groups of subspaces of the third storage space each include N data units for respectively storing N values on the Z-axis that have been obtained in sampling most recently, wherein N is an integer greater than or equal to 1; wherein, when determining the M respective X-axis motion identifiers based on values in the M groups of subspaces, determining of the M respective Y-axis motion identifiers based on values in the M groups of subspaces, and determining of the M respective Z-axis motion identifiers based on values in the M groups of subspaces, respectively, the controller is configured for subtracting a still value from the N values stored in the respective M groups, comparing the differences between the N values and the still value with a preset threshold value to obtain N comparison results, counting the N comparison results of the current group, and taking one of 1, -1 and 0 whose number of the N comparison results is the maximum as a motion identifier of a corresponding axis among the X-axis motion identifier, the Y-axis motion identifier, and the Z-axis motion identifier corresponding to the current group of subspace, wherein the N comparison results are obtained as follows: if a difference is larger than the preset threshold value, the corresponding comparison result is 1; if a difference is smaller than the negative preset threshold value, the corresponding comparison result is -1; if the absolute value of a difference is smaller than the preset threshold value, the corresponding comparison result is 0.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The above and other aspects, features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:
    • Fig. 1 illustrates a mobile terminal according to embodiments of the present invention;
    • Fig. 2 illustrates a flowchart according to a first embodiment of the present invention;
    • Fig. 3 illustrates a flowchart according to a second embodiment of the present invention;
    • Fig. 4A illustrates queue 1 for storing sampled data on X-axis according to embodiments of the present invention;
    • Fig. 4B illustrates queue 2 for storing sampled data on Y-axis according to embodiments of the present invention;
    • Fig. 4C illustrates queue 3 for storing sampled data on Z-axis according to embodiments of the present invention;
    • Fig. 5A illustrates writing the character of "L" with a mobile terminal according to embodiments of the present invention;
    • Fig. 5B illustrates executing an application after the character "L" is written with a mobile terminal according to embodiments of the present invention;
    • Fig. 6A illustrates writing the character of "Z" with a mobile terminal according to embodiments of the present invention; and
    • Fig. 6B illustrates executing an application after the character "Z" is written with a mobile terminal according to embodiments of the present invention.
    DETAILED DESCRIPTION OF EMBODIMENTS OF THE PRESENT INVENTION
  • Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, a detailed explanation of known related functions and constitutions may be omitted to avoid unnecessarily obscuring the subject matter of the present invention.
  • Hereinafter, although embodiments of the present invention will be described with reference to a mobile terminal as an example, the embodiments of the present invention may be applied to other similar apparatuses. Further, embodiments of the present invention may be applied to other apparatuses by those skilled in the arts through some modifications without departing from the scope of the present invention.
  • FIG. 1 is a block diagram of a mobile terminal according to embodiments of the present invention. Referring to FIG.1, a controller 100 executes a function of a mobile terminal and also performs an application control according to embodiments of the present invention. When a user writes using the mobile terminal, the controller 100 identifies motion traces that the mobile terminal has been moved, from accelerations detected by an acceleration sensor 104. When the identified motion traces are matched with a preset motion trace, a corresponding application is controlled by executing a control command mapped to the preset motion trace.
  • A communication unit 102 performs communication for a data communication function and a phone call executed by the controller 100. The acceleration sensor 104 detects the acceleration when the mobile terminal moves. A display 106 displays a screen according to the movement of the mobile terminal by a control of the controller 100. An input/output interface 108 provides a user interface for an input and an output by the user for the controller 100. A memory 110 stores data according to an execution by the controller 100 and stores acceleration values detected by the acceleration sensor 104. The memory 110 may further include an external memory or further include a storage device such as a Hard Disk Drive (HDD).
  • FIG. 2 shows a flowchart according to an embodiment of the present invention. Referring to FIG. 2, the controller 100 continuously collects acceleration values using an acceleration sensor 104 when the mobile terminal is being moved in step 200.
  • In practical applications, accelerations occur in the mobile terminal in general when a user holds a mobile terminal and writes with the handheld mobile terminal in a space. The acceleration sensor 104 detects an acceleration of the mobile terminal. A detected acceleration value is a vector including scalar quantity values on the coordinate axes of a three-dimensional space, i.e., a value on the X-axis, the Y-axis and the Z axis.
  • In step 202, the controller 100 identifies a motion trace indicating that the mobile terminal has been moved from the collected acceleration values.
  • The detected acceleration values are representative of the accelerations of the mobile terminal and cannot represent a motion trace. However, a plurality of continuous accelerations can represent a certain trend of the motion of the mobile terminal. Thus, in this step, a motion trace of the mobile terminal can be identified.
  • This method is described by taking the X-axis as an example. Assuming that an acceleration value on the X-axis of the mobile terminal is detected as 2 when the mobile terminal is moved to point A, an acceleration value on the X-axis of the mobile terminal is detected as 4 when the mobile terminal is moved to point B, and an acceleration value on the X-axis of the mobile terminal is detected as 6 when the mobile terminal is moved to point C during continuous detection, it can be derived that the mobile terminal is moved in the X-axis positive direction at all of points A, B and C and is moved in the positive direction from point A to point B, and then point C. A motion trace described in the present invention means a shape of trace formed by motion trends, rather than a trace formed by positions in the three-dimensional space where the mobile terminal is. Description is given only by taking the X-axis as the sample. Processing on the Y-axis and the Z-axis is similar to that on the X-axis and will not be described herein.
  • The method of identifying a motion trace is described in detail below.
  • First, it is assumed that a first storage space, a second storage space and a third storage space provided by the memory 110 are used to store respective sampled values. Among these storage spaces, the first storage space includes M groups of subspaces to store M groups of values on the X-axis, the second storage space includes M groups of subspaces to store M groups of values on the Y-axis, and the third storage space includes M groups of subspaces to store M groups of values on the Z-axis. Thus, when step 200 is carried out, the acceleration sensor 104 performs sampling at a preset time interval. In other words, at a certain sampling frequency, when the mobile terminal is being moved and stores values on the X-axis obtained in the sampling in the M groups of subspaces of the first storage space, stores values on the Y-axis obtained in the sampling in the M groups of subspaces of the second storage space, and stores values on the Z-axis obtained in the sampling in the M groups of subspaces of the third storage space. Herein, M is a positive integer greater than or equal to 1.
  • With the sampled data stored in the above manner, the values on the X-axis, the Y-axis, and the Z-axis can be separately processed as follows to identify a motion trace.
    1. I) For values in the first storage space, i.e., collected values (scalar quantity values) on the X-axis, M respective X-axis motion identifiers are determined based on values in the M groups of subspaces. An X-axis motion identifier is 1, -1 or 0. 1 denotes motion in the X-axis positive direction, -1 denotes motion in the X-axis negative direction, and 0 denotes no motion on the X-axis.
    2. II) For values in the second storage space, i.e., collected values (scalar quantity values) on the Y-axis, M respective Y-axis motion identifiers are determined based on values in the M groups of subspaces. A Y-axis motion identifier is 1, -1 or 0. 1 denotes motion in the Y-axis positive direction, -1 denotes motion in the Y-axis negative direction, and 0 denotes no motion on the Y-axis.
    3. III) For values in the third storage space, i.e., collected values (scalar quantity values) on the Z-axis, M respective Z-axis motion identifiers are determined based on values in the M groups of subspaces. A Z-axis motion identifier is 1, -1 or 0. 1 denotes motion in the Z-axis positive direction, -1 denotes motion in the Z-axis negative direction, and 0 denotes no motion on the Z-axis.
  • That is, no matter whether the values on the X-axis are positive, negative or zero, they represent a motion trace formed of motion trends on the X-axis so long as they are converted into M motion identifiers. This is because, assuming that there is one piece of sampled data in each of the groups of subspaces and the groups of subspaces are stored successively in a certain order, e.g., an order of group 1 to group M, as described above, if a sampled value is positive, it is converted into a motion identifier 1, which represents motion in the X-axis positive direction; if a sampled data is negative, it is converted into a motion identifier -1, which represents motion in the X-axis negative direction; if a sampled data is 0, it is kept unchanged and represents no motion on the X-axis. Thus, M successive converted motion identifiers can represent a motion trend on the X-axis. For values on the Y-axis and Z-axis, they can be converted into motion identifiers based on the above principle to represent a motion trend on the Y-axis/Z-axis.
  • Specifically, at least the following two methods are available to perform the above determining of M motion identifiers.
  • A first method: directly determining M motion identifiers based on values in M groups of subspaces, respectively. This method is the same as that in the above example. As another example, supposing that there are 8 groups of subspaces in a first storage space and values in the 8 groups of subspaces are 1, 2, 4, -2, -4, 2, 3 and 4, respectively, a motion identifier corresponding to a positive number is set as 1, a motion identifier corresponding to a negative number is set as -1, while a motion identifier corresponding to 0 is set as 0. Thus, the motion identifiers corresponding to the 8 groups of subspaces in the first storage space are 1, 1, 1, -1, -1, 1, 1, and 1, respectively. Values in the second storage space or third storage space are processed in the same manner. In sum, in the first method, motion identifiers are determined directly based on values in M groups of subspaces. Moreover, in the above example, it is assumed that M is equal to 8 and there is only one sample value in each subspace, while in practical applications, M and a number of sampled value in each subspace vary with sampling methods.
  • A second method: determining M motion identifiers based on values in M groups of subspaces, a still value, and a threshold value. In this method, an acceleration sensor 104 determines the mobile terminal's not being moved as being moved by mistake due to factors such as inaccurate operation of the user and/or an error of the sensor and thereby obtain a certain sampled value when a user writes with a handheld mobile terminal. To eliminate this error, a still value representing the terminal not being moved and a threshold value are set. A sampled value is compared with the still value. If the difference between the sampled value and the still value falls within a range of the threshold value, the mobile terminal is regarded as not having been moved. If the difference falls outside the range of the threshold value, the mobile terminal is determined as having been moved in a positive or negative direction based on the particular case.
  • Additionally, in order to increase accuracy of the identification, a high sampling frequency is set for more samples. To make processing more convenient, N data units can be provided in each subspace to store sampled values that have been obtained. For example, each of the M groups of subspaces of a first storage space includes N data units for respectively storing N values on the X-axis that have been obtained in sampling; each of the M groups of subspaces of a second storage space includes N data units for respectively storing N values on the Y-axis that have been obtained in sampling; each of the M groups of subspaces of a third storage space includes N data units for respectively storing N values on the Z-axis that have been obtained in sampling. Herein, N is an integer greater than or equal to 1.
  • Thus, to convert the values stored in the M groups of subspaces of each of the first, second and third storage spaces, the second method performs the following steps.
  • For the X-axis, the method includes the following steps:
    1. x1. selecting a first group as the current group;
    2. x2. subtracting a first still value from the N values stored in the current group and comparing the differences between the N values and the first still value with a preset first threshold value to obtain N comparison results, wherein the N comparison results are obtained as follows. If a difference is larger than the preset first threshold value, the corresponding comparison result is 1; if a difference is smaller than the negative preset first threshold value, the corresponding comparison result is -1; if the absolute value of a difference is smaller than the preset first threshold value, the corresponding comparison result is 0;
    3. x3. counting the N comparison results of the current group and taking one of 1, - 1 and 0 whose number of the N comparison results is the maximum as the X-axis motion identifier corresponding to the current group of subspace;
    4. x4. selecting the next group as the current group and returning to x2 unless all of the M groups of subspaces are processed.
  • Similar steps y1 to y4 and z1 to z4 occur for the Y-axis and Z-axis, respectively.
  • As described above, the processing methods for values on the X-axis, Y-axis, and Z-axis are similar. All of these methods subtract a corresponding still value from the N data in a group of subspace, to compare the differences with a corresponding threshold value to determine whether the differences fall within or outside a range of the threshold, and to determine the motion identifier of the group of subspace based on results of the comparison. Since each storage space has M groups of subspaces, M motion identifiers are obtained in all of the these methods after the above processing is performed. The M motion identifiers can represent a motion trace on a certain axis as in the first method. Of course, a motion trace identified by the second method should be more accurate than that by the first method since factors such as inaccurate operation of a user and an error of the sensor is eliminated in the second method.
  • Moreover, all of the first still value, the second still value and the third still value represent no motion. They are the same or different. Similarly, each of the first threshold value, the second threshold value and the third threshold value represents a certain range.
  • Step 204, the controller 100 matches the identified motion trace with a preset motion trace, and executes a control command mapped to the preset motion trace to control an application in the mobile terminal if the matching succeeds.
  • This step compares the X-axis motion identifier, the Y-axis motion identifier and the Z-axis motion identifier and determines whether the matching has succeeded based on the results of the comparisons. The control command mapped is automatically executed and thereby the application in the mobile terminal is controlled if the matching succeeds.
  • In another embodiment of the present invention, it is assumed that a sampling frequency of the acceleration sensor 104 in a mobile terminal is 400 Hz, i.e., the acceleration sensor collects data once every 0.0025 seconds. That is, every second, 400 pieces of data on the X-axis, 400 pieces of data on the Y-axis and 400 pieces data on the Z-axis are collected. It is also assumed that each of a first storage space, a second storage space and a third storage space to store sampled data includes M groups of subspaces, each of which includes N data units. Herein, M = 8, and N = 50. In this embodiment, the first storage space, the second storage space and the third storage space are shown using queue 1, queue 2, queue 3, respectively. Each of the queues has 8 nodes and each of the nodes includes 50 data units. Moreover, the sampled data is stored in an order of queue head to queue tail when sampled data are stored. The data at the queue head is deleted and a node storing the new sampled data is inserted to the queue tail when data units of all of the nodes are occupied. That is, in this embodiment, each of the queues always stores the 400 sampled data that have been last obtained. Sampled data is stored in another data structure other than a queue.
  • Moreover, it is also assumed in this embodiment that the controller 100 has an application control-setting module that operates in two modes, i.e., a motion trace-presetting mode and a motion trace-identifying mode.
  • FIG. 3 shows a flowchart of the method of this embodiment. Referring to FIG. 3, the controller 100 allows the acceleration sensor 104 to perform sampling at a rate of 0.0025 second and to store values on the X-axis obtained in the sampling in queue 1, store values on the Y-axis obtained in the sampling in queue 2, while store values on the Z-axis obtained in the sampling in queue 3, when the mobile terminal is being moved in step 300.
  • Queues 1, 2 and 3 in this step is shown as in Figs. 4A to 4C, respectively. As described above, the sampled data is stored in the order of queue head to queue tail. For example, the first group of 50 data are placed at the head node of a queue when 50 pieces sampled data are obtained. After 0.125 seconds (50 * 0.0025 seconds), the second group of 50 sampled data is stored at the second node. After another 0.125 seconds, the third group of 50 sampled data is stored at the third node, and so on. When all of 8 nodes of a queue are full, data at the head node of the queue is deleted and the node storing the 50 sampled data that have been recently obtained is stored at the tail of the queue. In sum, each of queues 1, 2 and 3 always stores the 400 pieces of data that have been obtained lately.
  • In step 302, the controller 100 determines whether the application control-setting module is running, and if so, step 304 is performed. Otherwise, step 306 is performed.
  • In step 304, the controller 100 determines the current mode for the application control-setting module. If the current mode is the motion trace-presetting mode, step 312 is performed. If the current mode is the motion trace-identifying mode, step 306 is performed.
  • In this embodiment, the application control-setting module is provided in the mobile terminal and the function of motion trace presetting or motion trace identifying is started by the application control-setting module. However, the application control-setting module is not provided in practical applications. Whether the application control-setting module is present does not affect the implementation of the solution, and steps 302 and 304 may be omitted.
  • In step 306, 8 X-axis motion identifiers are determined from the values in queue 1, 8 Y-axis motion identifiers are determined from the values in queue 2, and 8 Z-axis motion identifiers are determined from the values in queue 3.
  • Determining the 8 X-axis motion identifiers can be done in the method of the above steps x1 to x4, determining of the 8 Y-axis motion identifiers can be done in the method of the above steps y1 to y4, and determining of the 8 Z-axis motion identifiers can be done in the method of the above steps z1 to z4, all of which will not be described herein.
  • All of the nodes of a queue do not store sampled data when sampling has just started. For example, only the head node of the queue stores sampled data while none of the other nodes stores data at 0.125 second. Thus, only the motion identifier corresponding to the head node of the queue can be determined. After 0.125 second, the second node of the queue also stored sampled data as described in step 300. Thus, the motion identifier corresponding to the second node can be determined. Herein, the queue and the motion identifiers are not specified and can be for the X-axis, Y-axis or Z-axis.
  • Additionally, since an error may occur during operation, a first still value, a second still value and a third still value are to be further updated in this step when the 8 X-axis motion identifiers, the 8 Y-axis motion identifiers and the 8 Z-axis motion identifiers are determined. That is, between steps x3 and x4, the method further includes the following step to be performed when the 8 X-axis motion identifiers are determined. Determining whether the X-axis motion identifier corresponding to the current group is 0 and if so, calculating a mean of the N values (N values on the X-axis) of the current group and updating the first still value as the calculated mean; otherwise, returning to step x4. A similar step can occur between steps y3 and y4 and z3 and z4.
  • These steps are used to update the still values because during the operation by a user, a mobile terminal is substantially still if a motion identifier corresponding to the current group of data is 0. However, all of the N (in this embodiment, N is 50) pieces of data of the current group on each of the axes are not necessarily equal to the still value due to an error of operation or measurement. Thus, if the still value is set as a mean of the N values, the N values as a whole are closer to the still value, thereby, avoiding the case that the mobile terminal is not being moved is mistaken for being moved.
  • In step 308, the identified motion trace is matched with a preset motion trace, and step 310 is performed if the matching succeeds. Otherwise, the process returns to step 300.
  • Steps 300 to 308 form a large cycle, in which the acceleration sensor 104 in the mobile terminal performs sampling continuously and accesses the queues every 0.125 second. Accordingly, a motion trace is identified for the mobile terminal every 0.125 second. The identification may fail within the first one second from when sampling is started since all of the nodes in a queue do not store data and thus the determined 8 motion identifiers are inaccurate. However, all the nodes in the queue store sampled data after the one second. Thereby, effective 8 motion identifiers can be determined. As described in an embodiment above, X-axis motion identifiers, Y-axis motion identifiers and Z-axis motion identifiers represent a motion trace that a mobile terminal has been moved. In this embodiment, a preset motion trace is also represented using X-axis motion identifiers, Y-axis motion identifiers and Z-axis motion identifiers. Thus, whether the matching succeeds is determined by comparing the 8 X-axis motion identifiers, the 8 Y-axis motion identifiers and the 8 Z-axis motion identifiers determined in step 306 with the 8 X-axis motion identifiers, the 8 Y-axis motion identifiers and the 8 Z-axis motion identifiers of the preset motion trace, respectively.
  • The matching does not necessarily succeed at once after the one second of sampling. However, as described in step 300, the acceleration sensor may continue with sampling, new motion identifiers may be determined from the sampled data that have been obtained lately for the mobile terminal, and matching with the preset motion trace is made in step 308 again. Once the matching succeeds, the next step 310 is performed.
  • The matching in step 308 is performed in a manner of exact match or a manner of approximate matching in practical applications. The manner of exact matching means that the 8 X-axis motion identifiers, the 8 Y-axis motion identifiers and the 8 Z-axis motion identifiers of an identified motion trace are the same as the 8 X-axis motion identifiers, the 8 Y-axis motion identifiers and the 8 Z-axis motion identifiers of a preset motion trace, respectively. The manner of approximate matching means that the motion identifiers on the three axes are not completely the same but a number of pairs of motion identifiers being the same is larger than a preset matching threshold value. For example, a matching threshold value is preset as 6 and 7 motion identifiers on each of the axes of an identified motion trace are the same as those of a preset motion trace, respectively. In this case, the number, 7, is larger than the matching threshold value 6. This would be a case of matching succeeding.
  • A time threshold value also can be set. Identification is not performed anymore (i.e., no return to step 300) to avoid an infinite loop when the threshold value is exceeded.
  • In step 310, a control command mapped to the preset motion trace is executed to control an application in the mobile terminal, and this procedure is completed.
  • In step 312, acceleration values are continuously collecting using an acceleration sensor 104 when a mobile terminal is being moved. The same method as that for step 300 can be used for this step and will not be described herein.
  • In step 314, a motion trace that the mobile terminal has been moved is identified from the collected acceleration values and the identified motion trace is displayed to the user through the display 106.
  • The same method as that in step 306 is used to identify the motion trace. Moreover, after the motion trace is identified, the motion trace is visualized as a character to be displayed to the user through the display 106 for the convenience of confirmation by the user.
  • In step 316, whether a confirmation instruction is received from the user is determined, and if the confirmation instruction is received through the input/output interface 108, step 318 is performed. Otherwise, return to step 312.
  • In step 318, the identified motion trace is saved and this procedure is completed.
  • A preset motion trace occurs in steps 312 to 318. If there is an application control-setting module in a mobile terminal, this module also can provide functions of displaying, searching for, associating and deleting a preset motion trace for more exact operation by a user. Steps 312 to 318 make a preset motion trace and are not necessary in an identifying process. Thus, they can be omitted in identifying a motion trace and controlling an application.
  • It is easy to bind a control command to a preset motion trace in applications. For example, a database is provided to store corresponding relationships between preset motion traces and control commands. Thus, after matching succeeds in step 308, the motion trace whose matching succeeds is searched for in the database and a corresponding control command is executed.
  • Figs. 5A, 5B, 6A, and 6B each show examples illustrating controlling an application in accordance with an embodiment of the present invention. As shown in Fig. 5A, a user writes the character, "L", with a handheld mobile terminal in the air. A motion trace is formed during motion of the mobile terminal. Supposing that this motion trace is preset in the mobile terminal and is mapped to the control command of "Opening" the application "Notepad", "Notepad" is automatically opened once the matching succeeds. The result is shown in Fig. 5B.
  • In another example, as shown in Fig. 6A, a user writes the character, "Z", with a handheld mobile terminal in the air. A motion trace is formed during motion of the mobile terminal. Assuming that this motion trace is preset in the mobile terminal and is mapped to the control command of "Opening" the application "instant messenger", "instant messenger" is automatically opened once the matching succeeds. The result is shown in Fig. 6B.
  • By applying this embodiment, a motion trace formed during motion of the mobile terminal is identified by means of an acceleration sensor 104 when a user writes with a handheld mobile terminal. A corresponding control command is started when the identified motion trace succeeds in matching with a preset motion trace. Thus, the objective of controlling an application is achieved. The user can write any characters and pictures not only simple shakes with the handheld mobile terminal. Thus, a lot of motion traces can be formed and a lot of control commands can thereby be mapped. Consequently, it is possible to easily control various applications in the mobile terminal by a user writing with a handheld mobile terminal.
  • Embodiments of the present invention can be implemented by hardware, software, and a combination of the hardware and the software. For example, the software can be stored, regardless of possibility of omission or rewrite, in volatile or non-volatile storage devices such as memories such as a Read Only Memory (ROM), a Random Access Memory (RAM), a memory chip, a device or an integrated circuit, and optically or magnetically recordable and machine-readable storage media such as a Compact Disc (CD), a Digital Versatile Disc (DVD), a magnetic disk, a magnetic tape, and the like. A memory that can be included in the mobile terminal, is an example of a program including instructions for implementation of the embodiments of the present invention or the machine-readable readable storage media suitable for storing the programs. Therefore, the present invention includes a program including a code in order to implement the system or the method stated in certain claims of the specification, and the machine-readable storage media, which store such a program. Further, the program can be electronically transferred by certain media such as a communication signal transmitted through a wired or wireless connection.
  • The method of controlling applications in the mobile terminal according to embodiments of the present invention may be manufactured in a form of a program to be stored in a storage medium, and the mobile terminal can provide an application control to a user by downloading a program (e.g., an application) for controlling applications from a program-providing server, installing the downloaded program, and executing the installed program. That is, when the program for controlling the application is installed in the mobile terminal, the program is executed and then the application control is provided to the user. The program-providing server receives a transmission request for the program from the mobile terminal and responds to the transmission request to transmit the program to the mobile terminal. The program-providing server can determine whether the mobile terminal having made the transmission request for the program is a subscriber terminal and further perform a user authentication and a payment information identification. The program-providing server can include a storage medium storing the program. Further, the storage medium may be located inside the program-providing server or outside the program-providing server.
  • While the present invention has been shown and described with reference to certain embodiments and drawings thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention as defined by the appended claims.

Claims (7)

  1. A method of controlling applications in a mobile terminal (500), the method comprising:
    collecting (200) acceleration values continuously using an acceleration sensor (104), when the mobile terminal (500) is being moved;
    identifying (202) a motion trace, from the collected acceleration values, indicating that the mobile terminal (500) has been moved;
    matching (204) the identified motion trace with a preset motion trace; and
    executing (204) a control command mapped to the preset motion trace to control an application in the mobile terminal (500), if the matching succeeds;
    wherein each of the acceleration values is a vector including a value on an X-axis, a value on an Y-axis, and a value on an Z-axis, and wherein collecting (200) the acceleration values comprises:
    performing sampling (300) at a preset time interval, by the acceleration sensor, when the mobile terminal is moved, and storing (300) in a first storage space (110) including M groups of subspaces, values on the X-axis obtained in the sampling, storing (300) in a second storage space (110) including M groups of subspaces, values on the Y-axis obtained in the sampling, and storing (300) in a third storage space (110) including M groups of subspaces, values on the Z-axis obtained in the sampling,
    wherein M is a positive integer greater than or equal to 1;
    characterised in that
    identifying (202) of the motion trace comprises:
    determining (306) M respective X-axis motion identifiers based on values in the M groups of subspaces for the values in the first storage space (110), wherein each of the X-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in an X-axis positive direction, -1 denotes motion in an X-axis negative direction, and 0 denotes no motion on the X-axis;
    determining (306) M respective Y-axis motion identifiers based on values in the M groups of subspaces for the values in the second storage space (110), wherein each of the Y-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in a Y-axis positive direction, -1 denotes motion in a Y-axis negative direction, and 0 denotes no motion on the Y-axis; and
    determining (306) M respective Z-axis motion identifiers based on values in the M groups of subspaces for the values in the third storage space (110), wherein each of the Z-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in a Z-axis positive direction, -1 denotes motion in a Z-axis negative direction, and 0 denotes no motion on the Z-axis;
    wherein the M X-axis motion identifiers, the M Y-axis motion identifiers and the M Z-axis motion identifiers are used to represent a motion trace of motion of the mobile terminal (500);
    wherein
    the M groups of subspaces of the first storage space (110) each include N data units for respectively storing N values on the X-axis that have been obtained in sampling most recently,
    the M groups of subspaces of the second storage space (110) each include N data units for respectively storing N values on the Y-axis that have been obtained in sampling most recently,
    the M groups of subspaces of the third storage space (110) each include N data units for respectively storing N values on the Z-axis that have been obtained in sampling most recently, and
    wherein N is an integer greater than or equal to 1;
    wherein determining (306) the M respective X-axis motion identifiers based on values in the M groups of subspaces, determining (306) of the M respective Y-axis motion identifiers based on values in the M groups of subspaces, and determining (306) of the M respective Z-axis motion identifiers based on values in the M groups of subspaces comprises:
    selecting a first group of subspaces as the current group;
    subtracting a still value from the N values stored in the current group and comparing the differences between the N values and the still value with a preset threshold value to obtain N comparison results, wherein the N comparison results are obtained as follows: if a difference is larger than the preset threshold value, the corresponding comparison result is 1; if a difference is smaller than the negative preset threshold value, the corresponding comparison result is -1; and if the absolute value of a difference is smaller than the preset threshold value, the corresponding comparison result is 0;
    counting the N comparison results of the current group and taking one of 1, -1 and 0 whose number of the N comparison results is the maximum as a motion identifier of a corresponding axis among the X-axis motion identifier, the Y-axis motion identifier, and the Z-axis motion identifier corresponding to the current group of subspace; and
    selecting the next group as the current group and returns to the second step unless all of the M groups of subspaces are processed.
  2. The method of claim 1, further comprising:
    determining whether the motion identifier of the corresponding axis corresponding to the current group is 0; and
    calculating a mean of the N values of the current group and updating the still value as the calculated mean.
  3. The method of claim 1, further comprising:
    presetting (312-318) a motion trace for the mobile terminal (500), and binding the preset motion trace to a control command of an application, before collecting (200) the acceleration values.
  4. The method of claim 3, wherein presetting a motion trace for the mobile terminal (500) comprises:
    collecting (312) acceleration values using the acceleration sensor (104) continuously when the mobile terminal (500) is being moved;
    identifying (314) a motion trace that the mobile terminal (500) has been moved from the collected acceleration values, and displaying (314) the identified motion trace to a user; and
    saving (318) the identified motion trace if a confirmation instruction is received from the user; otherwise, returning to the first step (312).
  5. A mobile terminal (500) comprising:
    an acceleration sensor (104) for detecting an acceleration when the mobile terminal (500) is being moved;
    a controller (100) for collecting acceleration values using the acceleration sensor (104) continuously when the mobile terminal (500) is being moved, identifying a motion trace, from the collected acceleration values, indicating that the mobile terminal (500) has been moved, matching the identified motion trace with a preset motion trace, and executing a control command mapped to the preset motion trace if the matching succeeds, to control applications in the mobile terminal (500); and
    a memory (110) for providing first, second, and third storage spaces including respective M groups of subspaces,
    wherein the acceleration value is a vector including a value on the X-axis, a value on the Y-axis, and a value on the Z-axis and, when collecting the acceleration values, the controller is configured for performing sampling at a time interval preset by the acceleration sensor when the mobile terminal (500) is being moved, storing values on the X-axis obtained during the sampling in the M groups of subspaces of the first storage space, storing values on the Y-axis obtained in the sampling in the M groups of subspaces of the second storage space, and storing values on the Z-axis obtained in the sampling in the M groups of subspaces of the third storage space,
    wherein M is a positive integer greater than or equal to 1;
    characterised in that
    the controller (100) is configured for:
    determining M respective X-axis motion identifiers based on values in the M groups of subspaces for the values in the first storage space, when the motion trace is identified, wherein each of the X-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in an X-axis positive direction, -1 denotes motion in an X-axis negative direction, and 0 denotes no motion on the X-axis,
    determining M respective Y-axis motion identifiers based on values in the M groups of subspaces for the values in the second storage space, wherein each of the Y-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in Y-axis positive direction, -1 denotes motion in a Y-axis negative direction, and 0 denotes no motion on the Y-axis, and
    determining M respective Z-axis motion identifiers based on values in the M groups of subspaces for the values in the third storage space, wherein each of the Z-axis motion identifiers is 1, -1 or 0, wherein 1 denotes motion in a Z-axis positive direction, -1 denotes motion in a Z-axis negative direction, and 0 denotes no motion on the Z-axis,
    wherein the M X-axis motion identifiers, the M Y-axis motion identifiers and the M Z-axis motion identifiers are used to represent a motion trace of motion of the mobile terminal (500);
    wherein the M groups of subspaces of the first storage space (110) each include N data units for respectively storing N values on the X-axis that have been obtained in sampling most recently, the M groups of subspaces of the second storage space (110) each include N data units for respectively storing N values on the Y-axis that have been obtained in sampling most recently and the M groups of subspaces of the third storage space (110) each include N data units for respectively storing N values on the Z-axis that have been obtained in sampling most recently,
    wherein N is an integer greater than or equal to 1;
    wherein, when determining the M respective X-axis motion identifiers based on values in the M groups of subspaces, determining of the M respective Y-axis motion identifiers based on values in the M groups of subspaces, and determining of the M respective Z-axis motion identifiers based on values in the M groups of subspaces, respectively, the controller (100) is configured for subtracting a still value from the N values stored in the respective M groups, comparing the differences between the N values and the still value with a preset threshold value to obtain N comparison results, counting the N comparison results of the current group, and taking one of 1, -1 and 0 whose number of the N comparison results is the maximum as a motion identifier of a corresponding axis among the X-axis motion identifier, the Y-axis motion identifier, and the Z-axis motion identifier corresponding to the current group of subspace,
    wherein the N comparison results are obtained as follows: if a difference is larger than the preset threshold value, the corresponding comparison result is 1; if a difference is smaller than the negative preset threshold value, the corresponding comparison result is -1; if the absolute value of a difference is smaller than the preset threshold value, the corresponding comparison result is 0.
  6. The mobile terminal of claim 5, wherein the controller (100) is configured for calculating a mean of the N values of the current group, and updating the still value as the calculated mean, if the motion identifier on the corresponding axis is 0.
  7. The mobile terminal of claim 5, wherein the controller (100) is configured for presetting a motion trace of the mobile terminal, and mapping the preset motion trace to a control command of an application, before the acceleration values are continuously collected.
EP12155829.0A 2011-02-16 2012-02-16 Method and mobile terminal for controlling application in the mobile terminal Not-in-force EP2490104B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2011100413284A CN102647504B (en) 2011-02-16 2011-02-16 Method for controlling applications in mobile phone

Publications (2)

Publication Number Publication Date
EP2490104A1 EP2490104A1 (en) 2012-08-22
EP2490104B1 true EP2490104B1 (en) 2015-01-07

Family

ID=45607095

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12155829.0A Not-in-force EP2490104B1 (en) 2011-02-16 2012-02-16 Method and mobile terminal for controlling application in the mobile terminal

Country Status (4)

Country Link
US (1) US8914010B2 (en)
EP (1) EP2490104B1 (en)
KR (1) KR101892065B1 (en)
CN (1) CN102647504B (en)

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103677312B (en) * 2012-09-04 2017-05-03 中国移动通信集团公司 Mobile terminal function execution method and device based on accelerated speed inductor
CN102915184B (en) * 2012-09-18 2017-04-05 百度在线网络技术(北京)有限公司 For the interaction control method of mobile terminal, device and mobile terminal
CN103685712A (en) * 2012-09-25 2014-03-26 腾讯科技(深圳)有限公司 A method for refreshing a browser page of a mobile terminal and a mobile terminal
CN102929408A (en) * 2012-10-09 2013-02-13 广东欧珀移动通信有限公司 Method and device for quickly opening application
CN103326750A (en) * 2012-12-07 2013-09-25 天津唐虞科技发展有限公司 System based on Bluetooth wireless control display
CN102945089A (en) * 2012-12-07 2013-02-27 南京大学 Smart phone mouse moving method based on finite state machine and classifier
CN103338062A (en) * 2012-12-07 2013-10-02 天津唐虞科技发展有限公司 Mobile phone host terminal for controlling display system wirelessly based on Bluetooth
CN103354462A (en) * 2012-12-07 2013-10-16 天津唐虞科技发展有限公司 Method for controlling display wirelessly based on Bluetooth
CN103326751A (en) * 2012-12-07 2013-09-25 天津唐虞科技发展有限公司 Display client end based on Bluetooth wireless control display method and system
CN103152474A (en) * 2013-01-30 2013-06-12 深圳市金立通信设备有限公司 Mobile terminal recording method and mobile terminal
CN103227856A (en) * 2013-03-22 2013-07-31 中兴通讯股份有限公司 Non-touch dialing method and non-touch dialing device
CN103543906B (en) * 2013-03-27 2017-02-15 Tcl集团股份有限公司 Application icon interaction management method and touch terminal
CN104580089A (en) 2013-10-18 2015-04-29 深圳市腾讯计算机系统有限公司 User verification method and mobile terminal
CN104750239B (en) * 2013-12-30 2017-12-19 中国移动通信集团公司 A kind of method and apparatus of the application accessed based on space gesture in terminal device
CN103731561A (en) * 2014-01-14 2014-04-16 上海斐讯数据通信技术有限公司 Method for automatically switching telephone answering mode
CN104571876A (en) * 2014-12-11 2015-04-29 深圳市金立通信设备有限公司 Terminal
CN104571499A (en) * 2014-12-11 2015-04-29 深圳市金立通信设备有限公司 Method for controlling terminal and terminal
CN105786333B (en) * 2014-12-18 2019-03-12 黄大卫 The operating method and device of mobile terminal
CN104657638B (en) * 2015-01-22 2017-11-10 成都朝越科技有限公司 A kind of mobile phone unlocking method based on motion feature
CN106201239A (en) * 2015-05-08 2016-12-07 中兴通讯股份有限公司 The control method of a kind of application program, device and mobile terminal
CN106325467B (en) * 2015-06-15 2021-10-29 中兴通讯股份有限公司 Method and device for controlling mobile terminal and mobile terminal
CN106295423A (en) * 2015-06-29 2017-01-04 阿里巴巴集团控股有限公司 A kind of method for exhibiting data and client
CN107169769A (en) * 2016-03-08 2017-09-15 广州市动景计算机科技有限公司 The brush amount recognition methods of application program, device
TWI611321B (en) * 2016-04-29 2018-01-11 和碩聯合科技股份有限公司 Portable electronic device and control method thereof
CN108646931B (en) * 2018-03-21 2022-10-14 深圳市创梦天地科技有限公司 Terminal control method and terminal
CN109757449A (en) * 2019-02-26 2019-05-17 深圳市百泰实业股份有限公司 Float for fishing system
CN112437231B (en) * 2020-11-24 2023-11-14 维沃移动通信(杭州)有限公司 Image shooting method and device, electronic equipment and storage medium

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007531113A (en) * 2004-03-23 2007-11-01 富士通株式会社 Identification of mobile device tilt and translational components
CN100531424C (en) * 2006-08-01 2009-08-19 华为技术有限公司 Method and device for realizing mobile terminal step function
US8462109B2 (en) * 2007-01-05 2013-06-11 Invensense, Inc. Controlling and accessing content using motion processing on mobile devices
KR101460609B1 (en) * 2008-05-23 2014-11-14 삼성전자주식회사 Method and apparatus for setting telephony mode of mobile terminal
CN101399861B (en) * 2008-10-31 2012-09-05 华为终端有限公司 Vibration regulating device, method for a mobile communication terminal, and mobile communication terminal thereof
JP5176943B2 (en) * 2008-12-25 2013-04-03 富士通モバイルコミュニケーションズ株式会社 Information processing device
CN101494674A (en) * 2009-02-27 2009-07-29 深圳华为通信技术有限公司 Method and apparatus for inputting mobile communication terminal information
CN101576954A (en) * 2009-06-10 2009-11-11 中兴通讯股份有限公司 Stroke writing identification device, mobile terminal and method for realizing spatial writing
CN101674368A (en) * 2009-09-25 2010-03-17 宇龙计算机通信科技(深圳)有限公司 Incoming call processing method, system and mobile terminal
CN101866667A (en) * 2010-02-09 2010-10-20 宇龙计算机通信科技(深圳)有限公司 Music playing method and mobile terminal
CN101893988B (en) * 2010-06-09 2013-10-09 华为终端有限公司 Mobile communication terminal for hand input and input method thereof
CN101902526B (en) * 2010-07-21 2013-10-02 惠州Tcl移动通信有限公司 Mobile terminal and volume control method and device thereof

Also Published As

Publication number Publication date
KR101892065B1 (en) 2018-08-28
CN102647504B (en) 2013-07-17
US8914010B2 (en) 2014-12-16
US20120208555A1 (en) 2012-08-16
EP2490104A1 (en) 2012-08-22
CN102647504A (en) 2012-08-22
KR20120094447A (en) 2012-08-24

Similar Documents

Publication Publication Date Title
EP2490104B1 (en) Method and mobile terminal for controlling application in the mobile terminal
CN108615247B (en) Method, device and equipment for relocating camera attitude tracking process and storage medium
EP3276950B1 (en) Electronic device for providing slow motion video content
CN108463832B (en) Electronic device and process execution method based on hardware diagnosis result
CN108415739B (en) Hook method and device for dynamic link library function and storage medium
EP2614418B1 (en) Method of operating mobile device by recognizing user's gesture and mobile device using the method
CN102346859B (en) Character recognition device
US10126825B2 (en) Method for recognizing handwriting on a physical surface
EP3480790B1 (en) Method and apparatus for continuously displaying images on basis of similarity of images
JP2014038527A (en) Input device, input method and program
CN109922298A (en) Meeting room monitoring method and device
CN108509348B (en) System aging test method and mobile terminal
CN111797861A (en) Information processing method, information processing apparatus, storage medium, and electronic device
WO2023279970A1 (en) Blockchain-based data synchronization method and apparatus
CN111796925A (en) Method and device for screening algorithm model, storage medium and electronic equipment
CN108234758B (en) Application display method and device, storage medium and electronic equipment
CN108604142B (en) Touch screen device operation method and touch screen device
CN113342170A (en) Gesture control method, device, terminal and storage medium
CN107135312B (en) Method and terminal for shaking dialing
CN104714736A (en) Control method and terminal for quitting full screen lock-out state
CN111797867A (en) System resource optimization method and device, storage medium and electronic equipment
CN111796701A (en) Model training method, operation processing method, device, storage medium and equipment
US20130147701A1 (en) Methods and devices for identifying a gesture
CN113706606A (en) Method and device for determining position coordinates of spaced gestures
US20180300896A1 (en) Determining the orientation of image data based on user facial position

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SAMSUNG ELECTRONICS CO., LTD.

17P Request for examination filed

Effective date: 20130222

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: G06F 3/0346 20130101ALI20140722BHEP

Ipc: G06F 1/16 20060101ALI20140722BHEP

Ipc: H04M 1/725 20060101ALI20140722BHEP

Ipc: G06F 3/01 20060101AFI20140722BHEP

INTG Intention to grant announced

Effective date: 20140822

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 706144

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150215

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012004709

Country of ref document: DE

Effective date: 20150226

REG Reference to a national code

Ref country code: NL

Ref legal event code: T3

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 706144

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150107

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150407

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150407

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150507

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150408

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012004709

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150228

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150228

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20151030

26N No opposition filed

Effective date: 20151008

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150309

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150507

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20200123

Year of fee payment: 9

Ref country code: NL

Payment date: 20200121

Year of fee payment: 9

Ref country code: DE

Payment date: 20200121

Year of fee payment: 9

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602012004709

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20210216

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20210301

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210301

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210901

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210216