JP2013077180A - Recognition device and method for controlling the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve an orbit input operation following the intention of a user even on the rear face (back face) of equipment (casing).SOLUTION: The recognition device includes: a front input part and a back input part for detecting an orbit input by a user. The recognition device includes: acquisition means for acquiring an orbit input from the front input part or the back input part; specification means for specifying from which of the two input parts has the acquired orbit been input; and recognition means for recognizing the acquired orbit. When it is specified that the acquired orbit has been input from the back input part, the recognition means performs normalization processing accompanied by the inversion processing of the acquired orbit, and then executes the recognition of the orbit by using the same recognition algorithm as that in the case of recognizing the orbit input from the front input part.

Description

  The present invention relates to a technique for recognizing a trace drawn using a user's finger or stylus.

In recent years, an increasing number of mobile phones, game machines, and the like adopt a touch operation as a user interface (UI). As one of the touch operations, there is a “handwriting locus input operation” for inputting characters and gesture locus. In this operation, since the device recognizes the trajectory, it is needless to say that the direction in which the touching finger moves is important.
In addition, there are known devices in which a single device has a screen on which a plurality of touch operations can be performed.

JP 2010-26064 A

Consider a case where a touch panel is present on the rear surface (rear surface) of the screen corresponding to the front surface of the device (housing). In this case, when the user tries to write a trajectory such as characters on the back side while looking at the screen, the left and right (or top and bottom) that the user is conscious of and the left and right positions of the actual back side screen are reversed (so-called mirror image) turn into. Therefore, when operating such a device, it is impossible to input a handwritten locus on the back surface (back surface) in accordance with the user's intention.
The present invention has been made in order to solve the above-described problems, and has as its main object to enable operation of locus input in accordance with the user's intention even on the back surface (back surface) of the device (housing). To do.

  In order to solve the above problems, according to the handwriting recognition device of the present invention, a recognition device including a front input unit and a rear input unit for detecting a locus input by a user, the front input unit or the rear input unit. An acquisition means for acquiring a trajectory input from, a specifying means for specifying from which of the two input sections the acquired trajectory, and recognition for recognizing the acquired trajectory And when the acquired trajectory is input from the back surface input unit, the recognizing unit performs normalization processing that includes reversal processing of the acquired trajectory, and then from the front input unit. Trajectory recognition is performed using the same recognition algorithm as the input trajectory recognition.

  According to the present invention, it is possible to perform a locus input operation in accordance with the user's intention even on the back surface (back surface) of the device.

Functional configuration diagram of handwriting recognition device Flow chart showing the handwriting recognition process Diagram showing general handwriting trajectory recognition External view of a portable terminal incorporating a handwriting recognition device Rear input example Example of handwriting trajectory by back side input Functional configuration diagram of handwriting recognition device Example of handwritten locus pattern The figure explaining the distortion which occurs by the back input Handwritten locus correction table External view of mobile terminal Functional configuration diagram of handwriting recognition device Flowchart of handwriting locus recognition processing by back side input Functional configuration diagram of handwriting recognition device

<Embodiment 1>
FIG. 4 is an external view of a device (handwriting recognition device) applicable in the present embodiment. The size of the device shown here may be assumed to be, for example, the palm of the user or a size slightly larger than that.

  On the front surface (front surface) 401 of the device (housing), a GUI 402 is displayed, and a touch panel 403 capable of touching the GUI 402 is provided.

  Furthermore, the back surface (back surface) 404 of the device (housing) is provided with a touch pad 405 that allows touch operation. The user can operate the GUI 402 displayed on the surface 401 by touching the touch pad 405. That is, the GUI 402 displayed on the device can be operated from the touch panel 403 on the front surface 401 and can also be operated from the touch pad 405 on the back surface 404.

  In addition, the merit provided with the touch pad 405 on the back surface (back surface) is considered. For example, the user may be able to see all of the GUI 402 while operating the touchpad 405. In addition, the possibility that the touch panel 403 is soiled by a touch operation will be reduced.

  In the following, touch operation from the touch panel 403 is referred to as front input by the front input unit, and touch operation from the touch pad 405 is referred to as back input from the back input unit.

  FIG. 5 and FIG. 6 are diagrams illustrating a state where the user performs a back surface input to the device. Here, the front surface 501 of the device (housing) corresponds to the front surface 401 in FIG. The touch panel 502 corresponds to the touch panel 403 in FIG. A user's hand 503 holds the device. A character locus 504 visually indicates a locus handwritten by the user on the touch pad 405 in FIG. 4, but is not displayed on the touch panel 502.

  For example, the user does not touch the touch panel 402 on the front surface 401 of the device (housing), but inputs characters by handwriting on the touch pad 405 while looking at the touch panel 402.

  In this case, the user will not be able to actually see what character he / she is writing on the touchpad 405. That is, the user moves his / her finger while imagining the case where the character locus 504 is viewed from the front 501 in his / her head.

  When the back input as described above is performed, characters as shown in FIG. 6 are written on the touch pad 405. This is a mirror image of the characters intended by the user.

  In this embodiment, such a mirror image is recognized as a correct locus or character intended by the user.

  FIG. 1 is a functional configuration diagram of a device (handwriting recognition device 100) applicable in the present embodiment. A trajectory detection unit 101 is connected to the apparatus 100. This corresponds to, for example, the touch panel 403 that is the front input unit on the front side of the device and the touch pad 405 that is the rear input unit on the back side of the device. Information on the locus generated by the user's handwriting or the like generated in the locus detection unit 101 is input by the apparatus 100 via the locus input unit 102. Here, the trajectory detection unit 101 is described as being connected as an external device of the apparatus 100, but may exist as a processing unit built in the apparatus 100.

  Next, the trajectory information input here is output to the front / back input detection unit 103. The detection unit 103 specifies whether the input locus is input from the front of the device (touch panel 403) or input from the back of the device (touch pad 405).

  If the trajectory is input from the front of the device, the information on the trajectory is output to the front input trajectory recognition unit 104. On the other hand, if the trajectory is input from the back of the device, the trajectory is output to the back input trajectory recognition unit 105.

  The front input trajectory recognition unit 104 and the back input trajectory recognition unit 105 recognize the input trajectory information using a predetermined recognition algorithm, and output the trajectory recognition result. This recognition result is output as a result of handwriting input by the user to an external device of the apparatus 100 via the recognition result output unit 106 or used for various processes in the apparatus 100. In addition, as a specific example of the recognition result here, character data or symbol data, which is a result of character recognition or the like, corresponds.

  FIG. 2 is a diagram illustrating an operation flow of the apparatus 100. In the present embodiment, it is assumed that a software application for recognizing a trajectory is started when the user operates the GUI 402 or the like on the front surface 401 of the device. Then, this operation flow is started when input of a locus by a user's finger or stylus is started on the touch panel 403 on the front surface of the device or the touch pad 405 on the back surface of the device.

  When the operation flow in the figure is started, in step S201, the trajectory input unit 102 detects whether the input trajectory is a trajectory input from the front or a trajectory input from the back. Here, in order for the trajectory input unit 102 to facilitate the detection, a flag for identifying whether the trajectory is input from the front input or the back input is added to the information on each trajectory input from the trajectory detection unit 101. Shall be.

  In step S202, the input trajectory information is acquired. For example, information on the trajectory accumulated in the event queue of the system up to this point is acquired.

  In step S203, it is determined from the acquired trajectory whether the trajectory is the end of the trajectory input by the user. For example, when the acquired latest locus has not been added or updated for 500 ms or longer, it is determined that the input of the locus by the user has already been completed. In this embodiment, the locus recognition process is sequentially performed while the locus is input by the user, and when the locus input is completed, a recognition result candidate is output immediately. Therefore, if it is determined in step S203 that "the acquired trajectory is not yet the end (the trajectory is still being input by the user)", the process immediately proceeds to the trajectory recognition process in and after step S204.

  In step S204, it is checked whether or not the front input is determined in step S201 (S204). If the detection result in step S201 is front input, the process proceeds to step 205. On the other hand, if the detection result in step S201 is the back side input, the process proceeds to step 206.

  In step S205, the trajectory acquired in step S202 is recognized as a trajectory by front input. Specifically, the trajectory recognition is executed by pattern matching using a front input trajectory pattern 702 described later.

  In step S206, the locus acquired in step S202 is recognized as a locus by back input. Specifically, reversal processing for reversing the left and right of the trajectory is performed by normalization processing described later, and then the trajectory is recognized by pattern matching using the front input trajectory pattern 702.

  If it is determined in step 203 that the acquired trajectory is the end of the trajectory, it is not necessary to perform any further recognition processing, and the process proceeds to step S207.

  In step S207, the result of the recognition process in the previous step S205 or step S206 is output. Note that it is not necessary to determine one recognition result, and N candidates from the 1st to Nth candidates with a high possibility of correct answer (high recognition result score) are output as recognition results. Although not described in detail here, in the subsequent processing, the user or the device may select one of N candidates as necessary.

  FIG. 3 is a conceptual diagram showing basic functions for recognizing a locus in the locus recognizing units 104 and 105. It mainly comprises a feature extraction function 301, a pattern matching function 302, an output function 303, and a locus pattern 304. First, the feature extraction unit 301 performs feature extraction using the trajectory information input from the trajectory detection unit 101. As this feature, for example, each coordinate of the trajectory, a variation amount from the trajectory before a certain time, or the like is used. In step S302, the extracted feature value is subjected to pattern matching with a large number of pre-registered trajectory patterns 304. For pattern matching, various known pattern matching may be applied. For example, the degree of similarity may be calculated by taking into account fluctuations in the time direction by dynamic programming, or similar using stochastic calculation by expressing the trajectory pattern in Hidden Markov Model. The degree may be calculated. In step S303, as a result of the pattern matching, the top N pieces having similarities are output as recognition result candidates.

  FIG. 7 is a diagram showing in more detail the inside of the apparatus 100 of FIG. 1 described above. Here, further details of the front input trajectory recognition unit 104 and the back input trajectory recognition unit 105 are shown. Since the basic configuration is the same as that in FIG. 1, the description of the processing units other than those newly appearing in FIG.

  The front input trajectory recognition unit 104 includes a trajectory recognition unit A701. The trajectory recognition unit A 701 recognizes a trajectory input by the user using the front input trajectory pattern 702. The front input trajectory pattern 702 is a database that stores all patterns of trajectories that are highly likely to be drawn by handwriting when the user performs front input.

  FIG. 8 is a diagram illustrating an example of a locus pattern. In the figure, reference numeral 801 is an example of the above-described pattern of “a locus that is highly likely to be drawn by handwriting when the user performs front input”. On the other hand, reference numeral 802 in the figure is an example of a pattern of “a trajectory that is highly likely to be drawn by handwriting when the user makes a backside input”. That is, the front input locus pattern 702 is a database storing patterns corresponding to 801 in the figure.

  The back surface input trajectory recognition unit 105 includes a trajectory normalization unit 703 and a trajectory recognition unit B704. The trajectory normalization unit 703 normalizes the back side input trajectory information to the same data as the front trajectory by inverting left and right. The trajectory recognition unit B704 recognizes the normalized trajectory using the front input trajectory pattern 702 in the same manner as the trajectory recognition unit A701.

  According to the above, it is possible to recognize a highly accurate trajectory regardless of whether the trajectory is input from the front or the trajectory input from the back.

<Embodiment 2>
In the first embodiment, the trajectory input on the back side is recognized after normalization (horizontal inversion) of the input trajectory. However, there may be cases where it is effective to perform normalization other than left-right reversal as the normalization processing. An example will be described below.

  First, let us consider a case where the user makes a backside input. For example, the user may hold the device with a finger other than the index finger. In that case, the handwriting trajectory will be input with the index finger. Here, the trajectory input by the index finger has a characteristic shape, and the trajectory will be confined in the movable range of the index finger.

  Reference numeral 901 in FIG. 9 indicates a range in which one index finger can come into contact with the touch pad 405 at the time of back side input, in other words, a movable range of the index finger on the touch pad. When the user draws a trajectory with the index finger while holding the device, the trajectory originally intended to be drawn by the user is distorted inside the elliptical movable range. And if it is going to recognize this distorted locus | trajectory, it will be difficult to perform correct recognition. Therefore, in this embodiment, in addition to the left-right inversion which is normalization in the first embodiment, the above-described distortion is also eliminated. Specifically, as indicated by reference numeral 902 in the figure, a spatial transformation (mapping process) is performed for each point of the trajectory so as to transform the illustrated elliptical movable range into a rectangle. Thereby, the distortion of space is normalized.

  FIG. 13 is a sub-flow of the back surface input locus recognition process S206 performed by the back surface input locus recognition unit 105. First, in step S1301, as the first normalization process, the trajectory input to the back side is reversed horizontally. Next, in step S1302, as a second normalization process, the left-right inverted trajectory is spatially transformed as indicated by 902 in FIG. Next, in step S1303, the locus subjected to the above two types of normalization processing is recognized by pattern matching with the front input locus pattern 702.

  As a method of performing two types of normalization processing, for example, using a spatial conversion table as shown in FIG. 10, both first normalization (horizontal inversion) and second normalization (spatial conversion for distortion removal) are used. Spatial transformation may be executed. For example, using this table, the coordinates (100, 104) on the input locus are mapped to the coordinates (209, 52). With this mapping, both the first normalization (left-right inversion) and the second normalization (spatial transformation for distortion removal) are performed at a time.

  In the present embodiment, the second normalization has been described by taking the movable range of the index finger as an example, but the present invention is not limited to this. For example, the finger type may be another finger. In addition, a movable range in the case where the back side input is performed by a stylus or the like may be considered.

  According to the above, the trajectory input from the back can be recognized with higher accuracy.

<Embodiment 3>
In each of the above-described embodiments, the device that allows the user or the device to clearly distinguish between the front and back types, such as the touch panel 403 on the front of the device and the touch pad 405 on the back of the device, has been described. Hereinafter, an embodiment in which these cannot be clearly distinguished will be described.

  FIG. 11 is an example of a device that does not allow the user to distinguish between the front and the back. In the figure, a touch panel 1103 and a touch panel 1104 are provided on each of the A side 1101 and the B side 1102 of the device. The user operates without being aware of which side of the terminal is the front side A or side B. That is, both the A side and the B side can be the front side or the back side of the device.

  FIG. 12 is a functional configuration diagram of a device (handwriting recognition device 100) applicable in the present embodiment. Note that the same components as those in FIGS. 1 and 7 are given the same reference numerals and description thereof is omitted. A characteristic part in this embodiment is the operation of the front / back input detection unit 103.

  It should be noted that the touch panel 1200 may be a front touch panel or a rear touch panel. If the touch panel 1200 is a front touch panel, it corresponds to the same thing as the touch panel 1202. On the other hand, if the touch panel 1200 is a back touch panel, it is a touch panel on the opposite side of the touch panel 1202.

  In the present embodiment, the front / back input detection unit 103 can detect which side of the touch panel A or B is facing the front of the user. That is, the front / back input detection unit 103 can determine whether the touch panel 1200 including the trajectory detection unit 101 is a front touch panel or a back touch panel. As this detection method, various methods can be applied. For example, a camera is provided on the A side of the device, and shooting is performed with the A side camera when a trajectory is input. If it is determined that the face is included as a result of detecting the face of the image captured by this camera, it can be determined that the A-side is the front of the user. Conversely, if it is determined that no face is included, it may be determined that surface B is the front of the user.

  As another method, for example, an acceleration sensor (gravity sensor) may be provided in the device, and the surface facing upward from the A surface and the B surface may be determined as the front surface of the user.

  As described above, when a trajectory is input to the front touch panel for the user, it is determined as front input, and when a trajectory is input to the rear touch panel for the user, it is determined as rear input.

  In the present embodiment, the rear input track recognition unit 105 includes a front panel output unit 1201. Then, the trajectory normalized by the trajectory normalization unit 703 can be output to the touch panel (touch panel display) 1202 that is determined to be the front of the user via the front panel output unit 1201. This has an advantage that it is possible to clearly indicate to the user what the trajectory input on the back side is, and it is easy to input the trajectory according to the user's intention.

<Embodiment 4>
In each of the above embodiments, pattern matching with the front input trajectory pattern 702 is performed by normalizing the trajectory input on the back side. However, even if the spatial transformation using the table as shown in FIG. 10 is performed for all the coordinates of the input trajectory, the load is large. On the other hand, it is also possible to recognize the trajectory input on the back side at a high speed without normalizing the trajectory input on the back side. An example will be described below.

  FIG. 14 is a functional block diagram of the present embodiment. The back surface input locus pattern 1601 is a database similar to the back surface input locus pattern 802 in FIG. 8 described above. In this database, trajectories that can be input by back side input are stored. That is, the back input trajectory pattern 1601 corresponds to a collection of trajectories obtained by horizontally flipping trajectories that may be input by front input. In FIG. 14, when the front / back input detection unit 103 determines that the input is the back side, the trajectory recognition unit B 704 does not need the function of the trajectory normalization unit 703, and the trajectory pattern 1601 of the back input is promptly determined. Perform pattern matching. Other operations in FIG. 14 are basically the same as those in FIG.

  According to the above, the locus can be recognized at high speed by the back surface input.

<Modification>
In each of the above embodiments, the description has been made assuming that the front surface and the back surface face in directions different by 180 degrees, but the present invention is not limited to this. That is, the present invention can be applied to a device having at least two surfaces that the user feels is flat, and the two surfaces may be in any relationship that the user feels sensuously the front and the back. For example, even if the arrangement is such that the A side and the B side of the device form an angle of about 90 degrees, a user who views the A side as the front must be able to see the B side at the same time as the A side. For example, the B surface corresponds to the back surface. It is possible to apply the concept of the front and back of each embodiment mentioned above also about two surfaces of such a relationship.

  Each embodiment described above is also realized by executing the following processing. That is, a process of supplying software (computer program) that realizes each process and function of the above-described embodiment to the system via a network or a storage medium, and the computer (or CPU or the like) of the system reads and executes the program. It is. The computer program and a computer-readable storage medium storing the computer program are also included in the scope of the present invention. For example, when a computer composed of a CPU, ROM, RAM, and the like performs functions equivalent to the processing units of the device shown in FIG. 1, the computer corresponds to a recognition device, and a series of recognition operations. Controlling this corresponds to the control method of the recognition device.

Claims (4)

A recognition device including a front input unit and a back input unit for detecting a trajectory input by a user,
Obtaining means for obtaining a trajectory input from the front input unit or the back input unit;
A specifying means for specifying which of the two input units is the acquired trajectory;
Recognizing means for recognizing the acquired trajectory,
When the acquired trajectory is input from the back surface input unit, the recognizing unit performs a normalization process including a reversal process of the acquired trajectory, and then performs a normalization process of the trajectory input from the front input unit. A recognition apparatus that performs recognition of a trajectory using the same recognition algorithm as recognition.   A computer program that causes a computer to function as the recognition device according to claim 1 by being read and executed by a computer.   A computer-readable storage medium storing the computer program according to claim 2. A method for controlling a recognition device including a front input unit and a back input unit for detecting a trajectory input by a user,
An acquisition step of acquiring a trajectory input from the front input unit or the back input unit by an acquisition unit;
A specifying step of specifying, by specifying means, whether the acquired trajectory is a trajectory input from which of the two input units;
A recognition step of recognizing the acquired trajectory by a recognition means,
In the recognition step, when the acquired trajectory is input from a back surface input unit, after performing a normalization process accompanied by a reversal process of the acquired trajectory, the trajectory input from the front input unit A control method characterized by executing recognition of a trajectory using the same recognition algorithm as recognition.

Images