JP2002230485A - Character recognition device, portable terminal, method associated therewith, recording medium, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce degradation of the character recognition ratio, at change of emotion with respect to a character recognition device of pen input and to dissipate stress on a user of a portable terminal device. SOLUTION: Sentiments of the user are recognized (S4), and a dictionary for recognition is switched in accordance with the recognized emotion state of the user to use another recognition dictionary, corresponding to the emotion state of the user at this time (S5 and S6). When plural recognition result candidates are displayed, the number of conversion result candidates is changed, on the basis of the result of emotion recognition, and the number of candidate characters is made larger than normal, when the user roughly input characters by his hands due to lack of stability in his emotional state. Emotion of the user is recognized, and normal job processing is performed, when the user is in the normal state, but input processing indicated by the user is performed prior to other jobs operating in parallel, when the user is angry or is in a bad mood.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to character recognition for the purpose of flexibly coping with a change in the user's emotion in recognition of pen input incorporated in the function as input means of a portable terminal device or the like. The present invention relates to a device, a mobile terminal device that recognizes a user's emotional state, and controls internal processing based on the user's emotional state at that time, a related method, a recording medium, and a program.

[0002]

2. Description of the Related Art Conventionally, a pen-input character recognition device provided in a portable terminal device or the like scans a character input by a user with a pen at fixed intervals, and sets a vector value at each point and a length of each line segment. And the like are obtained as data values, the obtained input character data is collated with the character recognition dictionary of the character recognition device, and the character considered to be the closest is displayed on the display (display device) as a recognition character. ing. In addition, the character recognition device increases the capacity of character recognition dictionary data to increase the recognition level.

On the other hand, a conventional portable terminal device such as a handy terminal executes a process in accordance with a user's input process, and when performing a plurality of processes in parallel, the priority is determined by the priority level of an interrupt signal issued by each program. , And processing is performed according to an interval determined by each program.

[0004]

However, in the above-described conventional character recognition device, when the user's emotion is unstable due to anger, impatience, or the like, rough or rough pen input may occur. It is easy to get up, and in such a case, the recognition rate may be significantly lower than the recognition success rate at the normal time when the emotion of the user is relatively quiet.

[0005] In order to increase the recognition rate even in such various cases, in the prior art, it is necessary to increase the capacity of the recognition dictionary. However, the capacity of the recognition dictionary is increased. In such a case, a problem occurs such that the recognition time becomes extremely slow because the enlarged dictionary and the input data are compared.

In the above-mentioned conventional portable terminal device,
In some cases, the processing input by the user may be awaited by a job that has already been processed, or the performance may be degraded by a job running simultaneously behind the scenes. In such a case, when the user is in a bad mood, that is, when he is angry or frustrated, the user is further in a bad mood. In such a state, the handling of the device by the user may be violent and the device may be damaged, or secondary damage may occur.

A first object of the present invention is to provide a pen-input character recognition device provided in a portable terminal device or the like, and a recognition dictionary based on a user's emotional state. And reducing the recognition rate at the time of emotion change by using a recognition dictionary corresponding to the emotional state of the user at that time.

A further object of the present invention is to lower the criterion for determining a character candidate to be displayed as a conversion candidate even when the user's emotional state is unstable and the input method is complicated. Thus, the number of candidate characters is increased as compared with the normal case, and the trouble of re-input and re-conversion of the user is reduced.

A second object of the present invention is to control the priority of a plurality of jobs being internally processed in a portable terminal device such as a handy terminal in accordance with the emotional state of the user. When the user is in a normal state, the normal job processing is performed, and when the user is angry or in a bad mood, the input processing specified by the user is given priority over other jobs running in parallel. Another object of the present invention is to reduce user stress.

It is a further object of the present invention to prevent the user from handling rough equipment due to user stress as much as possible, and to extend the life of the equipment.

[0011]

According to a first aspect of the present invention, there is provided a character recognition apparatus for recognizing an emotional state of a user based on voice data input from voice input means. Recognition means, pen input means for inputting handwritten characters by pressing a writing pen against the touch panel, a first character recognition dictionary for normal use in which the user's emotions are relatively quiet, and user's emotions A non-normal second character recognition dictionary that is unstable due to anger, etc., and the first character recognition dictionary and the second character recognition dictionary according to the recognition result of the emotion recognition means. Dictionary selection means for selecting one of the dictionaries, and handwriting input to the pen input means using the first character recognition dictionary or the second character recognition dictionary selected by the dictionary selection means Character recognition means for recognizing characters, And having a recognition result display means for displaying the recognized character of the results serial recognized by the character recognition means.

Here, when the character recognition means causes the recognition result display means to display the candidate for the recognized character,
The number of recognition candidate characters to be displayed may be changed depending on whether the user's emotion is normal or non-normal.

[0013] The character recognition means may increase the number of recognition candidate characters when the user's emotion is non-normal.

[0014] In order to achieve the first object, a fourth aspect is provided.
The character recognition device of the invention, emotion recognition means for recognizing the emotional state of the user based on the voice data input from the voice input means, pen input means for pressing a writing pen on the touch panel to input handwritten characters, A character recognition dictionary for recognizing the handwritten characters, and a character number determining unit for determining a recognition candidate character number according to a recognition result of the emotion recognition unit;
A character recognition unit that compares the character recognition dictionary with handwritten character data input to the pen input unit and extracts recognition candidate characters of the number of recognition candidate characters determined by the character number determination unit; And a recognition result display means for displaying the recognition candidate character extracted by the means.

Here, the number-of-characters deciding means may be characterized in that the number of recognition candidate characters is increased when the emotion of the user is not normal.

The emotion recognition means may be characterized by recognizing not only voice but also the emotional state of the user together with voice data and image data such as a face color input from the image data input means. .

According to another aspect of the present invention, the second object is achieved.
The invention of the portable terminal device, the emotion recognition means for recognizing the emotional state of the user based on the voice data input from the voice input means, processing input means for performing input for causing the terminal to perform desired processing, Depending on the recognition result of the emotion recognition means,
The present invention is characterized in that it has a priority determining unit that compares a job currently being processed with a job received from the processing input unit and determines a priority of each job.

Here, the priority determining means may be characterized in that when the user's emotions are not normal, the job received from the processing input means has priority over the job currently being processed.

[0019] Further, the priority determination means sends an instruction to the communication means to change the number of packets to a value smaller than the current value when communication is being performed when the emotion of the user is not normal, It is possible to preferentially process a job received from the processing input unit between communications.

Further, the emotion recognition means may be characterized by recognizing a user's emotional state together with voice data as well as image data such as a face color input from the image data input means and voice data. .

[0021] In order to achieve the first object, a first aspect is provided.
The invention of the second aspect is a voice input means, a first character recognition dictionary for normal use in which the user's emotion is relatively quiet, and an unusual use in which the user's emotion is unstable due to anger or the like. Second for time
A character recognition method for a character recognition device having a character recognition dictionary and an emotion recognition step of recognizing a user's emotional state based on voice data input from the voice input means, and pressing a writing pen against a touch panel. A pen input step of inputting handwritten characters, a dictionary selecting step of selecting one of the first character recognition dictionary and the second character recognition dictionary according to a recognition result of the emotion recognition step, A character recognition step of recognizing a handwritten character input in the pen input step using the first character recognition dictionary or the second character recognition dictionary selected in the dictionary selection step; And displaying a recognition result as a recognition result.

In order to achieve the first object, a first aspect is provided.
According to a fifth aspect of the present invention, in a character recognition method of a character recognition device having a voice input unit and a character recognition dictionary for recognizing handwritten characters, a user's emotional state is recognized based on voice data input from the voice input unit. Emotion recognition step,
A pen input step of inputting handwritten characters by pressing a writing pen against a touch panel, a character number determining step of determining the number of recognition candidate characters according to a recognition result in the emotion recognition step, the character recognition dictionary and the pen input A character recognition step of comparing the handwritten character data input in the step and extracting recognition candidate characters of the number of recognition candidate characters determined in the character number determination step, and a recognition candidate character extracted in the character recognition step. And displaying a recognition result.

In order to achieve the second object, a first aspect is provided.
The invention of claim 8 is an emotion recognition step for recognizing a user's emotional state based on voice data input from voice input means, a processing input step for performing an input for causing a terminal to perform a desired process, and the emotion recognition step. And a priority determining step of determining the priority of each job by comparing the job currently being processed with the job received from the processing input step in accordance with the recognition result of the above.

In order to achieve the first object, a second aspect is provided.
The invention of the second aspect is a voice input means, a first character recognition dictionary for normal use in which the user's emotion is relatively quiet, and an unusual use in which the user's emotion is unstable due to anger or the like. Second for time
And a program for controlling a character recognition device having a character recognition dictionary by a computer, the program being provided to a computer based on voice data input from the voice input means. An emotional state is recognized, and a handwritten character with a writing pen pressed against the touch panel is input, and one of the first character recognition dictionary and the second character recognition dictionary is selected according to the recognition result of the emotional state. Is selected, and the selected first
Using the character recognition dictionary or the second character recognition dictionary to recognize handwritten characters input with the writing pen,
It is characterized by displaying a recognition character as a recognition result.

[0025] In order to achieve the first object, a second aspect is provided.
The invention of claim 5 is a recording medium storing a program for controlling a character recognition device having a voice input means and a character recognition dictionary for recognizing handwritten characters by a computer, wherein the program stores the program Recognizing the emotional state of the user based on the voice data input from the input means, inputting handwritten characters by pressing a writing pen on the touch panel, and determining the number of recognition candidate characters according to the recognition result of the emotional state, A character recognition dictionary and data of handwritten characters input with the writing pen are collated to extract recognition candidate characters of the determined number of recognition candidate characters, and to display the extracted recognition candidate characters. I do.

[0026] To achieve the second object, the present invention is applied to a second aspect.
The invention of claim 8 is a recording medium recording a program for controlling the portable terminal device by a computer, the program recognizing the computer to recognize the emotional state of the user based on voice data input from voice input means. And causing the terminal to perform an input for performing a desired process, and comparing the job currently being processed with the job received from the process input step according to the recognition result of the emotional state. The order is determined.

In order to achieve the first object, a third aspect of the present invention is provided.
The invention of the second aspect is a voice input means, a first character recognition dictionary for normal use in which the user's emotion is relatively quiet, and an unusual use in which the user's emotion is unstable due to anger or the like. Second for time
A character recognition dictionary having a character recognition dictionary, the program comprising: an emotion recognition step of recognizing a user's emotional state based on voice data input from the voice input means; A pen input step of pressing a writing pen to input handwritten characters,
Selecting one of the character recognition dictionary and the second character recognition dictionary, and the first character recognition dictionary or the second character recognition dictionary selected in the dictionary selection step. It has a character recognition step of recognizing a handwritten character input in the pen input step using a dictionary, and a recognition result display step of displaying a recognition character as a result of the recognition in the character recognition step.

[0028] To achieve the first object, a third aspect is provided.
According to a fifth aspect of the present invention, there is provided a program used for a character recognition device having a voice input unit and a character recognition dictionary for recognizing handwritten characters, wherein the program is used based on voice data input from the voice input unit. Recognition step for recognizing the emotional state of the person, a pen input step of pressing a writing pen on the touch panel to input handwritten characters, and determining the number of candidate characters for recognition in accordance with the recognition result in the emotion recognition step A character recognition step of comparing the character recognition dictionary with handwritten character data input in the pen input step and extracting recognition candidate characters of the number of recognition candidate characters determined in the character number determination step, A recognition result display step of displaying the recognition candidate character extracted in the character recognition step.

In order to achieve the second object, a third aspect is provided.
The invention of Claim 8 is a program used for a portable terminal device, the program comprising: an emotion recognition step of recognizing a user's emotional state based on voice data input from voice input means; A job input process for inputting a job to be executed, and a job currently being processed and a job received from the process input step are compared in accordance with a recognition result in the emotion recognition step to determine a priority of each job. And a priority order determining step.

[0030]

Embodiments of the present invention will be described below in detail with reference to the drawings.

(First Embodiment) FIG. 1 shows the configuration of a character recognition device according to a first embodiment of the present invention which can achieve the first object of the present invention. Here, reference numeral 1 denotes a microphone which is a means for inputting voice, and 2 denotes a microphone 1
An emotion recognizing unit 3 for recognizing the emotion from the voice data from the user, an emotion level assigning unit 3 for judging the level of the user's emotion from the recognition data from the emotion recognizing unit 2 and outputting the result of the judgment as data. Reference numeral 4 denotes a touch panel unit for the user to input characters using a pen, and 5 denotes a touch panel 4.
A character input data detecting unit 6 for detecting character input data input to the input unit 6 is an input data storage unit for storing data from the character input data detecting unit 6. Reference numeral 7 denotes a first character recognition dictionary when the user's emotion is normal, and reference numeral 8 denotes a second character recognition dictionary when the user's emotion changes (when the user is angry or in a bad mood). is there. Reference numeral 9 denotes a first character recognition dictionary 7 in a normal state based on data from the emotion level assignment processing unit.
And the second character recognition dictionary 8 at the time of emotion change, the character recognition unit recognizes the data from the input data storage unit 6, that is, the handwritten characters input by the user from the touch panel 4. is there. Reference numeral 10 denotes a recognized character display unit that displays a character recognized by the character recognition unit 9 on a display screen.

In the above configuration, if the emotion recognition unit 2 and the emotion level assignment processing unit determine that the emotion is angry based on the loudness and tone of the user's voice input from the microphone 1, the user touches the touch panel 4 Characters input from the touch panel input data detection unit 5
And the data is stored in the input data storage unit 6. When the input for one character is completed and the user issues a recognition instruction using a recognition button or the like, the character recognition unit 9
Since the data from the emotion level allocation processing unit 3 is data that determines that the user is angry, the dictionary data of the second character recognition dictionary 8 for emotion change and the input data storage unit as the recognition dictionary. 6 is compared with the handwritten character data stored in No. 6 to determine the closest character,
The recognition result is displayed on the recognition character display unit 10 as a recognition character.

Conversely, if the user's emotion is normal, the character recognition unit 9 selects the normal first character recognition dictionary 7 as the recognition dictionary, performs character recognition, and recognizes the result as a recognized character. It is displayed on the display unit 10.

FIG. 2 is a flowchart showing the flow of processing according to the first embodiment of the present invention. When the main routine of the character recognition process starts (step S1), first, it is determined whether or not the user has made a pen input via the touch panel 4 and the touch panel input data determination unit 5 (step S2). If not, the process returns to the start again. If there is a pen input, the touch panel data determination unit 5 scans the data input to the touch panel at regular intervals, obtains the data, and stores the data in the input data. It is stored in the unit 6 (step S3).

Next, the emotion level data of the user is obtained via the microphone-phone 1, the ring recognition unit 2, and the emotion level assignment processing unit 3 (step S4), and the recognition dictionary 7 or 8 corresponding to the emotion is obtained. (Step S
5).

The input handwritten character data obtained in step S3 is compared with the data in the recognition dictionary 7 or 8 selected in step S5, and the closest character is determined as a recognition character to perform character recognition ( Step S6), the recognized character is displayed on the display screen of the recognized character display unit 10 (Step S7), and the process returns to the first step, Step S1, to continue the recognition.

The flowchart shown in FIG.
4 shows a detailed process flow of acquiring data on the emotion level of the user No. 4. When the subroutine starts (step S8), it is determined whether or not a user's voice has been input through the microphone 1 (step S9). If no voice has been input, the process returns to the start again. Is an emotion recognition unit 2 based on the input voice data.
Then, the emotion state is recognized and converted into emotion recognition data (step S10). Subsequently, the emotion level is assigned to a weighted level from the emotion recognition data by the emotion level assignment processing unit 3 (step S11).

Next, the character recognizing section 9 executes step S in FIG.
6, the emotional state of the user at the time of character input is determined based on the emotional level data created by the emotional level assignment processing unit 3, and character recognition classified for each emotion is performed based on the determination. Corresponding dictionary data is selected from the dictionaries 7 and 8, so that the user at that time can recognize characters as much as possible according to the emotion.

As described above, in the character recognition according to the prior art, the character recognition rate is high when the user's emotion is in a normal state, but the user's emotion changes and pen input becomes coarse. The problem that the character recognition rate drops
In the present invention, since the character recognition dictionary to be used is switched according to the emotional state to recognize the character, it is possible to remarkably reduce the reduction in the character recognition rate.

Further, according to the present invention, since the character recognition dictionary to be used is switched according to the emotional state as described above, the character recognition is performed. Therefore, in order to cope with such a situation in the prior art, the capacity of the recognition dictionary is increased. There is almost no decrease in recognition speed that occurs when improving the recognition rate by doing
Character recognition suitable for the emotional state of the user can be performed.

(Modification of First Embodiment) FIG. 4 shows the configuration of a character recognition device according to a modification of the first embodiment of the present invention. The first embodiment is different from the first embodiment in that the recognition dictionary in FIG. 1 does not exist independently for normal times and for emotion changes, but is a single dictionary data 11. And different.

FIG. 5 is a flowchart showing a processing procedure of a modification of the first embodiment of the present invention. Here, step S
Steps 1 to S5 are the same as steps S1 to S5 in FIG. 2, and a description thereof will be omitted. First, the character recognition unit 9 determines the number of recognition candidate characters according to the emotion level data acquired through the emotion level assignment processing unit 3 (step S1).
6). Next, the character recognition unit 9 extracts the characters of the determined number of recognition candidate characters by comparing the dictionary data 11 with the input character data (step S17). Then, the extracted recognition candidate characters are displayed on the recognized character display unit 10 (step S18).

FIG. 6 shows an example of the number of recognition candidate characters extracted from the dictionary 7 in the normal state and the number of recognition candidate characters extracted from the dictionary 8 when the emotion changes, corresponding to the emotion state level data. Numeral 12 indicates the emotion level, numeral 13 indicates the number of candidate characters from the character recognition dictionary 7 in normal times, and numeral 14 indicates the number of candidate characters from the character recognition dictionary 8 when the emotion changes.

In the example of FIG. 6, the larger the value of the emotion level 12 is, the more unstable the user's emotion is.
As can be seen from the figure, the more the user's emotion is in an unstable state, the more the number of recognition candidate characters 13 from the character recognition dictionary 14 at the time of emotion change is increased, thereby increasing the total number of recognition candidate characters. Even when rough pen input is performed when the user's emotion is unstable, the possibility of displaying correct character recognition characters in candidate characters is increased.

As described above, according to this modification, the number of recognition candidate characters in which the user's emotion appears in normal times is
Since the number of recognition candidate characters when the user's emotion changes is increased, the user's emotion is unstable and pen input is complicated, so if the correct character candidate is not displayed conventionally and it must be re-entered However, the possibility that characters of correct character recognition will be displayed in the candidate characters in the places where the characters have not been changed becomes higher, so that the number of re-inputs can be significantly reduced.

(Second Embodiment) FIG. 7 shows a configuration of a portable terminal device according to a second embodiment of the present invention capable of achieving the above-described second object of the present invention. Here, 21 is a microphone which is a voice input unit, 22 is a camera for recognizing a user's facial expression, 23 is an emotion recognition unit for recognizing emotion from voice data from the microphone 21 and image information from the camera 22, 24 Is an emotion level assignment processing unit that determines the level of the user's emotion from the data from the emotion recognition unit 23 and outputs data of the operation emotion level when operating the device. Reference numeral 25 denotes a touch panel unit for performing pen input processing for the user to give an operation instruction to the portable terminal device.
Reference numeral denotes a touch panel input data determination unit for detecting data input to the touch panel 25, and reference numeral 27 denotes an input processing determination unit for determining what processing is to be performed by the touch panel input data determination unit 6.

Reference numeral 28 denotes an internal processing execution unit that actually performs processing in accordance with an instruction from the input processing determination unit 27. The internal processing execution unit 28 performs processing other than processing in accordance with an instruction from a user.
Naturally, processing by another program or the like can be performed, and two or more processings can be performed in parallel. Reference numeral 29 denotes a priority comparison unit, based on the operation emotion level output from the emotion level assignment processing unit 24, the internal processing currently being executed by the internal processing execution unit 28 and the processing to be newly performed according to an instruction from the user. Are compared, and a weight is given to each process regarding the order in which a plurality of processes are executed, and the priority order is determined. Then, the internal processing execution unit 2
Numeral 8 sequentially executes various jobs in an order according to the priority data determined by the priority comparison unit 29.

In the above configuration, for example, the microphone 2
1 and the level of the user's voice input from the camera 22 and the facial expression, the emotion recognition unit 3 and the emotion level assignment processing unit 24 determine the emotion level of the user. If it is determined that the emotion level at the time of performing is low, the priority comparison unit 28 compares the process instructed by the user from the touch panel 5 with the process currently being executed, according to the emotion level of the determination result.
When the user is in a bad mood, the processing from the user is determined to be the highest priority so as not to stress the user as much as possible.
Send to 8.

Conversely, when the user's emotion is at a good level, the priority comparison unit 28 determines the order of activation from among the processes running in parallel and the other conditions such as the condition for the time limit in each program. The priorities are determined, and the priorities are determined so that a malfunction does not occur in the operation of the entire device and the overall performance does not deteriorate.

FIG. 8 is a graph showing the level of each emotion (anger, happiness, sadness, fear, disgust, surprise) recognized by the emotion recognition unit 23 from the voice input data and the image data at a certain point in time. There is an example, where the broken line indicated by 30 is a graph of the emotion level from the video data, and the solid line indicated by 31 is a graph of the emotion level from the audio data. Also, A
The broken line indicated by is a graph of the standard emotion level at which the user can perform a normal operation when operating the portable terminal device.

The emotion level assignment processing section 24 departs from the level of each of these emotions of anger, happiness, sadness, fear, disgust, and surprise from the emotion level at which normal operation can be performed when operating the portable terminal device. Assign the emotion level.

FIG. 9 is a diagram showing the emotion level in operating the portable terminal device assigned by the emotion level assignment processing section 24. Here, 32 is a graph showing the emotion level from the comprehensive data of the audio data, 33 is a graph showing the emotion level from the comprehensive data of the image data, 34
7 is a graph showing emotion levels when operating a total portable terminal device combining voice data and image data.

FIG. 10 is a flowchart showing the flow of processing according to the second embodiment of the present invention. When the main routine of the job processing starts (step S21), first, voice input of the user through the microphone 21 and image input of the expression of the user via the camera 22 are performed (step S22). Emotion recognition for converting the emotional state of the user into data is performed from the input data (step S23), and the emotion level assignment processing unit 24 performs emotion level discrimination to level the emotion from the emotion data generated in step 23. (Step S24).

Next, it is determined by the priority comparing section 29 whether or not there is a program designated by the user (step S25). If there is no such program, the process returns to step S22 again. If there is a program designated by the user, it is determined whether or not there is a program currently being executed (step S26). Is executed (step S29). After the execution, the process returns to step S22.

On the other hand, if it is determined in step S26 that there is another program (process) currently being executed, the priority comparison unit 29 determines how urgent the program (process) being executed is. To judge whether
The processing is leveled (step S27).

Based on the urgency level thus determined and the emotional level determined in the user's emotional level determination step S24, the assignment of the currently executing processing to the processing specified by the user is determined. The processing order of the program is adjusted (step S28), and based on the determination of the priority, the internal processing execution unit 28 gives priority to the process specified by the user when the user is in a bad mood (step S29). .

As described above, according to the present invention, by grasping the emotional state of the user who uses the portable terminal device, when the user is in a bad mood, such as when the user is angry, the user can use the portable terminal device. Since the process to be instructed is executed with priority, the stress on the user can be reduced.

(Modification of Second Embodiment) FIG. 11 shows a configuration of a portable terminal device according to a modification of the second embodiment of the present invention. Here, 35 recognizes an emotional state from a voice or the like of a user in which the microphone 21, the camera 22, the emotion recognition unit 23, and the emotion level allocation processing unit 24 of the second embodiment of the present invention constitute one block, and recognizes it. It is an emotion recognition unit that is converted into data. 36 is an input unit from a user from a touch panel or a keyboard, etc., 37 is a central processing unit that controls the entire control, 38 is a communication unit that communicates with external devices, and 39 is a message required by the portable terminal device. It is a display for displaying information such as.

In the portable terminal device having the above configuration,
If the user instructs a certain operation through the input unit 36 while performing communication with the external device through the communication unit 38 by packet communication, the emotion recognition unit 35 recognizes the emotion of the user at that time. Is determined to be in a bad mood, and the determination data is sent to the central processing unit 17,
The central processing unit 17 performs the processing instructed by the user with the first priority, so that the number of packets is set to a value smaller than the current value so that the processing for packet communication is not completely performed during the current processing. An instruction is sent to the communication unit 38 to change.

Upon receiving the instruction, the communication section 38 starts communication by reducing the amount of data to be sent at one time, that is, the number of packets, from the current value. As a result, the time required for one packet communication is shorter than before changing the number of packets, and another process can be inserted between the communication. Therefore, the central processing unit 37 performs the operation instructed by the user during the packet communication, and the user can perform the desired operation because the portable terminal device is in communication, and can perform the communication without being frustrated. Despite being in the middle, it is possible to quickly execute the process that one wants to perform.

Conversely, if the emotional state of the user is regarded as normal by the emotion recognizing unit 15, the central processing unit 37 which has received the emotional data gives first priority to the packet communication currently being executed. The communication unit 38 does not give an instruction to change the packet, etc., but instead gives a message to the user on the display 39, for example, "Please wait for a while now," and ask the user to wait.

As described above, according to the present modification, even when the portable terminal device is communicating, it is possible to perform processing flexibly corresponding to the emotional state of the user. When the user is in a bad mood, the user can give priority to the processing specified by the user even during communication, minimizing the user's stress, and if the user's emotional state is normal, the communication being executed first Since the process is executed with priority, a reasonable response to the user's instruction is possible.

(Other Embodiments) The present invention is applied not only to a portable terminal device but also to a system including a plurality of other devices (for example, a host computer, an interface device, a reader, a printer, etc.). Alternatively, the present invention may be applied to a device including one device (for example, a copying machine, a facsimile machine, and the like).

Another object of the present invention is to provide a recording medium (storage medium) recording software program codes for realizing the functions of the above-described embodiments to a system or an apparatus, and to provide a computer for the system or the apparatus. Needless to say, the present invention is also achieved when the CPU (or the CPU or the MPU) reads out and executes the program code stored in the recording medium.

In this case, the program code itself read from the recording medium implements the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

As a recording medium for recording the program code and for recording variable data such as a table, for example, a floppy disk (FD), hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, A nonvolatile memory card (IC memory card), a ROM, or the like can be used.

When the computer executes the readout program code, not only the functions of the above-described embodiment are realized, but also the operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that an operating system) may perform some or all of the actual processing, and the processing may realize the functions of the above-described embodiments.

[0068]

As described above, according to the first embodiment of the present invention, the user's emotion is recognized, the recognition dictionary is switched according to the recognized user's emotional state, and the recognition dictionary is used. Since the recognition dictionary corresponding to the emotional state of the person is used, it is possible to reduce a decrease in the recognition rate when the emotion changes.

According to the modification of the first embodiment of the present invention, when displaying a plurality of recognition result candidates, the number of conversion result candidates is changed based on the result of emotion recognition.
Even when the user's emotions change and the emotional state is unstable and the input method becomes complicated, the standard for determining character candidates to be displayed as conversion candidates has been lowered and the number of candidate characters has been increased compared to normal cases Therefore, it is possible to reduce the user's labor of re-input and re-conversion.

Further, according to the second embodiment of the present invention, the user's emotion is recognized, and the priority of a plurality of jobs being internally processed is controlled according to the recognized user's emotional state. When the user is in a normal state, the normal job processing is performed, and when the user is angry or in a bad mood, the input processing instructed by the user is given priority over other jobs running in parallel. Therefore, the user's stress can be reduced, and the user can be prevented from handling rough equipment as much as possible, which contributes to extending the product life of the portable terminal device.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a character recognition device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing a flow of processing according to the first embodiment of the present invention.

FIG. 3 is a flowchart showing details of an emotion level data acquisition step in FIG. 2;

FIG. 4 is a block diagram illustrating a configuration of a character recognition device according to a modified example of the first embodiment of the present invention.

FIG. 5 is a flowchart illustrating a flow of a process according to a modification of the first embodiment of the present invention.

FIG. 6 is a diagram showing a configuration example of a table representing the number of recognized characters with respect to an emotion level in a modification of the first embodiment of the present invention.

FIG. 7 is a block diagram illustrating a configuration of a portable terminal device according to a second embodiment of the present invention.

FIG. 8 is a diagram illustrating an example of emotion level data according to the second embodiment of the present invention.

FIG. 9 is a graph showing an example of an emotion level when operating the portable terminal device according to the second embodiment of the present invention.

FIG. 10 is a flowchart illustrating a flow of a process according to a second embodiment of the present invention.

FIG. 11 is a block diagram illustrating a configuration of a mobile terminal device according to a modified example of the second embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Microphone 2 Emotion recognition part 3 Emotion level allocation processing part 4 Touch panel 5 Touch panel input data judgment part 6 Input data storage part 7 Normal first character recognition dictionary 8 Second character recognition dictionary when emotion changes 9 Characters Recognition unit 10 Recognition character display unit 11 Recognition dictionary data 21 Microphone 22 Camera 23 Emotion recognition unit 24 Emotion level assignment processing unit 25 Touch panel 26 Touch panel input data judgment unit 27 Input processing judgment unit 28 Internal processing execution unit 29 Priority comparison unit 35 Emotion recognition unit 36 Input unit 37 Central processing unit 38 Communication unit 39 Display

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 1/00 340 G06T 1/00 340A 7/00 7/00 P 7/20 300 7/20 300B G10L 15 / 10 G10L 3/00 531N F term (reference) 5B009 KB01 LA01 5B057 AA19 BA02 CA12 CA16 DA12 DB02 5B064 AB04 BA05 DA14 DA31 EA12 EA24 FA04 5D015 HH00 5L096 CA02 JA00

Claims (41)

[Claims] 1. An emotion recognition means for recognizing a user's emotional state based on voice data input from a voice input means; a pen input means for pressing a writing pen on a touch panel to input handwritten characters; A first character recognition dictionary for normal use in which emotions are relatively quiet; a second character recognition dictionary for non-normal use in which the user's emotions are unstable due to anger or the like; Dictionary selecting means for selecting one of the first character recognition dictionary and the second character recognition dictionary in accordance with the recognition result of the emotion recognizing means; and the first character dictionary selected by the dictionary selecting means. Character recognition means for recognizing handwritten characters input to the pen input means using a character recognition dictionary or the second character recognition dictionary, and a recognition result for displaying a recognition character as a result of recognition by the character recognition means Having display means A character recognition device characterized in that: 2. The method according to claim 1, wherein the character recognition unit displays the recognition character candidate when the recognition result display unit displays the recognition character candidate when the user's emotion is normal and non-normal. The character recognition device according to claim 1, wherein the number of characters is changed. 3. The character recognition apparatus according to claim 2, wherein said character recognition means increases the number of said recognition candidate characters when the user's emotion is non-normal. 4. An emotion recognition means for recognizing a user's emotional state based on voice data input from voice input means, a pen input means for pressing a writing pen on a touch panel to input handwritten characters, and said handwritten characters A character recognition dictionary for recognizing a character, a character number determining means for determining the number of recognition candidate characters according to a recognition result of the emotion recognition means, and a data of handwritten characters input to the character recognition dictionary and the pen input means. And character recognition means for extracting recognition candidate characters of the number of recognition candidate characters determined by the character number determination means, and recognition result display means for displaying the recognition candidate characters extracted by the character recognition means. Character recognition device characterized by the above-mentioned. 5. The character recognition apparatus according to claim 4, wherein the character number determination unit increases the number of recognition candidate characters when a user's emotion is non-normal. 6. The emotion recognition means for recognizing a user's emotional state in combination with not only voice but also image data such as a face color input from image data input means and voice data. 6. The character recognition device according to any one of claims 5 to 5. 7. A portable terminal device comprising the character recognition device according to claim 1. 8. An emotion recognition means for recognizing a user's emotional state based on voice data input from a voice input means, a processing input means for performing an input for causing a terminal to perform a desired process, and the emotion recognition means. And a priority determining unit that determines a priority of each job by comparing the job currently being processed with the job received from the processing input unit in accordance with the recognition result of the mobile terminal device. 9. The apparatus according to claim 8, wherein the priority determining unit gives priority to the job received from the processing input unit over the job currently being processed when the user's emotion is non-normal. Mobile terminal device. 10. The communication device according to claim 1, wherein the priority determination unit sends an instruction to the communication unit to change the number of packets to a value smaller than a current value when the user is communicating during a non-normal time. 9. The portable terminal device according to claim 8, wherein a job received from the processing input unit is preferentially processed between communications. 11. The emotion recognition means for recognizing a user's emotional state in combination with not only voice but also image data such as a face color input from image data input means and voice data. 11. The mobile terminal device according to any one of claims 10 to 10. 12. A voice input means, a first character recognition dictionary for normal use in which the user's emotion is relatively quiet, and an unusual use in which the user's emotion is unstable due to anger or the like. A character recognition method for a character recognition device having a second character recognition dictionary for time, an emotion recognition step of recognizing a user's emotional state based on voice data input from the voice input means, and writing on a touch panel. A pen input step of pressing a pen to input a handwritten character; and a dictionary for selecting one of the first character recognition dictionary and the second character recognition dictionary according to a recognition result of the emotion recognition step. A selecting step; and a sentence recognizing a handwritten character input in the pen input step using the first character recognition dictionary or the second character recognition dictionary selected in the dictionary selecting step. Character recognition method characterized in that it comprises a recognition step and a recognition result display step of displaying the recognized character resulting from the recognition in the character recognition step. 13. The character recognition step, wherein, when the recognition character candidates are displayed in the recognition result displaying step, the recognition candidates to be displayed when the user's emotions are normal and non-normal. 13. The character recognition method according to claim 12, wherein the number of characters is changed. 14. The character recognition method according to claim 13, wherein in the character recognition step, the number of the recognition candidate characters is increased when a user's emotion is non-normal. 15. A character recognition method for a character recognition device having voice input means and a character recognition dictionary for recognizing handwritten characters, wherein an emotional state of a user is recognized based on voice data input from the voice input means. An emotion recognition step, a pen input step of inputting a handwritten character by pressing a writing pen against a touch panel, a character number determination step of determining the number of recognition candidate characters according to a recognition result in the emotion recognition step, A character recognition step of collating a dictionary with handwritten character data input in the pen input step and extracting recognition candidate characters of the number of recognition candidate characters determined in the character number determination step; And displaying a recognition result character. 16. The character recognition method according to claim 15, wherein the character number determination step increases the number of recognition candidate characters when a user's emotion is non-normal. 17. The apparatus according to claim 12, wherein in the emotion recognition step, the emotional state of the user is recognized in combination with not only voice but also image data such as a face color input from image data input means and voice data. 17. The character recognition method according to any one of items 16 to 16. 18. An emotion recognition step of recognizing a user's emotional state based on voice data input from voice input means, a processing input step of performing input for causing a terminal to perform a desired process, and the emotion recognition step. And a priority determining step of determining the priority of each job by comparing the job currently being processed with the job received from the processing input step in accordance with the recognition result of the mobile terminal device. Job control method. 19. The job according to claim 18, wherein, in the priority order determining step, when the emotion of the user is not normal, the job received from the processing input step is given priority over the job currently being processed. Job control method for a portable terminal device. 20. In the priority determination step, when communication is being performed when the user's emotion is non-normal, an instruction is sent to the communication means to change the number of packets to a value smaller than the current value. 19. The job control method for a portable terminal device according to claim 18, wherein a job received in the process input step is preferentially processed between communications. 21. The emotion recognizing step includes recognizing a user's emotional state together with voice data as well as image data such as a face color input from image data input means and voice data. 21. The job control method for a portable terminal device according to any one of claims to 20. 22. Speech input means, a first character recognition dictionary for normal use in which the user's emotion is relatively quiet, and an unusual use in which the user's emotion is unstable due to anger or the like. A recording medium for recording a program for controlling a character recognition device having a second character recognition dictionary for use by a computer, the program stores, for a computer, voice data input from the voice input means. Based on the recognition result of the emotional state, the first character recognition dictionary and the second character recognition The user selects one of the dictionaries and uses the selected first character recognition dictionary or the second character recognition dictionary to recognize handwritten characters input with the writing pen. Recording medium, characterized in that to display the recognized character. 23. The program, when causing the computer to display the recognized character candidates, changes the number of recognized candidate characters to be displayed depending on whether the user's emotion is normal or non-normal. 23. The recording medium according to claim 22, wherein the recording medium is executed. 24. The recording medium according to claim 23, wherein the program causes the computer to increase the number of the recognition candidate characters when the user's emotion is non-normal. 25. A recording medium storing a program for controlling a character recognition device having a voice input means and a character recognition dictionary for recognizing handwritten characters by a computer, wherein the program transmits the voice to the computer. Recognizing the emotional state of the user based on the voice data input from the input unit, inputting the handwritten character pressed against the writing pen on the touch panel, and determining the number of recognition candidate characters according to the recognition result of the emotional state, A character recognition dictionary is compared with handwritten character data input with the writing pen to extract recognition candidate characters of the determined number of recognition candidate characters, and to display the extracted recognition candidate characters. Recording medium. 26. The recording medium according to claim 25, wherein the program causes the computer to increase the number of recognition candidate characters when the user's emotion is non-normal. 27. The program according to claim 27, wherein the program causes the computer to recognize the emotional state of the user together with not only voice but also image data such as a face color input from the image data input means and voice data. 27. The recording medium according to any one of 22 to 26. 28. A recording medium recording a program for controlling a portable terminal device by a computer, comprising:
The program causes the computer to recognize the emotional state of the user based on the voice data input from the voice input means, perform input for causing the terminal to perform a desired process, and respond to the recognition result of the emotional state. A job that is currently being processed and a job received from the process input step are compared to determine a priority of each job. 29. The program according to claim 28, wherein when the user's emotion is in a non-normal state, the program gives priority to a job received from the processing input step to a job currently being processed. Recording medium. 30. The program according to claim 26, wherein the communication with the computer is performed when the emotion of the user is in a non-normal state.
29. The communication device according to claim 28, wherein the communication unit is instructed to change the number of packets to a value smaller than the current value, and the job received in the processing input step is preferentially processed between communications. A recording medium according to claim 1. 31. The program according to claim 31, wherein the program causes the computer to recognize the emotional state of the user together with the voice data as well as the image data such as the face color input from the image data input means and the voice data. 31. The recording medium according to any one of 28 to 30. 32. Voice input means, a first character recognition dictionary for normal use in which the user's emotions are relatively quiet, and an unusual use in which the user's emotions are unstable due to anger or the like. A program for use in a character recognition device having a second character recognition dictionary for time use, the program comprising: an emotion for recognizing a user's emotional state based on voice data input from the voice input means; A recognition step; a pen input step of pressing a writing pen against a touch panel to input handwritten characters; and a first character recognition dictionary and a second character recognition dictionary according to a recognition result of the emotion recognition step. A dictionary selecting step of selecting either one of the first and second character recognition dictionaries selected in the dictionary selection step; A program characterized by comprising: a character recognition step for recognizing handwritten characters, and a recognition result display step of displaying the recognized character resulting from the recognition in the character recognition step. 33. The character recognition step, wherein when the recognition character candidates are displayed in the recognition result display step, the recognition candidates to be displayed when the user's emotions are normal and non-normal. 33. The program according to claim 32, wherein the number of characters is changed. 34. The program according to claim 33, wherein in the character recognition step, the number of the recognition candidate characters is increased when the user's emotion is non-normal. 35. A program used in a character recognition device having voice input means and a character recognition dictionary for recognizing handwritten characters, wherein the program is used based on voice data input from the voice input means. Recognition step for recognizing the emotional state of the person, pen input step of pressing a writing pen on the touch panel to input handwritten characters, character number determination for determining the number of recognition candidate characters according to the recognition result in the emotion recognition step A character recognition step of comparing the character recognition dictionary with handwritten character data input in the pen input step and extracting recognition candidate characters of the number of recognition candidate characters determined in the character number determination step, Displaying a recognition candidate character extracted in the character recognition step. Program. 36. The program according to claim 35, wherein in the character number determining step, the number of recognition candidate characters is increased when a user's emotion is non-normal. 37. The emotion recognizing step includes recognizing a user's emotional state together with voice data as well as image data such as a face color input from image data input means and voice data. The program according to any one of Items 36 to 36. 38. A program for use in a portable terminal device, the program comprising: an emotion recognition step of recognizing an emotional state of a user based on voice data input from voice input means; A process input step of performing an input for causing a job to be processed, and a job currently being processed is compared with a job received from the process input step to determine a priority of each job in accordance with a recognition result in the emotion recognition step. A program comprising a priority order determining step. 39. The method according to claim 38, wherein in the priority order determining step, when the emotion of the user is not normal, the job received from the processing input step is given priority over the job currently being processed. Program. 40. In the priority order determining step, when communication is being performed when the user's emotion is non-normal, an instruction is sent to the communication means to change the number of packets to a value smaller than the current value. 39. The program according to claim 38, wherein a job received in the processing input step is preferentially processed between communications. 41. The emotion recognition step includes recognizing a user's emotional state together with voice data as well as image data such as a face color input from image data input means in the emotion recognition step. 41. The program according to any one of items 40 to 40.