JP2010170501A - Portable device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow a user to select a conversion input candidate according to an image recognition result. <P>SOLUTION: A portable device includes an image display means (a display control unit 62) for displaying images obtained by an imaging means and images stored in a storage means; an image recognition means (an image recognition unit 64) for recognizing the images; a conversion input candidate acquisition means (a conversion input candidate control unit 66) for acquiring conversion input candidates according to the recognition result of the images; a display control means (a display control unit 62) for simultaneously displaying the recognition images and the conversion input candidates; and a conversion input selection means (a conversion input candidate control unit 66) for selectively inputting the conversion input candidates therefrom. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a portable device that can selectively input from conversion input candidates according to an image recognition result.

  Since the conventional mobile terminal device has a limited number of character selection buttons, a large number of characters are assigned to one character selection button so that different characters can be input according to the number of times the character selection button is pressed. It has become. Therefore, when inputting a desired text, it is necessary to press each character selection button many times, which is very troublesome.

  Therefore, it is possible to input various texts (hereinafter referred to as text and expressions) including characters, character strings, words, phrases, pictograms, characters, and reduced images without frequently pressing the character selection button. Some have been developed. One of them is a text input technique based on image recognition. In this technique, a predetermined image is recognized, and text corresponding to the recognized result is acquired from a memory and input. For this reason, the effort at the time of inputting a text can be reduced significantly.

  In Patent Document 1, a plurality of types of messages are stored in a storage unit in association with images of a predetermined pose or gesture, and when a call arrives, the pose or gesture is recognized from the image captured by the camera unit, and the recognized pose is detected. Alternatively, a technique is disclosed in which a message corresponding to a gesture is selected from a storage unit, and the selected message is transmitted to a source terminal. According to this technique, in a situation where a user of a mobile terminal device having a call function cannot respond to an incoming call, it is possible to easily inform the caller of the situation where the user cannot respond.

  In the technique of Patent Document 2, when imaging is performed with a mobile phone camera facing a landscape, image analysis processing is performed based on the captured image, and average brightness and RGB average values are calculated. Next, the peripheral information transmitted from the base station is received. A message corresponding to the image analysis result determination condition and the peripheral information determination condition satisfying the image analysis result and the peripheral information is read from the output message determination table. Then, the voice according to the read message “Today is good weather” is output from the speaker, and the message is further displayed on the display unit. This realizes a captured image processing apparatus or the like that produces various effects based on the captured image and the captured location.

  Patent Document 3 discloses an image analysis system including an operation server and an analysis server managed by an operator, and communication terminals of a large number of users. The operation server publishes a Web page on the Internet, and browses a photographic image with a message for each registered member. The operation server stores the image-attached e-mail sent to this site in the image database in association with the mail document data and image data. The analysis server analyzes the RGB data and CMYK values for each bit of the image data in the image database, calculates the average value of the entire image data, calculates the average feature color, and extracts adjectives from the document data, Each is stored in a color database associated with each. Average feature colors and adjectives can be extracted mechanically based on one. As a result, a database in which images, colors and words are associated with each other objectively and classified is constructed, and images, colors and words are mechanically extracted.

  In the technique of Patent Document 4, when the start of emoticon input is selected under the text creation mode, the imaging mode is activated while text data is stored. When a subject such as a human face is imaged by the built-in digital camera in the imaging mode, processing for extracting face image data from the image data is performed, and the imaging mode is terminated. Search the emoticon corresponding to the extracted face image data using the stored face image emoticon correspondence table, and if there is a corresponding emoticon, the text for which the character string data constituting the emoticon is being created Append to data. Thus, there is provided an emoticon input device that enables emoticons to be input by allowing emoticons corresponding to the face image data extracted from the image data to be input.

JP 2005-303832 A (publication date: October 27, 2005) JP 2005-109995 A (publication date: April 21, 2005) JP 2005-182703 A (publication date: May 5, 2005) JP 2007-199908 A (publication date: August 9, 2007)

  However, in the conventional technique, when the user views the displayed text, the image to be recognized has already disappeared from the screen. Therefore, there arises a problem that the user cannot easily determine whether the text selected according to the image recognition result surely matches the content of the image.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide a portable device including a conversion input method that allows a user to select conversion input candidates according to an image recognition result. There is.

In order to solve the above problems, a portable device according to the present invention
An image display means for displaying an image acquired by the imaging means and an image in the storage means;
Image recognition means for recognizing the image;
Conversion input candidate acquisition means for acquiring conversion input candidates according to the recognition result of the image;
Display control means for simultaneously displaying the image and the conversion input candidate;
Conversion input selection means that can select and input from the conversion input candidates is provided.

  According to the above configuration, the conversion input candidate corresponding to the recognition result of the image is displayed on the display unit of the portable device together with the image to be recognized. Therefore, the user can confirm the recognized image and the corresponding conversion input candidate at the same time. Thus, the user can easily determine whether or not the displayed conversion input candidate is appropriate that matches the content of the recognized image.

In the portable device according to the present invention,
It is preferable that the apparatus further comprises display control means capable of switching the input method of the conversion input candidate to an input method based on pressing of a character selection button when a predetermined key is pressed.

  Thereby, according to the situation, the user can select characters, character strings, words, phrases, by a method of directly inputting characters by pressing a character selection button, instead of a method of selecting and inputting from conversion input candidates according to the image recognition result. Various texts including pictographs, characters, and reduced images (hereinafter: text and expressions) can be input. Therefore, the flexibility of the text input can be further increased.

In the portable device according to the present invention,
The conversion input candidate according to the recognition result of the image is displayed together with the input candidate at the time of character input based on pressing of the character selection button after the switching process by the display control unit, and the conversion input It is preferable to provide conversion input selection means that can select and input from candidates.

  According to the above-described configuration, the number of conversion input candidates that can be selected by the user is increased, so that diversity during text input can be increased.

In the portable device according to the present invention,
Using the display control means capable of simultaneously displaying the image, the conversion input candidate according to the recognition result of the image, and the conversion input candidate in the input method based on pressing of the character selection button, it is possible to selectively input from the conversion input candidates It is preferable to provide conversion input selection means.

  According to the above-described configuration, the number of conversion input candidates that can be selected by the user is increased, so that diversity during text input can be increased.

In the portable device according to the present invention,
Using the display control means for displaying the conversion input candidate according to the recognition result of the image and the conversion input candidate at the time of character input based on pressing of the character selection button in different display forms, and using the display control means It is preferable to provide conversion input selection means that can select and input from input candidates.

  According to the above configuration, the user can easily distinguish the conversion input candidate according to the image recognition result from other conversion input candidates.

In the portable device according to the present invention,
It further comprises time acquisition means for acquiring the current time,
It is preferable that the apparatus includes a conversion input selection unit that can selectively input from the conversion input candidates using display control means for displaying the conversion input candidates according to the recognition result of the image and the acquired current time.

  According to the above configuration, the portable device can display and input a morning greeting text corresponding to, for example, a morning time zone as a conversion input candidate. Therefore, the user can input an appropriate text input according to the current time.

Moreover, the portable device according to the present invention further includes:
A history determination means for determining whether there is a history of predetermined processing in the portable device;
Preferably, the conversion input candidate acquisition unit and the display control unit acquire and display the conversion input candidate according to the recognition result of the image and the presence or absence of the history.
Furthermore, it is preferable to include conversion input selection means that can select and input from the acquired and displayed conversion input candidates.

  According to the above configuration, conversion input candidates corresponding to the presence / absence of a history of predetermined processing (for example, e-mail transmission processing) in the portable device are acquired and displayed. For example, if there is no other e-mail transmission history in the past specific time zone for the e-mail transmission partner currently being created in the portable device, there is no mail transmitted to the other party at the specific time (for example, in the morning). Then, conversion input candidates (for example, morning greetings) that are considered to be input in a mail sent for the first time in the morning are acquired and displayed. On the other hand, if there is such a history, since the mail has already been sent to the other party in the morning time, it is determined that the input of the morning greeting is unnecessary, and conversion input candidates other than the greeting are acquired and displayed, Selectable input from conversion input candidates. As a result, the user can input the text suitable for the current situation from the acquired and displayed conversion input candidates with a simple operation.

In the portable device according to the present invention,
Brightness measuring means for measuring ambient brightness is further provided, and the conversion input candidate acquisition means and the display control means acquire the conversion input candidates according to the recognition result of the image and the measured brightness. And display. Furthermore, it is preferable to include conversion input selection means that can select and input from the acquired and displayed conversion input candidates.

  According to the above configuration, the portable device acquires and displays conversion input candidates corresponding to both the image recognition result and the measured brightness, and can select and input from the conversion input candidates. For example, when the image falls below the standard brightness, conversion input candidates corresponding to these recognition results and measurement results are acquired and displayed, and can be selected and input from the conversion input candidates. Therefore, the suitability of the conversion input candidate can be further improved as compared with the case where the conversion input candidate is acquired, displayed, and input based only on the image recognition result.

  The portable device according to the present invention preferably further includes storage means for storing the captured image and the acquired conversion input candidate in association with each other.

  In the portable device according to the present invention, it is preferable that the conversion input candidate acquisition unit acquires image information included in the captured image as a conversion input candidate corresponding to the image.

  According to the said structure, the kind of conversion input candidate acquired and displayed can be increased more.

  Further, in the portable device according to the present invention, it is preferable that the imaging of the image and the recognition of the image are started when the portable device is stationary for a predetermined reference time. Alternatively, it is preferable that when the imaging button of the portable device is pressed by the user, the imaging of the image and the recognition of the image are started. Alternatively, it is preferable that the image is recognized every predetermined time while the image is continuously captured.

  Moreover, in the portable device according to the present invention, it is preferable that the display of the conversion input candidate is terminated when the imaging of the image is terminated.

  As described above, the portable device according to the present invention can allow the user to select conversion input candidates according to the image recognition result.

It is a block diagram which shows the principal part structure of the mobile telephone which concerns on one Embodiment of this invention. It is a figure which shows the external appearance of a mobile telephone. It is a figure showing each screen displayed on the main display part at the time of mail preparation. It is a figure showing each screen displayed on the main display part at the time of mail preparation. 6 is a flowchart showing a flow of processing when inputting text according to an image recognition result and a current time in a mobile phone. It is a flowchart which shows the flow of a process at the time of acquiring another conversion input candidate from the same image in a mobile telephone, and showing it to a user. It is a flowchart which shows the flow of a process when a mobile telephone switches a text input system. It is a flowchart which shows the flow of a process when a mobile telephone recognizes an image based on the set priority mode, and inputs the conversion input candidate according to the said recognition result.

  An embodiment according to the present invention will be described below with reference to FIGS.

  FIG. 1 is a block diagram showing a main configuration of a mobile phone 1 according to an embodiment of the present invention. As shown in this figure, the mobile phone 1 includes a radio antenna 2, a radio unit 4, a communication control unit 6, a control unit 8, an Ir transmission unit 10, an Ir reception unit 12, an indicator 14, a memory 16, a large capacity memory 18, TV antenna 20, TV tuner 22, power supply control unit 24, battery 25, program memory 26, input operation unit 28, main camera 30, sub camera 32, main display unit 34, sub display unit 36, state detection unit 38, vibration sensor 40, a GPS antenna 42, a GPS unit 44, a microphone 46, a speaker 48, an audio processing unit 50, an RFID antenna 52, an RFID reader 54, an illuminance sensor 56, and a distance sensor 58.

  The wireless unit 4 performs wireless communication through the wireless antenna 2. The communication control unit 6 controls wireless communication by the wireless unit 4.

  The control unit 8 comprehensively controls each member provided in the mobile phone 1. The control unit 8 includes a mail processing unit 60, a display control unit 62, an image recognition unit 64, and a conversion input candidate control unit 66. The role and function of these members will be described later.

  The Ir transmitter 10 transmits infrared rays. The Ir receiver 12 receives infrared rays.

  The indicator 14 indicates the state of the mobile phone 1 by lighting.

  The memory 16 stores various data and information. In this embodiment, a number preset, a phone book, a preset message, address data, wallpaper data, mail history, phone history, location information, and action history are stored. Since these data or information are publicly known, further detailed description is omitted.

  The large-capacity memory 18 stores large-size data and information that cannot be stored in the memory 16. In this embodiment, map information, image information, and character candidate information are stored.

  The television tuner 22 receives a television broadcast through the television antenna 20.

  The power supply control unit 24 controls the power supply in the mobile phone 1 based on the power supplied from the battery 25.

  The program memory 26 stores various programs (software) executed on the mobile phone 1.

  The input operation unit 28 receives an input operation on the mobile phone 1 by the user.

  The main camera 30 includes an optical system 30a, an imaging unit 30b, and a video signal processing unit 30c. With this configuration, an imaging target outside the mobile phone 1 is imaged. The sub camera 32 includes an optical system 32a, an imaging unit 32b, and a video signal processing unit 32c. With this configuration, an imaging target outside the mobile phone 1 is imaged.

  The main display unit 34 and the sub display unit 36 are both so-called displays that display information such as various texts and images. For example, it is mounted as a liquid crystal display or a plasma display.

  The state detection unit 38 detects the state of the housing of the mobile phone 1. The “case state” here means, for example, a folded state, a rotated state, a cycloid state, or a slide state.

  The vibration sensor 40 detects a vibration state in the mobile phone 1 and outputs a signal corresponding to the degree of vibration.

  The GPS unit 44 receives information indicating the current position of the mobile phone 1 through the GPS antenna 42. Thereby, the mobile phone 1 realizes a GPS (Global Positioning System) function.

  The microphone 46 converts audio outside the mobile phone 1 into an electrical signal. The speaker 48 outputs sound. The voice processing unit 50 controls voice recording by the microphone 46 and voice output by the speaker 48.

  The RFID reader 54 reads information from the wireless tag through the RFID antenna 52.

  The illuminance sensor 56 detects the illuminance around the mobile phone 1 and outputs an electrical signal corresponding to the detected illuminance.

  The distance sensor 58 measures the distance between the measurement object and the mobile phone 1 and outputs an electrical signal corresponding to the measured distance.

(Overview of mobile phone 1)
FIG. 2 is a diagram showing an overview of the mobile phone 1. As shown in FIG. 2A, the mobile phone 1 has a foldable structure and has two housings. One housing is provided with a main display unit 34, and the other housing is provided with an input operation unit 28. The input operation unit 28 includes a determination key 70, a multi guide key 72, a left soft key 74, a right soft key 76, and a character selection button 78. In addition, as shown in FIG. 2B, the main camera 30 is provided facing the outside of the mobile phone 1 when the mobile phone 1 is folded.

  The user can use the mobile phone 1 to create a mail or upload an article on a blog. At that time, write the body of the email or article. For this writing, various texts including characters, character strings, words, phrases, pictograms, characters, reduced images, and the like (hereinafter: text and expression) are input. The mobile phone 1 has a function for facilitating these inputs. The function will be described below with reference to FIGS.

(Email text creation screen)
3 and 4 are diagrams showing screens displayed on the main display unit 34 when mail is created.
FIG. 5 is a flowchart showing the flow of processing when the mobile phone 1 inputs text according to the image recognition result and the current time.

  As shown in FIG. 5, the mail processing unit 60 activates a mail creation function (step S1). As a result, the display control unit 62 displays the mail creation screen on the main display unit 34 (FIG. 3A). In this screen, a text display area 80 and a menu display area 82 are set. In the text display area 80, text input by the user is displayed. In the menu display area 82, a plurality of commands that can be selected by the user are displayed side by side. In this embodiment, three commands are displayed. In FIG. 3A, “menu”, “complete”, and “cancel” are arranged in order from the left as viewed from the user.

  When the user presses the left soft key 74 in the state of FIG. 3A, the “menu” displayed in the menu display area 82 is selected. When this menu is expanded hierarchically, there is an item “image citation”, which the user selects. Then, the control part 8 starts an image citation function (step S2).

  When the image quoting function is activated, the mobile phone 1 first operates the main camera 30 to image the periphery of the mobile phone 1. Next, a screen for text input by image citation is displayed on the main display unit 34. (Step S3, FIG. 3B).

  This screen includes an image display area 84 and a conversion input candidate display area 86 in addition to the text display area 80 described above. The display control unit 62 displays the image captured by the main camera 30 in the image display area 84 (step S4). In the example of FIG. 3B, an image obtained by capturing a blue sky through the curtain is displayed. The menu display area 82 displays commands for “recognition switching”, “decision”, and “image switching” in order from the left as viewed from the user.

  After displaying the image, the image recognition unit 64 recognizes the image (step S5). When the screen of FIG. 3B is displayed, the mobile phone 1 operates in a mode for recognizing an image with priority on brightness / color tone. Therefore, when recognizing an image, information on brightness and color tone included in the image is preferentially extracted. An icon representing the currently set mode may be displayed somewhere on the screen.

  After recognizing the image, the control unit 8 detects the current time (step S6). This time is obtained from, for example, a local clock in the mobile phone 1. Next, the control unit 8 determines whether or not the current time is morning (step S7). In the mobile phone 1, it is predetermined that the time in a predetermined time zone (for example, 7:00 to 9:00) is morning. Therefore, the control unit 8 determines whether or not the detected current time is a time within a predetermined time zone determined to be morning.

  When the result of the determination in step S <b> 7 is “false” (NO), the mobile phone 1 ends the process shown in FIG. 5. In addition, you may perform the process according to time zones other than the morning instead of complete | finishing a process. For example, the mail transmission history during the daytime or night time period is confirmed, and processing according to the presence or absence of the history is executed. Specifically, it is determined whether or not the current time is a noon time zone (for example, from 12:00 pm to 14:00 pm). It is determined whether there is a history of sent mail. If there is no mail sent the day of the time period if, to get because it is the first time send e-mail to the noon of time zone, a text such as "Hello" as a conversion input candidates as the day of the salutation should be input in the mail body, It is displayed in the conversion input candidate display area 86. As a result, the user can input a noon greeting in the mail text with a simple operation.

  On the other hand, when the result of determination in step S7 is “true” (YES), the mail processing unit 60 searches the memory 16 for the mail transmission history of this morning regarding the mail transmission partner (step S8). And it is determined whether there exists the said log | history (step S9). Here, this morning is the morning of the day when the mobile phone 1 is operating, and the “morning” is the time within the predetermined time zone (for example, 7:00 to 9:00) described above. It is predetermined in the telephone 1.

  When the result of determination in step S <b> 9 is “true” (YES), the mobile phone 1 ends the process shown in FIG. 5. That is, since there is already a morning mail transmission history (that is, mail has already been sent to the other party in the morning time), it is considered that a morning greeting has already been entered in such sent mail. As a result, it is determined that there is no need to input the morning greeting again in the mail text currently being created. According to this determination, conversion input candidates for morning greetings are not acquired.

  On the other hand, when the result of determination in step S9 is “false” (NO), the conversion input candidate control unit 66 acquires a plurality of morning greetings as a plurality of conversion input candidates from the large-capacity memory 18 (step S10). ). Here, in the large-capacity memory 18 of the mobile phone 1, the morning greeting text includes the image recognition result shown in FIG. 3B (when priority is given to brightness / color tone), and the morning time zone (7 o'clock). To 9 o'clock) are stored in advance in association with each other.

  The display control unit 62 displays the plurality of conversion input candidates acquired by the conversion input candidate control unit 66 in the conversion input candidate display area 86 (step S11, FIG. 3C). In the example of FIG. 3B, a total of four greetings such as “Good morning” and “Good morning” are displayed as conversion input candidates. Here, when the image shown in FIG. 3B is recognized, and it is determined that the image exceeds the brightness based on the reference, the pictogram (sun May be obtained and displayed as conversion input candidates. An example of this display is shown in FIG. In FIG.3 (d), unlike FIG.3 (c), the sun pictogram is uniformly added to the end of each greeting.

  When the image shown in FIG. 3B is recognized, and the image is determined to be lower than the standard brightness, a pictograph (cloud cloud) is added at the end of these morning greetings. A picture) may be added and acquired and displayed as conversion input candidates.

  Further, the mobile phone 1 may perform such brightness determination based on the output from the illuminance sensor 56. Further, the determination accuracy may be further improved by combining the determination of brightness by image recognition and the determination of brightness based on the output of the illuminance sensor 56.

  The user selects one of the plurality of conversion input candidates displayed in the conversion input candidate display area 86 to be input (step S12). Specifically, the multi-guide key 72 is pressed to move the cursor up and down, and the cursor is brought to the position of the selected conversion input candidate, whereupon the enter key 70 is pressed. As a result, the display control unit 62 displays the conversion input candidate selected by the user in the text display area 80 (FIG. 3E). In the example of FIG. 3E, the text “Ohayo-yo” is selected and displayed. As a result, the user can input a morning greeting by a simple operation in the mail sent to the partner for the first time in the morning.

(Switch the acquisition source of recognition target images)
The mobile phone 1 can change the image to be recognized from one captured by the main camera 30 to one captured by the sub camera 32 or one stored in the large-capacity memory 18 at any time. As shown in the screen of FIG. 3C, in the menu display area 82, “recognition switching”, “decision”, and “image switching” are arranged in order from the left as viewed by the user as commands that can be selected by the user. Yes.

  When the user presses the left soft key 74 in the state of FIG. 3C, “recognition switching” displayed in the menu display area 82 is selected. Then, this menu is expanded, and sub-items “brightness”, “shape”, and “position” are displayed. A check mark may be added to the currently selected sub-item. Here, when the user selects “brightness”, the mobile phone 1 operates in the brightness / color tone priority mode. On the other hand, when the user selects “shape”, the mobile phone 1 operates in the shape / contour priority mode. When the user selects “position”, the mobile phone 1 operates in the position priority mode. At this time, the mobile phone 1 can acquire information indicating the current position from a GPS satellite using the GPS unit 44.

  When the user presses the right soft key 76 in the state of FIG. 3C, “image switching” displayed in the menu display area 82 is selected. Then, this menu is expanded, and sub items “main camera”, “sub camera”, and “built-in image” are displayed. The cellular telephone device 1 recognizes an image corresponding to an item selected by the user. Specifically, when the user selects “main camera”, the mobile phone 1 switches the image to be recognized to the image of the main camera 30. On the other hand, when the user selects “sub camera”, the image to be recognized is switched to the image of the sub camera 32. When the user selects “built-in image”, the image to be recognized is switched to an image stored in the large-capacity memory 18.

  Furthermore, when the mobile phone 1 switches the recognition target image to one captured by the sub camera 32 in a state where the conversion input candidate corresponding to the recognition result of the image captured by the main camera 30 is displayed, the main camera 30 The conversion input candidate corresponding to the image recognition result of the image and the conversion input candidate corresponding to the image recognition result of the sub camera 32 may be simultaneously displayed in the conversion input candidate display area 86. The same applies even if the switched image is an image in the large-capacity memory 18. Thereby, since the number of conversion input candidates increases, the user can select the text that he / she wants to actually input from among a wider variety of options.

(Action / Effect)
As described above, the user of the mobile phone 1 can easily input text to be input without trouble by simply capturing (recognizing) an image. Since this conversion input candidate is related to both the image recognition result and the information indicating the current status of the mobile phone 1, the user can input text corresponding to the current status quickly and in real time. it can.

  In addition, since all the conversion input candidates are displayed on the main display unit 34 together with the citation source image (recognition target image), the user can confirm whether the presented conversion input candidates are appropriate to match the content of the image. It is possible to quickly determine whether or not. Therefore, the mobile phone 1 can more reliably input text based on the image recognition result.

  There may be at least one conversion input candidate displayed in the conversion input candidate display area 86, and there is not necessarily a plurality of conversion input candidates. However, it is desirable to use a plurality. When a plurality of conversion input candidates are acquired and displayed in the conversion input candidate display area 86, the user can input a wide variety of texts, so that the width of text input can be expanded.

  Further, since the user can input text corresponding to the image recognition result as long as the image is captured (recognized), it is not necessary to press a character key for text input. This can reduce the number of key presses during text input.

  Also, conversion input candidates corresponding to the presence / absence of a history of predetermined processing (for example, transmission of e-mail) in the mobile phone 1 are acquired and displayed. In the present embodiment, the predetermined process is an e-mail transmission, but the present invention is not limited to this and may be another process. In addition, the history is a transmission history of other e-mails in a specific time zone in the past with respect to the e-mail transmission partner currently being created, but is not limited thereto, and other histories (for example, reception histories) Good. In any case, conversion input candidates corresponding to the presence or absence of the history are acquired and displayed, so that the user can input text suitable for the current situation with a simple operation.

  Also, if there is a mail transmission history of this morning regarding the mail transmission partner, the morning greeting is not entered in the mail body. If there is already a history, there is a high possibility that the previous mail was sent with a greeting. Such a device can make the mail text more natural.

  The mobile phone 1 also detects the conversion input candidate and presents it to the user in consideration of the current time in addition to the recognition result of the captured image. Thereby, the process which shows a morning greeting as a conversion input candidate to a user in the morning time slot | zone can be performed more reliably.

(Continue to display other conversion input candidates)
After completing the process shown in FIG. 5, the cellular phone 1 continues to execute the process shown in FIG. In the process of FIG. 6, recognition processing is executed from the same captured image in different priority modes, and a plurality of different conversion input candidates corresponding to new recognition results are presented to the user. FIG. 6 is a flowchart showing a flow of processing when the mobile phone 1 acquires another conversion input candidate from the same image and presents it to the user.

  First, the display control unit 62 continues to display the image shown in FIG. 3E (step S21). By doing so, the user is made to know what kind of image recognition result the conversion input candidate is displayed on. Next, based on the image recognition result, it is determined whether or not the image is bright, that is, whether or not the reference brightness is exceeded (step S22). When the determination result in step S22 is “false” (NO), the conversion input candidate control unit 66 acquires a conversion input candidate representing weather such as cloudy or rain from the large-capacity memory 18 as a conversion input candidate ( Step S24). Such conversion input candidates are stored in advance in the large-capacity memory 18 in association with the recognition result when the image falls below the standard brightness.

  On the other hand, when the result of determination in step S22 is “true” (YES), the image recognition unit 64 determines whether the image is blue based on the image recognition result (step S23). Here, when the result of the determination in step S23 is “false” (NO), the conversion input candidate control unit 66 executes the process of step S24. In the large-capacity memory 18, conversion input candidates representing cloudy or rain-related weather are stored in advance in the large-capacity memory 18 in association with the recognition result when the image is other than blue.

  On the other hand, when the result of determination in step S24 is “true” (YES), the conversion input candidate control unit 66 acquires a conversion input candidate representing sunny weather from the large-capacity memory 18 as a conversion input candidate ( Step S25). Such conversion input candidates are stored in advance in the large-capacity memory 18 in association with the recognition result when the image exceeds the reference brightness and is blue.

  The display control unit 62 displays the conversion input candidates acquired by the conversion input candidate control unit 66 in the conversion input candidate display area 86 (step S26, FIG. 3 (e)). In the example of FIG. 3E, sentences such as “It's good weather” and “It looks like the weather is good” are displayed as conversion input candidates representing sunny weather. At this time, the determination system may be further enhanced by reading the direction of the mobile phone 1 with a three-axis sensor (not shown) and determining whether or not the sky direction is imaged.

  The user operates the multi guide key 72 and the enter key 70 to select one of conversion input candidates (step S27). Here, “Weather is good” is selected. The display control unit 62 displays the conversion input candidate selected by the user in the text display area 80 (step S28, FIG. 3 (f)).

  Next, the mobile phone 1 switches the image of the conversion input candidate citation source. Specifically, the user operates the mobile phone 1 and images the coffee cup with the main camera 30. Thereby, the display control unit 62 displays the image of the coffee cup as the next image in the image display area 84 (step S28, FIG. 3F). Here, when the user presses the left soft key 74, “recognition switching” displayed in the menu display area 82 is selected. Then, this menu is expanded, and sub-items “brightness”, “shape”, and “position” are displayed. Here, the user selects “shape”. By this selection, the cellular phone 1 operates in the shape / contour priority mode. Thereby, when recognizing an image, the image recognition unit 64 preferentially executes a recognition process based on detection of a shape or a contour.

  The image recognition unit 64 recognizes the next displayed image (step S29). In the example of FIG. 3 (f), the shape of the coffee cup is recognized. Thereby, the conversion input candidate control unit 66 acquires the conversion input candidate corresponding to the recognition result from the large-capacity memory 18. As a result, the display control unit 62 displays the conversion input candidate corresponding to the recognition result in the conversion input candidate display area 86 (FIG. 3G). In the example of FIG. 3G, sentences such as “from now on” and “breakfast and eating” are displayed as conversion input candidates. Note that these conversion input candidates are stored in the large-capacity memory 18 in advance in association with the recognition result representing the shape of the coffee cup.

  As described above, the mobile phone 1 displays different conversion input candidates corresponding to different situations by the recognition processing corresponding to different priority modes from the same recognition image, and allows the user to select. Therefore, the user can input a wide variety of texts.

(Switching text input method)
The mobile phone 1 can switch between the above-described image citation type text input method and key press type text input method at any time. The processing flow in this case will be described below with reference to FIG. FIG. 7 is a flowchart showing the flow of processing when the mobile phone 1 switches the text input method.

  As shown in this figure, the mobile phone 1 first acquires and displays conversion input candidates by an image citation method. This process is as shown in steps S31 to S36, and details of this process have already been described, and are omitted here. It is assumed that the screen of FIG. 3B is displayed on the main display unit 34 when the process of step S36 is completed.

  After displaying the conversion input candidates, the control unit 8 determines whether or not the character selection button 78 has been pressed by the user (step S37). When the determination result in step S37 is “false” (NO), the mobile phone 1 selects a conversion input candidate (step S38). Specifically, the user operates the multi guide key 72 to select one of a plurality of conversion input candidates, and the mobile phone 1 accepts the selection process. The control unit 8 displays the selected conversion input candidate in the text display area 80. Next, the control unit 8 determines whether or not to end the text input (step S39). For example, when the user selects the “End” menu, the text input is ended. Here, when the result of the determination in step S39 is “false” (NO), the processing in FIG. 7 returns to step S33. That is, the mobile phone 1 repeats the text input process using the image citation method. On the other hand, when the result of the determination in step S <b> 39 is “true” (YES), the mobile phone 1 ends the process shown in FIG. 7.

  Here, when the result of the determination in step S37 is “true” (YES), the mobile phone 1 switches the text input method from the current image citation method to the normal character text input method. The normal method here is a text input method based on character input by pressing the key of the character selection button 78. In this method, the character assigned to the key pressed by the user is directly input. By this switching process, the display control unit 62 displays a normal text input screen on the main display unit 34 (step S40). Here, in the present embodiment, it is assumed that the display control unit 62 displays the screen of FIG. 4A on the main display unit 34 while the mobile phone 1 is executing the text citation method text input process. In this case, in step S40, the display control unit 62 displays the screen of FIG. 4B instead of the screen of FIG. In the screen of FIG. 4B, a text display area 80, a menu display area 82, and a conversion input candidate display area 86 are set.

  The user inputs two characters “sa” and “n” by pressing the key of the character selection button 78, and these characters are displayed in the text display area 80 as temporary input text. Here, the display control unit 62 displays a plurality of texts in the conversion input candidate display area 86 as conversion input candidates corresponding to these input characters (step S41). Since the conversion input candidate display area 86 in FIG. 4B is wider than that in FIG. 4A, more conversion input candidates are displayed simultaneously.

  The display control unit 62 also displays the conversion input candidates acquired according to the immediately preceding image authentication result in the conversion input candidate display area 86 in addition to the conversion input candidates corresponding to the characters input by pressing the key. It is also possible (FIG. 4C). For example, in FIG. 4C, “good weather” and “good weather” are displayed in addition to “three” and “reference”. Thereby, since the range of conversion input candidates spreads, the input efficiency can be improved. The conversion input candidates corresponding to the image recognition results are desirably displayed in a form different from the conversion input candidates based on key presses. For example, the fonts or colors are different from each other. Thereby, a user can distinguish both easily.

  The image display area 84 is not set on the screen of FIG. Therefore, when displaying the screen of FIG. 4B, the display control unit 62 does not display the image displayed in FIG. 4A (that is, the image to be recognized). Instead, a predetermined image citation icon 88 is displayed in the conversion input candidate display area 86. That is, the image quotation icon 88 is displayed along with a plurality of conversion input candidates.

  In the state where the screen of FIG. 4B is displayed, the user operates the multi guide key 72 to select one conversion input candidate (step S42). As a result, the display control unit 62 displays the selected conversion input candidate in the text display area 80. In this case, since the user has selected “san”, the display control unit 62 displays “san” next to “good morning” in the text display area 80 (FIG. 4D). In the example of FIG. 4D, the display control unit 62 predicts a text that is likely to be input after inputting “san” and displays it in the text display area 80 as a new conversion input candidate.

  After the process of step S42 is complete | finished, the control part 8 determines whether the image quotation icon 88 was selected by the user (step S43). Here, when the result of the determination in step S43 is “true” (YES), the processing of FIG. 7 returns to step S31. Thereby, the text input method in the mobile phone 1 returns to the image citation type text input method. Therefore, the cellular phone 1 can switch between the key press type text input method and the image citation type text input method at any time. As a result, since the user can select any input method according to the situation, the convenience at the time of text input can be further improved.

  On the other hand, when the result of the determination in step S43 is “false” (NO), the control unit 8 determines whether or not to end the text input (step S44). Here, when the user presses the enter key 70 (that is, when the “complete” button in the menu display area 82 is pressed), the control unit 8 determines to end the text input. Therefore, the process shown in FIG.

  On the other hand, when the user presses the right soft key 76 (that is, when the “cancel” button in the menu display area 82 is pressed), the control unit 8 confirms whether or not the text input can really be terminated. For this purpose is displayed on the main display unit 34. For example, the message “Do you want to finish text entry? (Yes / No)” is displayed. When the user selects “Yes”, the control unit 8 determines to end the text input, and thus the process illustrated in FIG. 7 ends. On the other hand, when the user selects “No”, the processing shown in FIG. 7 returns to Step S41.

(Simultaneous display of recognition target image and conversion input candidate)
Note that the display control unit 62 displays an image to be recognized, a conversion input candidate (candidate A) corresponding to the recognition result of the image, and a conversion input candidate (candidate B) for an input method based on pressing of the character selection button 78. Can also be displayed on the main display unit 34 at the same time. For example, images are displayed in the image display area 84 and candidates A and B are displayed in the conversion input candidate display area 86. Thus, the user selects and inputs from the displayed conversion input candidates. As a result, the number of conversion input candidates that can be selected by the user is increased, so that the diversity at the time of text input can be increased.

(Priority mode for image recognition)
When recognizing an image, the mobile phone 1 can set in advance what is prioritized. In this embodiment, it is set that priority is given to any one of brightness / color tone, shape / contour, or position.

  FIG. 8 is a flowchart showing a flow of processing when the mobile phone 1 recognizes an image based on the set priority mode, and acquires and displays a conversion input candidate according to the recognition result.

  Since steps S51 to S53 shown in FIG. 8 have already been described, description thereof is omitted here. After displaying the recognition target image, the user inputs a priority mode for image recognition (step S54). Here, the user presses the left soft key 74. As a result, “recognition switching” displayed in the menu display area 82 is selected. Then, this menu is expanded, and sub-items “brightness”, “shape”, and “position” are displayed. The user selects a desired sub-item, thereby inputting a priority mode for the mobile phone 1 to recognize an image.

  The image recognition unit 64 determines whether or not the input mode is the brightness / tone priority mode (step S55). When the determination result in step S55 is “true” (YES), the image recognition unit 64 sets the mode to the brightness / color tone priority mode (step S56).

  On the other hand, when the determination result in step S55 is “false” (NO), the image recognition unit 64 determines whether or not the input mode is the shape / contour priority mode (step S57). When the determination result in step S57 is “true” (YES), the image recognition unit 64 sets the mode to the shape / contour priority mode (step S58). On the other hand, when the result of determination in step S is “false” (NO), the image recognition unit 64 sets the mode to position priority mode (step S59).

  The image recognition unit 64 recognizes an image according to the set priority mode (step S60). For example, as described above, in the example of FIG. 3B (recognizing an image obtained by capturing a blue sky through a curtain), the priority mode is set to the brightness / color tone priority mode. Recognize with priority. As a result, it is recognized whether the image is bright and whether the image is blue. On the other hand, since the priority mode is set to the shape / contour priority mode in the example of FIG. 3F (recognizes the coffee cup image), the shape and contour in the image are recognized with the highest priority. As a result, the shape of the coffee cup in the image is recognized.

  Since the processing from step S61 to S63 has already been described, the details are omitted. Following step S63, the mobile phone 1 determines whether or not to end the text input (step S64). When the result of the determination in step S64 is “false” (NO), the processing in FIG. 8 returns to step S54. As a result, the user can switch the priority mode for image recognition one after another, perform various recognitions, and display various conversion input candidates according to each recognition result. Note that when the result of the determination in step S64 is “true” (YES), the processing shown in FIG. 8 ends.

  The present invention is not limited to the above-described embodiments. Those skilled in the art can make various modifications to the present invention within the scope of the claims. That is, a new embodiment can be obtained by combining appropriately changed technical means within the scope of the claims.

(Scope of application)
The present invention can be applied to all devices capable of inputting text. Therefore, the present invention is not limited to the text input at the time of e-mail creation described above, and can be applied to various text input processes such as at the time of blog article creation and memo creation.

(Image capture and recognition timing)
When the mobile phone 1 acquires an image by the imaging of the main camera 30 and recognizes the image (1) Automatically when the mobile phone 1 is stationary for a predetermined reference time (for example, an extremely short time of about 1 to 2 seconds) Whether to start imaging and image recognition, (2) start imaging and image recognition when the user presses the imaging button, or (3) continuously capture images and acquire images as needed ( Image recognition may be performed at regular intervals. In any case, after the image is recognized, the mobile phone 1 holds the conversion input candidate corresponding to the recognition result in the large-capacity memory 18, so that it can be referred to and displayed on the screen as necessary.

(Display conversion input candidates for breakfast)
The mobile phone 1 can also acquire and display conversion input candidates relating to breakfast. The flow of processing for realizing this display is as follows. The image recognition unit 64 recognizes text (characters, words, etc.) on the surface of the package such as milk, bread, miso soup, or the like from the image to be recognized. The conversion input candidate control unit 66 acquires at least one conversion input candidate related to breakfast from the large-capacity memory 18 based on the recognition result. Conversion input candidates in this case include “Now breakfast” and “Today's breakfast is bread”. The display control unit 62 displays conversion input candidates related to the acquired breakfast in the conversion input candidate display area 86. In the large-capacity memory 18, conversion input candidates related to breakfast are stored in advance in association with the recognition result.

(Display conversion input candidates for work)
The mobile phone 1 can also acquire and display conversion input candidates related to work. The flow of processing for realizing this display is as follows. The image recognition unit 64 recognizes a certain number or more of sentences from the recognition target image. It is assumed that the control unit 8 obtains the current time from a local clock or the like, and that time is a time indicating work (for example, any time from 9:00 to 18:00). The conversion input candidate control unit 66 acquires at least one conversion input candidate related to work from the large-capacity memory 18 based on these recognition results and time. In this case, conversion input candidates include “I'm working hard” and “I'm busy and can't take my hands off”. The display control unit 62 displays the conversion input candidates related to the acquired work in the conversion input candidate display area 86. Note that conversion input candidates related to work are stored in advance in the large-capacity memory 18 in association with the recognition result and the time during the work.

(Multi function)
When the user selects a predetermined multi-function while the image quoting function is activated, the mobile phone 1 displays a screen corresponding to the function on the main display unit 34 in full screen. For example, it is assumed that the user selects a “setting” item in the “top menu” and further selects an “external connection” item that is a sub-item of the setting item. This “external connection” is an example of a multi-function. The main display unit 34 displays a screen for setting “external connection”. At this time, the image recognition unit 64 recognizes text included in the displayed screen, acquires the recognized text as a conversion input candidate, and displays it in the conversion input candidate display area 86. For example, texts such as “Bluetooth (registered trademark)”, “infrared communication”, and “USB connection code” included in the screen are recognized.

(Display of conversion input candidates for vehicles)
The mobile phone 1 may recognize an object from the image, and acquire and display text corresponding to the recognition result as a conversion input candidate. For example, it is assumed that a vehicle (car or train) is recognized from the image. In this case, the conversion input candidate control unit 66 acquires conversion input candidates related to the vehicle. For example, texts such as “going to drive now” and “get on the train now” are acquired from the large-capacity memory 18. The display control unit 62 displays conversion input candidates related to the acquired vehicle in the conversion input candidate display area 86. In the large-capacity memory 18, conversion input candidates related to the vehicle are stored in advance in association with the recognition result.

(Display conversion input candidates related to time)
The mobile phone 1 can also acquire and display conversion input candidates relating to time. The flow of processing for realizing this display is as follows. The image recognition unit 64 recognizes a clock arrangement (numbers 1 to 12 are clockwise arrangements) from the recognition target image. The conversion input candidate control unit 66 acquires at least one conversion input candidate related to time from the large-capacity memory 18 based on the recognition result. In this case, conversion input candidates include “working soon” and “slightly late”. The display control unit 62 displays the conversion input candidates related to the acquired work in the conversion input candidate display area 86. In the large-capacity memory 18, conversion input candidates related to time are stored in advance in association with the recognition result.

(Display conversion input candidates for telephone)
The mobile phone 1 can also acquire and display conversion input candidates related to the phone. The flow of processing for realizing this display is as follows. The image recognition unit 64 recognizes the telephone from the recognition target image. For example, when an array including numbers and * # is detected, it is determined that the telephone is recognized. The conversion input candidate control unit 66 acquires at least one conversion input candidate related to the telephone from the large-capacity memory 18 based on the recognition result. In this case, conversion input candidates include “I want a call” and “Can I call you?”. The display control unit 62 displays conversion input candidates related to the acquired telephone in the conversion input candidate display area 86. In the large-capacity memory 18, conversion input candidates related to the telephone are stored in advance in association with the recognition result.

(Display conversion input candidates for TV)
The mobile phone 1 can also acquire and display conversion input candidates for the television. The flow of processing for realizing this display is as follows. The image recognizing unit 64 recognizes “a state where there are a large number of color tones in a rectangular rectangle” from the image to be recognized. The conversion input candidate control unit 66 acquires at least one conversion input candidate related to the television from the large-capacity memory 18 based on the recognition result. In this case, conversion input candidates include “I'm watching TV” and “I'm relaxing”. The display control unit 62 displays the conversion input candidates related to the acquired television in the conversion input candidate display area 86. In the large-capacity memory 18, conversion input candidates related to the television are stored in advance in association with the recognition result.

(Display of conversion input candidates for going to bed)
The cellular phone 1 can also acquire and display conversion input candidates related to sleeping. The flow of processing for realizing this display is as follows. The image recognition unit 64 recognizes a pillow or a futon from the image to be recognized. For example, when a predetermined contour representing the shape of a pillow or a futon is detected, it is determined that the pillow or the futon has been recognized. The conversion input candidate control unit 66 acquires at least one conversion input candidate related to sleeping from the large-capacity memory 18 based on the recognition result. In this case, conversion input candidates include “sleep now” and “good night”. The display control unit 62 displays the acquired conversion input candidates related to bedtime in the conversion input candidate display area 86. In the large-capacity memory 18, conversion input candidates related to bedtime are stored in advance in association with the recognition result.

  In addition to the recognition result of the pillow or the futon, the conversion input candidate may be acquired in consideration of the measurement result of the brightness around the mobile phone 1. Specifically, the illuminance sensor 56 measures the brightness around the mobile phone 1. When the measured brightness is lower than the reference value, that is, when the surroundings of the mobile phone 1 are dark, the conversion input candidate control unit 66 sets the conversion input candidate corresponding to the recognition result and the measurement result to the large-capacity memory 18. Get from. As a result, the appropriateness of the conversion input candidate selected for the recognition target image can be further enhanced.

(Display conversion input candidates for crowds)
The mobile phone 1 can also acquire and display conversion input candidates related to crowding. The flow of processing for realizing this display is as follows. The image recognition unit 64 recognizes a large number of people from the recognition target image. For example, it is determined that a large number of people have been recognized when a predetermined number of contours representing the shape of a person are detected by a predetermined number. Based on the recognition result, the conversion input candidate control unit 66 acquires at least one conversion input candidate related to sleeping from the large-capacity memory 18. In this case, there are conversion candidates such as “It is very crowded” and “It is a great human wave”. The display control unit 62 displays the conversion input candidates related to the acquired crowd in the conversion input candidate display area 86. In the large-capacity memory 18, conversion input candidates related to crowding are stored in advance in association with the recognition result.

(Display conversion input candidates for work)
The mobile phone 1 can also acquire and display conversion input candidates related to work. The flow of processing for realizing this display is as follows. The image recognition unit 64 recognizes the personal computer from the recognition target image. For example, when a QWERTY keyboard is detected, it is determined that a personal computer has been recognized. Based on the recognition result, the conversion input candidate control unit 66 acquires at least one conversion input candidate related to work from the large-capacity memory 18. Conversion input candidates in this case include “still working…” and “busy ...”. The display control unit 62 displays the conversion input candidates related to the acquired work in the conversion input candidate display area 86. In the large-capacity memory 18, conversion input candidates related to work are stored in advance in association with the recognition result.

(Display conversion input candidates for location)
The mobile phone 1 can also acquire and display conversion input candidates related to the place. Specifically, the GPS unit 44 or the distance sensor 58 acquires information on the current location (position), and displays the acquired information and conversion input candidates corresponding to the image recognition result in the conversion input candidate display area 86. Thereby, for example, a text such as “I'm at XXX” can be displayed as a conversion input candidate.

(Display conversion input candidates corresponding to gestures)
The mobile phone 1 may acquire conversion input candidates corresponding to “human gestures” included in the recognition target image and display them in the conversion input candidate display area 86. For example, when the mobile phone 1 recognizes “the state where the hand is placed on the face” from the recognition target image, the mobile phone 1 acquires texts such as “still sleepy…” and “I finally woke up” as conversion input candidates, It is displayed in the conversion input candidate display area 86. On the other hand, when “tooth” is recognized from the image to be recognized, texts such as “I am fine”, “happy”, and “fun” are acquired as conversion input candidates and displayed in the conversion input candidate display area 86. In addition, when “the index finger is near the eyes” is recognized from the image to be recognized, text such as “sad”, “lonely”, and “spicy” is acquired as conversion input candidates, and the conversion input candidate display area 86. Also, when “open palm” is recognized from the image to be recognized, text such as “wait a minute” and “so then” is acquired as a conversion input candidate and displayed in the conversion input candidate display area 86.

  In the large-capacity memory 18, conversion input candidates related to each gesture are stored in advance in association with the recognition result of each gesture.

(Other sensors)
The mobile phone 1 may acquire conversion input candidates according to outputs from various sensors such as a temperature sensor or a humidity sensor in addition to the image recognition result, and may display the conversion input candidates in the conversion input candidate display area 86. In this case, the user can select text corresponding to the measurement target (temperature, humidity, etc.) of the sensor.

(Save recognition image)
When the mobile phone 1 recognizes an image captured by the main camera 30, acquires a conversion input candidate according to the recognition result, and displays it in the conversion input candidate display area 86, the recognized image is used for input candidate acquisition. These dedicated images may be stored in a dedicated folder in the large-capacity memory 18. At this time, the conversion input candidates corresponding to the image recognition result are stored in association with the image. Accordingly, the user can appropriately edit (change, add, or delete) the conversion input candidates associated with the images stored in the dedicated folder. Furthermore, the user can also register unique conversion input candidates in association with newly stored images.

(Use of image information)
The cellular phone 1 acquires image information (Exif information such as imaging date / time, position, data type, data capacity, file name, imaging device, etc.) included in the recognition target image as a text conversion input candidate, and converts the input. It can also be displayed in the candidate display area 86.

(Program and recording medium)
Finally, each block included in the mobile phone 1 may be configured by hardware logic. Alternatively, it may be realized by software using a CPU (Central Processing Unit) as follows.

  That is, the mobile phone 1 includes a CPU that executes instructions of a program that realizes each function, a ROM (Read Only Memory) that stores the program, a RAM (Random Access Memory) that expands the program into an executable format, and A storage device (recording medium) such as a memory for storing the program and various data is provided. With this configuration, the object of the present invention can be achieved by a predetermined recording medium.

  The recording medium only needs to record the program code (execution format program, intermediate code program, source program) of the program of the mobile phone 1 which is software for realizing the above-described functions so as to be readable by the computer. This recording medium is supplied to the mobile phone 1. Thereby, the cellular phone 1 (or CPU or MPU) as a computer may read and execute the program code recorded on the supplied recording medium.

  The recording medium for supplying the program code to the mobile phone 1 is not limited to a specific structure or type. That is, the recording medium includes, for example, a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. System, a card system such as an IC card (including a memory card) / optical card, or a semiconductor memory system such as a mask ROM / EPROM / EEPROM / flash ROM.

  Further, even if the mobile phone 1 is configured to be connectable to a communication network, the object of the present invention can be achieved. In this case, the program code is supplied to the mobile phone 1 via the communication network. This communication network is not limited to a specific type or form as long as it can supply the program code to the mobile phone 1. For example, it may be the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication network, and the like.

  The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, even in the case of wired lines such as IEEE 1394, USB (Universal Serial Bus), power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA and remote control, Bluetooth (registered trademark), 802.11 It can also be used by radio such as radio, HDR, mobile phone network, satellite line, terrestrial digital network. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention can be widely used as various mobile devices (for example, mobile terminal devices and mobile phones) that can input text.

1 Mobile phone (mobile device)
8 Control unit (time detection means, history search means)
18 Large-capacity memory (storage means)
30 Main camera (camera)
34 Main display (display)
38 State detection unit 44 GPS unit (position information acquisition means)
46 GPS antenna (location information acquisition means)
56 Illuminance sensor (brightness measurement means)
58 Distance sensor (Position information acquisition means)
62 Display control unit (image display means, display control means)
64 Image recognition unit (image recognition means)
66 Conversion input candidate control unit (conversion input candidate acquisition means, conversion input selection means)

Claims (15)

An image display means for displaying an image acquired by the imaging means and an image in the storage means;
Image recognition means for recognizing the image;
Conversion input candidate acquisition means for acquiring conversion input candidates according to the recognition result of the image;
Display control means for simultaneously displaying the image and the conversion input candidate;
A portable device comprising conversion input selection means capable of selecting and inputting from the conversion input candidates.   The display control means may further comprise a display control means capable of switching the input method of the conversion input candidate to an input method based on pressing of the character selection button when a character selection button in the input operation unit is pressed. Item 2. The portable device according to Item 1.   Using the display control means for displaying the conversion input candidates according to the recognition result of the image together with the conversion input candidates at the time of character input based on pressing of the character selection button after the switching process by the display control means, the conversion input The portable device according to claim 2, further comprising conversion input selection means capable of selectively inputting from candidates.   Using the display control means capable of simultaneously displaying the image, the conversion input candidate according to the recognition result of the image, and the conversion input candidate in the input method based on pressing of the character selection button, it is possible to selectively input from the conversion input candidates 2. The portable device according to claim 1, further comprising conversion input selection means.   Using the display control means for displaying the conversion input candidate according to the recognition result of the image and the conversion input candidate at the time of character input based on pressing of the character selection button in different display forms, and using the conversion input 5. The portable device according to claim 3, further comprising conversion input selection means capable of selectively inputting from candidates. It further comprises time acquisition means for acquiring the current time,
The image processing apparatus includes a conversion input selection unit that uses a display control unit that displays the conversion input candidate according to the recognition result of the image and the acquired current time, and that can select and input from the conversion input candidate. The portable apparatus of any one of Claims 1-5. A history determination means for determining whether there is a history of predetermined processing in the portable device;
Conversion that can be selectively input from the conversion input candidates using conversion input candidate acquisition means for acquiring the conversion input candidates according to the recognition result of the image and the presence or absence of the history, and display control means for displaying the conversion input candidates The portable device according to claim 1, further comprising an input selection unit.   8. The portable device according to claim 7, wherein the history is a transmission history of other e-mails in a specific time zone in the past with respect to an e-mail transmission partner currently being created. Further comprising a brightness measuring means for measuring ambient brightness,
The conversion input candidate acquisition unit and the display control unit acquire and display the conversion input candidate according to the recognition result of the image and the measured brightness,
9. The portable device according to claim 1, further comprising conversion input selection means that can select and input from the acquired and displayed conversion input candidates.   The portable device according to claim 1, further comprising a storage unit that stores the captured image and the acquired conversion input candidate in association with each other.   The said conversion input candidate acquisition means acquires the image information contained in the said imaged image as a conversion input candidate according to the said image, The any one of Claims 1-10 characterized by the above-mentioned. Portable device.   The portable device according to claim 1, wherein when the portable device is stationary for a predetermined reference time, imaging of the image and recognition of the image are started.   The portable device according to claim 1, wherein when the imaging button of the portable device is pressed by a user, the imaging of the image and the recognition of the image are started.   The mobile device according to any one of claims 1 to 11, wherein the image is recognized at regular intervals while the image is continuously captured.   The portable device according to claim 1, wherein the display of the conversion input candidate is terminated when the imaging of the image is terminated.

Images