JP2010217962A - Program for camera-equipped mobile terminal device, and camera-equipped mobile terminal device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily call a function using an image generated by a camera. <P>SOLUTION: In an application management system 100 set in a control part of a camera-equipped mobile terminal device, when a preview image processing part 101 supplies a preview image acquired from the camera to a recognition processing part 102, processing of detecting a characteristic set as a processing target by applications 111-114 is executed by respective detection parts 105-108 inside the recognition processing part 102. A GUI control part 103 allocates a key for selection operation to the application processing the characteristic whose detection succeeds, and displays information of the key together with an application name. Furthermore, when operation of the allocated key is performed, the GUI control part 103 decides that the application is selected, communicates an application start part 104, and the application start part 104 starts the selected application. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  According to the present invention, a control unit by a computer is provided in a housing provided with a camera, an operation unit, and a display unit, and a predetermined process involving recognition processing for an image generated by the camera by an application incorporated in the control unit. The present invention relates to a camera-equipped mobile terminal device that executes information processing and a program incorporated in the device.

  In recent years, a digital camera (hereinafter simply referred to as “camera”) is mounted on many portable terminal devices such as mobile phones, and two-dimensional code reading is performed using an image generated by the camera. And a function for executing recognition processing such as character recognition processing.

  For example, many mobile phones are equipped with a function of reading address information (URL) of a web page from a two-dimensional code image and accessing the web page of the address information (see Patent Document 1). Some have a function of recognizing characters from a character string image and outputting them to a document creation system such as a mail editor (see Patent Document 2). An application for converting a face image detected from an image into a portrait image using a face image recognition technique (see Patent Document 3) has also been developed.

JP 2009-33411 A JP 2008-250818 A Patent No. 4218348

  As described above, the functions of the camera-equipped mobile terminal device are diversified, but the user interface for calling these functions is insufficient, and generally the desired function is called unless a plurality of operations are performed. I can't. For this reason, some users complain that they have a function they want to use but they cannot use it because they do not know the procedure for calling that function. There are also many users who are not fully aware of what functions the device they own.

  The present invention pays attention to the above-mentioned problems, and an object thereof is to improve the convenience of a camera-equipped mobile terminal device by making it possible to easily call a function using an image generated by a camera.

  A mobile terminal device with a camera to which the present invention is applied includes a camera, an operation unit and a display unit, and a computer control unit. The control unit detects a feature satisfying a specific condition from an image and detects the feature. A plurality of programs for applications that execute predetermined information processing using the features described above are incorporated. As will be described later, the operation unit may be provided integrally with the display unit as a touch panel, or may be provided separately from the display unit.

  The features detected by the above application include, for example, optical information codes such as two-dimensional codes and bar codes, and character strings. In some cases, a human face, a pre-registered pattern image, or the like is used as a feature to be detected. Each application detects these features, outputs the detection results, and executes an editing process using the detected features.

The program according to the present invention causes the above-described control unit to function as the following feature detection means, GUI (graphical user interface) control means, and application activation means.
The feature detection unit executes, for each feature type, a process of detecting a feature to be processed by each application from the preview image while the camera generates a preview image. The GUI control unit sets a state for accepting a selection operation by the operation unit for all applications corresponding to the feature detected by the feature detection unit, and indicates a function and a selection operation method for each of these applications. Display information on the display. The application activation unit activates the selected application in response to an operation for selecting one of the applications to be displayed by the GUI control unit.

  According to the above program, when the user points the camera at a feature to be processed for the purpose of performing some processing using a recognition technique based on image processing, the above feature is detected from a preview image generated by the camera. Is done. Further, the display unit displays information indicating the function of the application that processes the detected feature and the selection operation method. Here, when the user selects an application according to this display, the selected application is launched by the application activation means. Therefore, even if the user does not know how to call the target function, the user can easily call the target function based on the displayed information.

  Also, in addition to the function that the user is trying to use, even when the features processed by other functions are included in the field of view of the camera, the application functions corresponding to each feature are displayed individually. The target application can be selected without error.

  Further, even when the user holds the mobile terminal device with a camera without being particularly conscious, if the feature to be processed by any application is included in the field of view of the camera, the feature detection means and Processing by the GUI control means is performed, and available functions and selection operation methods are displayed. Therefore, the user can recognize what function is incorporated in the apparatus from the displayed information, and can easily try the function.

  Note that the preview image is generated by driving the camera at regular intervals, and is preferably generated as a low-resolution image using a part of the image sensor of the camera. While the power is being supplied to the apparatus, the preview image may be generated at all times except in the standby state. However, the preview image is generated in response to an operation instructing the use of the camera. May be started.

Hereinafter, five preferred embodiments of the above program will be described.
First, in the first aspect, the GUI control unit assigns all applications corresponding to the features detected by the feature detection unit to specific one keys of the operation unit, and the application names of these applications and the assigned keys. Is displayed on the display unit.

  According to the first aspect, the user can easily grasp the functions that can be used by the displayed application name. In addition, when any function is selected, it is only necessary to operate the key once, and the conventional problem that the operation is complicated can be solved.

  A 2nd aspect presupposes that the operation part is integrally provided in the display part as a touch panel. The GUI control unit of this aspect displays the application names of all the applications corresponding to the features detected by the feature detection unit on the display unit, and corresponds to the respective positions corresponding to the display positions of the application names on the touch panel. Set the status to accept the operation to select the application.

  According to the second aspect, by displaying the application name, the function of the application is displayed, and at the same time, the operation location for selecting the application is shown. Accordingly, the user can easily recognize the functions that can be used, and can easily start up the application by touching the target application name.

  The third mode is also based on the premise that the operation unit is integrally provided on the display unit as a touch panel. The GUI control unit corresponds to the feature detected by the feature detection unit on the display screen of the preview image. A location to be identified is displayed, and an application name corresponding to the feature identified and displayed in the vicinity of each identification display is displayed. In addition, a state of accepting an operation for selecting an application corresponding to each feature that is identified and displayed is set at a location corresponding to each identification display on the touch panel.

  According to the above aspect, the function that can use the feature in the preview image can be shown, and the place where the feature is displayed can be notified as the operation place for selecting the application.

  Next, in the fourth aspect, the GUI control means displays a preview image in which frame areas for arranging features to be processed by the selected application are superimposed in response to an operation for selecting the application. The processing to be displayed on the section is further executed.

  Even if a feature to be processed can be detected from a preview image and a corresponding application can be activated, recognition processing by the application may not be performed correctly depending on the position and size of the feature. In the fourth aspect, paying attention to this point, a preview image in which the frame area is superimposed is displayed, so that the user can adjust the imaging range so that the feature to be processed falls within this frame area. Processing by the activated application can be performed with high accuracy.

  In the fifth aspect, the GUI control unit displays the information to be displayed on the preview image generated by the camera. In addition, it is possible to accept a shooting instruction operation from the operation unit together with an application selection operation. Further, the program of this aspect causes the control unit to function as a shooting control unit that drives the camera in response to the reception of the shooting instruction by the GUI control unit and takes in the generated image.

  According to the above aspect, it is possible to receive a selection of an application and execute a process of starting a program of the selected application in parallel with normal photography. Therefore, even when a feature (for example, a character) to be processed by the application is included in the field of view of the camera at the time of taking a picture, the processing according to the user's intention can be performed without any trouble.

  A portable terminal device with a camera according to the present invention is for an application that detects a feature that satisfies a specific condition from a camera, an operation unit and a display unit, and performs predetermined information processing using the detected feature. And a computer control unit in which a plurality of programs are incorporated. Further, the control unit is provided with functions of a feature detection unit, a GUI control unit, and a program start unit by the above-described program.

  According to the above program and the camera-equipped mobile terminal device, when the user performs information processing using the camera, even if the application name and the calling method are not known, the necessary function is called according to the information displayed on the display unit. be able to. Therefore, the burden on the user can be reduced and the usability of the apparatus can be improved. In addition, since the user can recognize the functions installed in the device used by the user based on the displayed information, the frequency of using the installed functions is increased, and the installation of various functions is useless. Can be prevented.

It is a block diagram which shows the structure of the mobile telephone to which this invention is applied. It is a functional block diagram which shows the structure of the application management system and application group incorporated in said mobile phone. It is a figure which shows the example of a change of a display screen when a portrait creation application is called. It is a figure which shows the example of a change of a display screen when a business card reader is called. It is a figure which shows the example of a change of a display screen when a two-dimensional code reader is called by the printed material containing a two-dimensional code, and imaging | photography aimed at this printed material. It is a flowchart which shows the process sequence by an application management system.

FIG. 1 shows a configuration of a mobile phone to which the present invention is applied.
This mobile phone includes a control unit 1, a camera 2, an image processor 3, a display unit 4, an operation unit 5, a communication circuit 6, and the like, each having a computer as a main component. The camera 2, the display unit 4, and the operation unit 5 are assembled in a casing (not shown), and the control unit 1, the image processor 3, and the communication circuit 6 are mounted on a control board in the casing. Although not shown, a circuit such as an interface is provided between the display unit 4, the operation unit 5, the communication circuit 6, and the control unit 1.

  The control unit 1 includes a CPU 11, a main memory 12, an image memory 13, and the like. The main memory 12 stores programs, setting data, and the like. On the other hand, the image memory 13 stores image data set as a processing target when the control unit performs a recognition process and image data generated by taking a picture accompanying a user's shutter operation.

  The operation unit 5 includes a plurality of keys including a numeric keypad, and the display unit 4 includes a liquid crystal panel and a drive circuit thereof. The communication circuit 6 is used for a call or connection to the Internet.

  The camera 2 is a digital camera 2 having a CCD or a CMOS, and is connected to the control unit 1 via the image processor 3. The image processor 3 normally drives the camera 2 at regular intervals, thins out the pixel unit image data output from the camera 2 based on a predetermined rule, and generates a preview image with low resolution. On the other hand, when an imaging instruction is received from the CPU 11, a high-resolution image is generated using the image data of all pixels.

  The preview image output from the image processor 3 is output to the display unit 4, displayed in real time, and also output to the CPU 11. Similarly, the high-resolution image is output to the CPU 11 and then stored in the image memory 13 by the CPU 11. Further, the CPU 11 fixedly displays the image on the display unit 4 until the next operation of the user is accepted.

  Next, the main memory 12 of the mobile phone of this embodiment has a function (application) for detecting a feature satisfying a specific condition from an image generated by the camera 2 and performing predetermined information processing based on the detection result. A plurality of programs to be given to the CPU 11 are stored.

  Further, the main memory 12 stores a program for supporting an operation when the user selects an application to be activated and performing control for starting the application selected by this operation. Hereinafter, a function set in the CPU 11 by this program is referred to as an “application management system”.

  FIG. 2 shows a functional block diagram for the above application management system (indicated by reference numeral 100 in this figure) and applications managed by the system 100. As shown in this figure, the mobile phone of this embodiment incorporates a two-dimensional code reader 111, a portrait creation application 112, a business card reader 113, and OCR software 114 as applications that use image recognition processing. Yes.

  The two-dimensional code reader 111 detects a QR code (registered trademark) as a two-dimensional code. The two-dimensional code reader 111 detects a two-dimensional code from an image and decodes information encoded in the code. The caricature creation application 112 (shown to the user with the name “caricature app” as shown in FIG. 3) detects a human face image from the image, and based on the contour line, the caricature image is displayed. Generate.

  The OCR software 114 detects a character string from the image, reads the constituent characters of the detected character string, and converts them into text data. The OCR software 114 operates independently and outputs the reading result to other applications (not shown) (e.g., mail editor, translation software). In addition, when the business card reader 113 is activated, the OCR software 114 responds to a request from the business card reader 113. The corresponding reading process is executed.

  The business card reader 113 detects the outline shape of the business card from the image and requests the OCR software 114 to perform character recognition processing in the area surrounded by the outline. In response to this request, the OCR software 114 recognizes characters in the area and converts them into text data for each character string. The business card reader 113 analyzes each text data and sets attributes such as name and company. A process of storing in a business card database (not shown) is executed.

  For the above application group, the application management system 100 of this embodiment includes a preview image input unit 101, a recognition processing unit 102, a GUI control unit 103, an application activation unit 104, and the like.

  The preview image processing unit 101 inputs the preview image generated by the image processor 3. Each time an image is input, the amount of change of the new input image relative to the previous input image is detected, and the image when the amount of change is equal to or less than a predetermined allowable value is provided to the recognition processing unit 102.

  The recognition processing unit 102 is for recognizing whether the preview image provided from the preview image processing unit 101 includes a feature to be processed by each application. Specifically, a dedicated detection unit is provided for each feature type, such as a two-dimensional code detection unit 105, a face detection unit 106, a business card detection unit 107, and a character string detection unit 108.

  The two-dimensional code detection unit 105 detects three finder patterns unique to the QR code (registered trademark). Further, when this detection is successful, an area in which the two-dimensional code is included is specified based on each finder pattern.

  The face detection unit 106 detects an area in which a predetermined number or more of pixels having a color close to the skin color are distributed from the image (hereinafter referred to as “skin color area”), and further detects a feature point in the skin color area. . Then, the distribution of each detected feature point is collated with a rule relating to the arrangement of facial organs such as eyes, nose, and mouth, and when each feature point conforms to this rule, the skin color area is specified as the face area.

  The character string detection unit 108 captures the result of the edge extraction processing performed on the preview image by the edge extraction unit (not shown) set in the recognition processing unit 102, and extracts the linear component in the image. Identify. Furthermore, a connected body of linear components is detected as one character, and each character is grouped based on the distance between characters. Note that the detection at this stage is intended to determine the presence / absence of characters, so that it is not necessary to perform character recognition processing like the OCR software 114.

  The business card detection unit 107 also captures the processing result by the edge extraction unit and detects an edge indicating a straight line component by Hough transform or the like. Further, a portion where a rectangular area is formed by four linear components is detected, and if this detection is successful, the ratio of the vertical and horizontal directions of the rectangular area is calculated.

When the above ratio corresponds to the business card standard registered in advance, the detection result of the character string detection unit 108 for the rectangular area is referred to and a predetermined number or more of characters are detected in the rectangular area. Detects the area as a business card. On the other hand, if the aspect ratio of the rectangular area does not correspond to the business card standard, or if the number of detected characters is less than a predetermined number, it is determined that the business card feature is not included in the preview image.
When the business card detection unit 107 detects a business card, it notifies the character string detection unit 108 to that effect. In response to this communication, the character string detection unit 108 discards the detection result corresponding to the area detected as the business card.

  As described above, the recognition processing unit 102 detects various features from the preview image by the detection units 105 to 108 set for each feature type, and supplies the detection result to the GUI control unit 103. The GUI control unit 103 identifies an application that can process the detected feature based on the supplied information, and assigns a selection key to each identified application. Further, as shown in a specific example to be described later, information indicating the application name of the identified application and the assigned key is displayed on the display unit 4, and the user's key operation is accepted.

  When one of the keys assigned here is operated, the GUI control unit 103 determines that the application corresponding to this key has been selected, and passes the determination result to the application activation unit 104. In response to this, the application activation unit 104 activates the selected application.

  Thereafter, the process described above is executed by the activated application. In this embodiment, the GUI control unit 103 also sets the feature to be processed to a position suitable for application recognition at this stage. Guidance display for arrangement (details will be described with reference to specific examples in FIGS. 3 to 5) is performed.

  Note that the image processor 3 of this embodiment always drives the camera 2 and waits for a photographing instruction from the CPU 11 except for the standby mode while the power is supplied. The application management system 100 executes the above processing in parallel with the processing by the image processor 3. In addition to accepting an application selection operation, the GUI control unit 103 accepts a shutter operation. When the shutter operation is accepted, the GUI control unit 103 communicates with a system (not shown) that controls photography. By such control, even when the user takes a picture, it is possible to cope without trouble.

  3 to 5 show an example in which a predetermined feature is detected from the preview image by the processing of the application management system 100, and the display on the display unit 4 changes according to the detection result and the user's operation. In any of the examples, a preview image display area 20 is set in a large part of the screen, and below that, function keys of the operation unit 5 (not shown, but five function keys are provided in this embodiment). ) Is provided with a display field 21 indicating the functions assigned to it. In the display of the display field 21, “shooting” in the center indicates the position of the function key for the shutter operation.

Each example will be specifically described below.
First, in the example of FIG. 3, shooting for a person is performed. A screen G31 in FIG. 3 shows a preview image when the alignment of the camera 2 with the subject is stable and the amount of change with respect to the preview image of the previous stage is below the reference value. At this time, processing by each of the detection units 105 to 108 of the recognition processing unit 102 is executed, and the display changes like a screen G32 based on the processing result.

  In the case of the example of FIG. 3, the face area is detected by the face detection unit 106. Accordingly, a rectangular frame 201 indicating the detected face area is displayed on the screen G32 in accordance with the preview image. The Further, on this screen G32, a name 202 of an application for processing the detected feature (face image) (caricature application) and a message 202 for informing a key assigned to this application are displayed.

In this embodiment, each numeric key of the numeric keypad is assigned to the application specified by the detected feature one by one. The caricature creation application identified in the example of FIG. 3 is assigned a key “1”.
Here, when the user operates the assigned key, the corresponding application (caricature creation application) is activated, and the display changes as in a screen G33. In this screen G33, a rectangular frame 203 having a predetermined size is displayed in a superimposed manner on the preview image. Note that “superimpose display” refers to a process of always displaying the rectangular frame 203 at the same place even when the preview image is updated.

  Further, on the screen G33, a message 204 instructing to put the face to be processed into the rectangular frame 203 is displayed. When the user sets a state in which the face of the subject enters the rectangular frame 203 according to this instruction and performs a shutter operation, a high-resolution image is generated and stored in the image memory 13 in accordance with the operation. In response to this, the display on the display unit 4 also changes to a screen G34 showing the stored image. Further, although not shown, when the portrait creation process is finished, a screen showing the created portrait is displayed.

  On the other hand, when the user intends to take a picture of the subject person, the shutter operation is performed when the screen G32 is displayed. In response to this operation, a high-resolution image is generated, and a screen G35 showing the high-resolution image is displayed.

  In this screen G35, the display of the function key display field 21 is also changed to indicate processing for the generated image. “Save” in the display column 21 corresponds to a process of saving the image in the image memory 13 in the image database in the main memory 12. “Reshoot” corresponds to a process of deleting an image in the image memory 13 without saving it in the image database.

  As described above, even when an application that can process a preview image is presented, if the user ignores this display and performs a shutter operation, the application does not start and normal photography is executed. Is done. The following examples of FIGS. 4 and 5 do not show the process associated with the shutter operation. However, in these examples as well, the same process as in the example of FIG. 3 is executed when the shutter operation is performed.

  Next, in the example of FIG. 4, photographing for a business card is performed. A screen G41 in the figure is a display when the amount of change with respect to the preview image of the previous stage is below the reference value. In this example, the processing by each of the detection units 105 to 108 of the recognition processing unit 102 is performed, and it is assumed that the business card detection unit 107 has successfully detected the business card, whereby the display of the display unit 4 changes to the screen G42. To do. On this screen G42, a rectangular frame 204 is displayed in the area recognized as a business card, and below the name of the application (business card reader) that processes the features in this frame and the key assigned to this application (this In this embodiment, the key 205 is “1”.) A message 205 is displayed.

  Here, when the user operates the notified key, the business card reader 113 starts up, and the display changes to the screen G43 accordingly. In this screen G43 as well as the screen G33 in FIG. 3, a rectangular frame 206 having a predetermined size is displayed superimposed on the preview image. In addition, a message 207 instructing to put the feature (business card) to be processed into the frame 206 is displayed.

  The last screen G44 is when the user adjusts the position of the camera 2 in accordance with the message 207. At this stage, the business card reader 113 contacts the OCR software 114 to process and recognize the character string in the business card. When this recognition process ends, the display is changed to a screen (not shown) showing the recognition result.

  Next, in the example of FIG. 5, a part of a printed material 300 (for example, a leaflet) including a two-dimensional code is shown in the upper (1) (FIG. 5 (1)), and the printed material 300 is photographed in the lower (2). An example of a change in the screen of the display unit 4 when it is performed will be shown.

  In this embodiment, the camera 2 is aligned for the purpose of reading the two-dimensional code 301 of the printed material 300. A screen G51 in the figure is a preview image when the alignment is completed, and this image includes a target two-dimensional code 301 and a character string printed in the vicinity thereof. Therefore, in this case, the two-dimensional code detection unit 105 detects the two-dimensional code and the character string detection unit 108 detects the character string.

  With these detections, the display on the display unit 4 changes to the screen G52. In this screen G52, the names of the application (two-dimensional code reader 111) that processes a two-dimensional code and the application (OCR software 114) that processes a character string among the two types of detected features, and these applications A message 211 for informing the keys (“1” and “2”) assigned to is displayed.

  Also in this screen G52, as in the previous embodiment, rectangular frames 208, 209, and 210 are displayed for each range in which each feature is detected, but the place where the message 211 is superimposed is not displayed. Set to state.

  When the user operates the “1” key assigned to the two-dimensional code reader 111 under the above display state, the display changes to the screen G53. In this screen G53 as well as the screen G33 in FIG. 3 and the screen G43 in FIG. 4, a rectangular frame 212 of a predetermined size is displayed and an instruction to put the processing target two-dimensional code into the rectangular frame 212 is given. A message 213 is displayed.

  The two-dimensional code reader 111 continues to detect the finder pattern until three finder patterns can be detected within the rectangular frame 212. If the detection is successful, the image processor 3 outputs an imaging instruction to generate a high-resolution image. A two-dimensional code reading process is executed using this image. The last screen G54 corresponds to a display in a state where the reading process is being performed. When this process ends, a screen (not shown) indicating the read information is displayed.

  As shown in each of the above examples, according to the application management system 100 of this embodiment, by pointing the camera 2 to the target feature of the user, an application that can process the feature is specified, and the application name and the Since the keys assigned to the application are displayed, the user can select the application by operating the displayed keys. In any example, since one of the numeric keys of the operation unit 5 is assigned to each application, the user can activate the application by operating the corresponding key once. Therefore, there is no possibility that the operation for calling the application becomes complicated.

  Further, when the application is started, the rectangular frames 203, 206, and 212 for aligning the processing target features are displayed on the preview image so that the started application obtains an image suitable for recognition. Therefore, it is possible to execute highly accurate processing. Therefore, it is possible to prevent the processing on the application side from being performed while the user does not properly align the camera 2, and to ensure the accuracy of the processing.

  FIG. 6 shows a schematic procedure of processing executed by the application management system 100. Hereinafter, along with this flowchart, the processes described so far will be organized and described.

  The processing according to this flowchart is started not only when the power is turned on but also when processing by any application or photography is completed. In this process, the detection result of each feature is managed by setting a detection flag for each type of feature detected by each of the detection units 105 to 108 of the recognition processing unit 102.

  First, after inputting the first preview image in the first step (ST1), all detection flags are reset and set to OFF (ST2).

  Thereafter, the preview image output from the image processor 3 is input every time, and the amount of change with respect to the previous image is calculated (ST4). In this embodiment, on the basis of the “gradient method”, a vector indicating the magnitude and direction of a change occurring between images is obtained, and the length of this vector is calculated.

  Next, the amount of change calculated by the above processing is checked against a reference value. If the amount of change exceeds the reference value (ST5 is “NO”), the process returns to ST3 and the next preview image is input.

  Thereafter, similar processing continues, and when the amount of change becomes equal to or less than the reference value at a predetermined time (ST5 is “YES”), various features are detected by the detection units 105 to 108. If the detection is successful, the corresponding feature detection flag is set to ON (ST6).

  When all the detection processes are completed, each detection flag is checked (ST7). Here, when there is no detection flag set to ON, ST7 becomes "NO" and returns to ST3.

  On the other hand, if at least one detection flag is on, ST7 becomes “YES” and the process proceeds to ST8, and an application whose processing target is a feature corresponding to the detection flag in the on state is specified. Further, the numeric keys of the operation unit 5 are sequentially assigned to the identified applications such as “1”, “2”, “3”,... (ST9).

  Thereafter, a rectangular frame is displayed in a range corresponding to the detected feature in the preview image (ST10), and in ST11, the name of the identified application and the assigned key type are placed at a predetermined position of the preview image. Is displayed (corresponding to the display of screens G32, G42, and G52). When one of the keys assigned in this state is operated (ST12 is “YES”), the application corresponding to the key is activated (ST13). Further, in ST14, a rectangular frame including the feature to be processed is displayed (corresponding to display of screens G34, G44, G54).

  On the other hand, when the shutter operation is performed after the processing of ST11 is completed (“NO” in ST12 and “YES” in ST15), the system that manages the photography is notified that the shutter operation has been performed. The process ends (ST16).

  In the above-described embodiment, an application for processing the detected feature and a display for notifying the assigned key are displayed as message information. However, the display is not limited to this, and a menu-type display may be performed. In addition, when the operation unit 5 is provided integrally with the display unit 4 as a touch panel, a menu with an application name corresponding to the detected feature is displayed, and according to an operation of touching the display location of any application name, A corresponding application may be activated.

  Further, when the touch panel type operation unit 5 is provided, a function as a key for selecting an application can be set at a position corresponding to a frame region surrounding the detected feature. For example, in the example of FIG. 4, the rectangular frame 204 is displayed in response to the detection of the business card, and “business card reader” is displayed in the vicinity of the rectangular frame 204, and a pressing operation into the rectangular frame 204 is performed. In this case, the business card reader 113 can be activated. In the example of FIG. 5, “code reader” is displayed in the vicinity of the rectangular frame 209 corresponding to the two-dimensional code, and “OCR” is displayed in the vicinity of the rectangular frames 208 and 210 corresponding to the detected positions of the characters. The corresponding application can be activated in response to the pressing operation into any of the frames. In this case, only the rectangular frame is displayed at first, and the application name corresponding to the feature in the rectangular frame is displayed in response to the operation in the rectangular frame being performed once, and the second operation is performed. The application may be activated when it is released.

  In the above-described embodiment, only applications that directly execute processing for detecting features from the preview image are selected. However, applications (for example, OCR software) that receive processing results from these applications and execute processing are selected. 114 may be selected including translation software, e-mail editor, etc. that receive text data from 114.

DESCRIPTION OF SYMBOLS 1 Control part 2 Camera 3 Image processor 4 Display part 5 Operation part 11 CPU
DESCRIPTION OF SYMBOLS 12 Main memory 100 Application management system 102 Recognition processing part 103 GUI control part 104 Application starting part 111,112,113,114 Application

Claims (7)

A camera, an operation unit and a display unit, and a computer-based control unit in which a plurality of programs for an application for detecting a feature satisfying a specific condition from an image and executing predetermined information processing using the detected feature are incorporated A program incorporated in the control unit of the camera-equipped mobile terminal device comprising:
Feature detection means for executing, for each type of feature, processing for detecting a feature to be processed of each application from the preview image while the camera is generating a preview image;
For all applications for which the feature detected by the feature detection means is to be processed, a state for accepting a selection operation by the operation unit is set, and information indicating the function and the method of the selection operation is set for each application. GUI control means for displaying on the display unit,
An application activation unit that activates the selected application in response to an operation of selecting one of the applications to be displayed by the GUI control unit;
A program for a camera-equipped mobile terminal device configured to cause the control unit to function as: The GUI control means assigns all applications corresponding to the features detected by the feature detection means to a specific one key of the operation unit, and information indicating the application names and assigned keys of these applications. Display on the display unit,
A program for a mobile terminal device with a camera according to claim 1. The program according to claim 1,
Assuming that the operation unit is incorporated in a mobile terminal device with a camera that is integrally provided in the display unit as a touch panel,
The GUI control means displays application names of all applications corresponding to the features detected by the feature detection means on the display unit, and corresponds to the display positions of the application names on the touch panel, respectively. A program for a camera-equipped mobile terminal device that sets a state of accepting an operation of selecting an application to be performed. The program according to claim 1,
Assuming that the operation unit is incorporated in a mobile terminal device with a camera that is integrally provided in the display unit as a touch panel,
The GUI control means identifies and displays a location corresponding to the feature detected by the feature detection means on the display screen of the preview image, and an application name corresponding to the feature identified and displayed at a position near each identification display And, further, in a location corresponding to each identification display of the touch panel, a state for accepting an operation for selecting an application corresponding to each identified and displayed feature is set.
A program for a mobile terminal device with a camera. In the program according to any one of claims 1 to 4,
The GUI control unit is configured to display, on the display unit, a preview image in which frame regions for arranging features to be processed by the selected application are overlaid in response to an operation for selecting the application. Perform further,
A program for a mobile terminal device with a camera. In the program according to any one of claims 1 to 5,
The GUI control unit is configured to display information to be displayed so as to be superimposed on a preview image generated by the camera and to accept a shooting instruction operation from the operation unit.
A program for a camera-equipped mobile terminal device that causes the control unit to further function as a photographing control unit that drives the camera in response to the photographing instruction operation received by the GUI control unit and captures a generated image. A camera, an operation unit, a display unit, and a control unit by a computer in which a plurality of programs for an application for detecting a feature satisfying a specific condition from an image and executing predetermined information processing using the detected feature In a mobile terminal device with a camera comprising:
Feature detection means for executing, for each type of feature, processing for detecting a feature to be processed by each application from the preview image while the camera generates a preview image;
For all applications corresponding to the features detected by the feature detection means, a state for accepting a selection operation by the operation unit is set, and information indicating the function and the method of the selection operation is set for each application. GUI control means for displaying on the display unit,
And a program activation means for activating the selected application in response to an operation of selecting one of the applications to be displayed by the GUI control means being provided in the control unit. A mobile terminal device with a camera.

Images