JP2005318091A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor with high operability which is used for a portable telephone with a camera or the like, and by which a user selects various processing methods only by moving a telephone without executing complicated key operation. <P>SOLUTION: The moving direction of the portable telephone is detected by detecting the moving direction of an image acquired from a camera (S303) in the portable telephone with a camera having a function of the image processor. When it is detected that the portable telephone is moved to the left (Yes at S304), the processing "to store an image in a memory housed in the portable telephone", which is the processing made to correspond to its moving direction (the left), is selected and executed (S305). When it is detected that the portable telephone is moved to the right (Yes at S306), the processing "to store an image in a memory outside the portable telephone", which is the processing made to correspond to its moving direction (the right), is selected and executed (S307). <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus that captures an image and selects a process.

  Recently, performance improvements such as storage capacity and display size of mobile devices having an imaging function such as digital electronic cameras and camera-equipped mobile phones have been remarkable. Due to this performance improvement, various applications such as transferring an image captured by a camera by a communication means or using it as a character input means have been spreading.

  As described in “Prior Art” of Patent Document 1, a conventional portable terminal is, for example, out of data registered by calling a telephone number entered by pressing a number key or entering characters in a telephone book in advance. , One of them can be selected and transmitted by key operation or voice input.

  A mobile phone SH505i manufactured by Sharp Corporation can store a captured image by selecting an internal memory or an extended memory, for example, a miniSD® card, which has been previously selected by a key operation. The captured image can be sent by inputting a mail address or selecting a mail address registered in the address book and attaching it to an e-mail. Further, as disclosed in Patent Document 1, characters such as a telephone number, an e-mail address, and a name can be recognized from a captured image and input to the apparatus as a character string.

  However, in the input processing to the mobile phone device described above, there is a problem in operability because it is necessary for the user to operate with an operation button at the time of starting an application such as telephone call, Internet browser, mail transmission, address book, etc. It can be said that there is.

  Japanese Patent Laid-Open No. 2004-26883 calculates the amount of movement between frames from the correlation of the data of the captured image, and determines the amount of movement and moving speed of the captured image. The detection method is a technique that can be used in the present application. The movement amount and movement speed thus made are not linked to subsequent processing.

In Patent Document 3, a digital electronic camera that automatically selects a function mode depending on a direction in which a digital electronic camera unit mounted on a mobile phone or PHS is currently directed (inside or outside or the direction of a camera attached to the device) Has been proposed. This selection of the function mode is a change in the operation mode and is different from the selection of the application in the present invention.
JP 2003-163736 A Japanese Patent Laid-Open No. 9-23419 Japanese Patent Laid-Open No. 11-112860

  In the conventional image processing apparatus as disclosed in Patent Document 1, when the same information is input to the portable device and different processing is performed, the user needs to explicitly instruct by a key operation such as selecting from a menu. There is. That is, in the conventional technique, photographing and function selection cannot be performed by a series of operations, and the work is interrupted when a user gives an instruction. Furthermore, there is a problem that work efficiency is greatly reduced due to the necessity of time for giving an instruction and interruption of thought to perform an operation for giving an instruction.

  As an example of the above instruction, in the case of a conventional apparatus, when there are a plurality of locations for storing captured images, such as an internal memory and a memory card, it is necessary to set storage in advance or specify by key operation after shooting There is something to do.

  Further, the method of Patent Document 3 has a problem that only one function can be selected depending on the direction of the digital camera unit, and when a specific selection is made from a plurality of processes, an instruction with a key or the like is required separately. . For example, when the camera part is pointed at the user and the user is photographed, the screen can be displayed without flipping the image to be photographed. The operation for the instruction by etc. is necessary.

  Further, the operation for the instruction is performed by selecting a key indicating processing from a plurality of keys and pressing it, or selecting from the menu with an up / down key or a cross key. Since the operation for instructing these intended processes is performed using a small key particularly in a small portable device, there is a problem that an operation error is often caused. This is a cause of inefficiency in the image processing work because it requires a careful operation by a person who is not familiar with the operation.

  The present invention has been made to solve these problems, and provides an image processing apparatus with high operability that can select various processing methods only by moving them without performing complicated key operations. With the goal.

  In order to achieve the above object, according to one aspect of the present invention, an image processing apparatus moves imaging data determined by a captured image determining means such as a shutter button from an image obtained from an imaging unit such as a digital electronic camera. The process determined by the process determination unit is executed based on the movement amount and direction of the image detected by the detection unit. As the movement detection direction means, for example, the movement direction is detected from the input image of the imaging means by a method as disclosed in Patent Document 2. Based on the detection result, the image processing selection means selects and confirms the processing content for the image performed by the image processing means. The method disclosed in Patent Document 2 can be used as a movement detection unit because it can detect a movement direction that is the direction of movement and a movement amount that indicates how much the movement has occurred.

  Further, the image processing apparatus according to the aspect of the present invention defines the captured image processing means in each direction of the display screen in advance, and moves the display indicating the captured image determined on the display screen, It is possible to clearly indicate to the user which process is being selected. Similarly, a display indicating the captured image confirmed on the display screen, an icon or a character indicating processing, and a display indicating the image are moved according to the result of the movement detection unit, so that any of the confirmed images is displayed. Specify to the user whether they are going to select the process. Examples of the captured image processing means include applications such as WWW (World Wide Web) access, e-mail transmission, and address book, in addition to an instruction for a storage location of a captured image in an image storage unit in RAM or an external storage device. .

  The image processing apparatus according to an aspect of the present invention can select a character string candidate converted into a character code by the character recognition unit based on the result of the movement detection unit when the input data is a character string. Is possible. In addition, processing such as address book storage, URL access, transition to a mail transmission screen, and outgoing call can be easily selected by the process determination means for character string candidates. In addition, character string shaping is performed to change the character string candidate in a form suitable for the processing content, and only the character string necessary for the selected application is input. Here, the character string shaping means processing such as extracting only necessary character recognition result candidate strings or deleting unnecessary character strings by dictionary matching.

  Further, the image processing apparatus according to the aspect of the present invention is such that when the image indicating the captured image is moved on the display screen and the image processing apparatus is continuously moved in the same direction, the image indicating the captured image is displayed. Be sure to display on the display screen at a rate above a certain value. For the user, it is possible to clearly indicate which operation is being selected.

  Further, the image processing apparatus according to the aspect of the present invention tries to detect the movement again from the screen determined to have exceeded when the image detection fails or when the detection amount greatly exceeds the specified value. As a result, even if the input is different from the captured image, the operation can be performed.

  That is, an image processing apparatus according to an aspect of the present invention performs a plurality of processes on one captured image in a series of operations by continuously operating a movement detection unit and a process selection unit. It becomes possible to select.

  In addition, the image processing apparatus according to the aspect of the present invention determines the movement of the apparatus from the movement of the image, and performs character recognition from the image and inputs the character string candidate selection operation converted into the character code. Character recognition result candidates can be easily changed.

  According to the present invention, it is possible to select processing for an image captured by moving the apparatus without performing a key operation from the operation of capturing an image to the image processing apparatus. This means that the user can easily select a process such as storing in a predetermined place and an application to be executed by an input operation and a series of operations to the image processing apparatus. Since there is no need to perform key operations such as number buttons, it is possible to process efficiently without interrupting operations to continuously shoot and perform different processes.

  Further, according to the present invention, it is possible to determine the moving direction of the image processing apparatus without adding a special device such as a speed sensor, so that the image processing apparatus can be simplified and the operation performed by the user can be performed. Both simplification and compatibility are possible.

  Furthermore, by providing a function that allows the user to indicate when to select processing, it is possible to prevent erroneous processing when the user moves in an unintended direction, making it more efficient. Processing becomes possible.

  Similarly, for example, it is possible to select an application activation condition such as activating an application by designating a specific party from a plurality of mail transmission partners.

  Further, according to the present invention, an image indicating processing is displayed using a display device, and a user selects which function by moving a captured image or a display indicating a captured image in accordance with the moving direction or moving amount of the device. You can clearly indicate what you are trying to do. Also, by displaying the function in an easy-to-understand figure such as an icon, and when the user has moved the display showing the captured image to the position of the icon, etc., by performing reverse display of the icon, color conversion, etc. Can clearly communicate to users. As a result, user's erroneous operation can be prevented in advance.

  Similarly, in an image processing apparatus that inputs a character string extracted from an image, a telephone call using the image processing character string as in the image, Internet access via a browser, display of a mail transmission screen, storage in an address book, etc. The process can be easily selected.

  In addition, by extracting the character string part required by the application to be executed from the recognition result candidate character string and performing character string shaping processing that deletes unnecessary recognition result candidate characters, etc., without correcting the recognition result Processing such as telephone calls and e-mails can be performed. As a typical example, when an e-mail address using a telephone number is used as an input, it is easy to properly use telephone and e-mail processing.

  If there is no image showing the captured image on the display device, the user cannot determine which process is being selected, and cannot move to a position indicating the target process. In other words, since an image indicating a captured image is always displayed at a certain ratio or more, it is clearly indicated for the user which operation is to be selected.

  In addition, when the movement direction and movement amount exceed the allowable amount, or when the detection fails, the movement on the screen is not performed, and the image detection is performed again from there, thereby canceling the extreme operation. Furthermore, by performing movement detection from there, it is possible to eliminate the burden of maintaining the system for photographing by the user. Further, the same operation can be performed even after the photographing object has moved.

  Further, in the present image processing apparatus, the processing content for the captured image is not limited to one, but a plurality of processes can be selected by a series of continuous operations. Compared with the above, the processing efficiency is significantly improved.

  Even when performing character recognition, it is possible to select character candidates without performing key operations such as number buttons. It becomes possible to process.

[First Embodiment]
FIG. 1 is a diagram showing a specific example of the configuration of an image processing apparatus according to an embodiment of the present invention. Examples of the image processing apparatus according to the present embodiment include an image of a mobile phone with a camera or a personal computer with a camera. An apparatus having an acquisition function and an image processing function for processing the acquired image corresponds to the imaging unit 101, the display unit 102, the operation unit 103, a RAM (Random Access Memory) 105, a ROM (Read Only Memory) 107, and an external device. The storage device 106 and a central control unit 104 that controls these are included.

  The imaging unit 101 obtains digital image data by engineeringly imaging a subject with a digital electronic camera or the like. The display unit 102 displays an image created by an image display program using a liquid crystal panel or the like. The operation unit 103 has operation buttons such as a shutter button for a captured image determination unit. The external storage device 106 includes an external storage medium such as a memory card.

  The RAM 105 includes an image buffer for temporarily storing captured images, an image storage unit for storing image data, and a work buffer for the central control unit 104 to execute a program. The ROM 107 includes an area for storing a movement detection program, a process determination program, an image storage program, an image display program, and other programs.

  The movement detection program is a program used by movement detection means for detecting movement of an image input to the image processing apparatus such as left and right, up and down, and rotation. The processing determination program is a program for determining processing to be performed on an image used by the image processing selection unit, and the image display program is a program for generating an image to be displayed on the display unit 102. The other program is a program for executing processing other than the above, and includes a program used by the captured image processing means.

  The central control unit 104 executes each program stored in the ROM 107 and controls the operation of each unit.

  FIG. 2 is a diagram for explaining the definition of the moving direction of the image processing detected by the image processing apparatus according to the present embodiment.

  With reference to FIG. 2, in the image processing apparatus according to the present embodiment, the moving direction of the image processing apparatus detected when the image captured from the imaging unit 101 moves to the left is defined as the left, The moving direction of the image processing apparatus detected when the image moves up is defined as up. Similarly, the moving direction of the image processing apparatus including the up / down / left / right direction and the combination thereof is defined. In addition, the moving direction of the image processing apparatus detected when the captured image becomes large by moving the image processing apparatus back and forth, the moving direction of the image processing apparatus detected when the captured image becomes small, Further, the moving direction of the image processing device detected when the image is rotated counterclockwise is defined as the left rotating direction, and the moving direction of the image processing device detected when the image is rotated clockwise is defined as the right rotating direction. To do.

  FIG. 3 is a flowchart showing image processing in the image processing apparatus according to the present embodiment, and will be described below with reference to this figure. The processing shown in the flowchart of FIG. 3 is realized by the central control unit 104 reading and executing each program stored in the ROM 107 and controlling the operation of each unit shown in FIG.

  In the image processing apparatus according to the present invention, a process defined for each moving direction of the image processing apparatus is selected and executed. In this embodiment, as shown in FIG. 4, the captured image moves to the left. Is an example of an image processing apparatus that performs a process of storing an image in an image storage unit included in the RAM 105 inside the device, and that stores an image in the external storage device 106 when moved to the right. Note that the processing listed here is only an example, and another specific example of processing defined by direction is to send an email with an image attached to different parties when moving to the right and when moving to the left. It is also possible to do.

  Referring to FIG. 3, first, in step S <b> 201, a user operates the shutter button or the like of operation unit 103 to determine a captured image, and digital image data determined in step S <b> 202 is stored in an image buffer in RAM 105. Is done.

  Next, in step S203, the central control unit 104 executes a movement detection program as a movement direction detection unit, and the direction in which the image processing apparatus has moved from the image captured by the imaging unit 101 is determined. Detection is performed by obtaining the movement amount of the image processing apparatus. The image change of the image captured by the imaging unit 101, which is the amount of movement of the image processing apparatus, is obtained from the difference between the image sent at a certain time and the image sent before that time.

  In the present embodiment, the central control unit 104 includes all the pixels included in the image captured at time t as illustrated in FIG. 5, all the pixels included in the image captured at time t−1, All pixels that can be calculated by moving all the pixels included in the image captured at a certain time t + 1 in units of one pixel within a range of ± 25% of the imaging width in the x-axis direction and the y-axis direction. Find the difference in values. The amount of movement between the x-axis and the y-axis that minimizes the average difference in pixel values is obtained as the amount of movement of the image processing apparatus.

  The moving direction of the image processing apparatus that needs to be detected in the present embodiment is only the left and right (x-axis direction on the captured image) direction, but it is difficult for the user to move the image strictly left and right. In order to absorb direction blur, it is preferable to detect the moving direction by calculating the moving amount in the y-axis direction as shown in FIG.

  If it is detected in step S204 that the image processing apparatus has moved to the left (Yes in S204), in step S205, the central control unit 104 performs processing for storing the image in the image buffer in the image storage unit. If it is detected in step S206 that it has moved to the right (Yes in S206), a process of storing in the external storage device 106 is performed in step S207.

  If no movement in any direction is detected (No in S204 and No in S206), the process returns to step S203, and the same processing is repeated until movement of the image processing apparatus is detected. The processing in step S204 and the processing in S206 are the image processing selection step in the image processing selection unit, and step S205 or S207 is the captured image processing step in the captured image processing unit for the captured image.

  After the image storage process is performed in step S205 or S207, in step S208, the central control unit 104 activates the image display program and displays that the predetermined process has been completed.

  In the present embodiment, the horizontal movement is detected in the image processing. However, the vertical movement can be detected by switching the x-axis and the y-axis, and the automatic focusing device (autofocus) of the imaging apparatus is also possible. ) Can be used to detect movement in the front-rear direction. In addition, it is possible to detect rotation about the imaging direction by calculating the correlation between the captured image and the image subjected to affine transformation. Even using these movement amounts such as up / down, left / right, and front / back, it is possible to select a process defined for each direction.

  As described above, the image processing apparatus according to the present embodiment uses the input from the imaging unit 101 that is the imaging apparatus to detect the movement direction by the central control unit 104 that executes the movement detection program that is the movement detection apparatus, The storage location of the image can be selected and confirmed according to the detected moving direction. That is, it is possible to select and confirm the processing for the captured image defined for each moving direction.

[Second Embodiment]
FIG. 6 is a flowchart showing image processing in the image processing apparatus according to the present embodiment, and will be described below with reference to this figure. The processing shown in the flowchart of FIG. 6 is also realized by the central control unit 104 reading and executing each program stored in the ROM 107 and controlling the operation of each unit shown in FIG.

  Referring to FIG. 6, the captured image is determined in step S301 as in the first embodiment, and stored in the image buffer in step S302. In step S303, the movement detection program detects the moving direction of the image processing apparatus.

  If it is determined in step S304 that the image processing apparatus has moved to the left (Yes in S304), in step S305, the central control unit 104 performs processing for storing the image buffer image in the image storage unit inside the device. If it is determined in step S306 that it has moved to the right (Yes in S306), a process of storing in the external storage device 106 is performed in step S307.

  After the process of step S305 and step S307, if the key indicating the process end is pressed in step S308 (Yes in S308), the central control unit 104 determines that the process is completed, and the predetermined process is completed in step S309. Display. If not pressed (No in S308), the process returns to step 303 to continue the movement detection process and the image process selection process, and the process of selecting the storage location or storing the captured image multiple times in the same storage location is continued. It is done in the process.

  As described above, the image processing apparatus according to the present embodiment continues even after the process for the captured image is selected and confirmed by the central control unit 104 that executes the movement detection program and the process determination program that are image processing selection apparatuses. Performs movement detection and process selection, and a plurality of types of processing (in this embodiment, selection of an image storage location) or multiple processing (in this embodiment, the same storage location is the same for a certain captured image) A plurality of images) can be selected and executed in a continuous series of operations.

[Third Embodiment]
FIG. 7 is a flowchart showing image processing in the image processing apparatus according to the present embodiment, and will be described below with reference to this figure. The processing shown in the flowchart of FIG. 7 is also realized by the central control unit 104 reading out and executing each program stored in the ROM 107 and controlling the operation of each unit shown in FIG.

  In the present embodiment, the captured image on the display screen is moved based on the result of determining the movement of the image processing device based on the movement of the image captured by the imaging unit 101, which is an imaging device, and is moved to each corner of the display screen in advance. This is an example of an image processing apparatus that displays icons indicating Internet access, telephone call, album storage, and mail transmission processing in which display positions are defined, and selects and executes processing required by the user from these. FIG. 8 is a diagram showing the correspondence between the detected movement direction of the image processing apparatus and the processing content defined for each movement direction of the image processing apparatus in the present embodiment.

  Referring to FIG. 7, first, in step S <b> 401, an imaging operation for determining a captured image is performed by the user operating the shutter button of operation unit 103. In step S402, the confirmed captured image is stored as digital image data in the image buffer in the RAM 105. In step S403, the central control unit 104 executes the image display program, and the captured image stored in the image buffer. And an image indicating each operation defined in advance at a predetermined position on the display screen are superimposed on each other and displayed on the display unit 102.

  FIG. 9A shows a display example in step S403, in which four corners on the screen are defined as predetermined positions. In FIG. 9, icons are used as images indicating the operations. However, even if characters or symbols are displayed instead of the icons, the present invention is not affected even if the display location is not the four corners of the display screen. Further, in each icon in FIG. 9, the upper left corner indicates activation of the Internet access application (Internet browser), the upper right corner indicates telephone call, the lower left corner indicates storage in the album, and the lower right corner indicates activation of the mail transmission application. These functions are merely examples, and any function may be used, or a form designated in advance by the user may be used. As an example of the function, designation of a mail address to which an image is attached, designation of a location (folder or directory) for storing an image, and the like can be given.

  Next, in step S404, the central control unit 104 executes a movement detection program, and in the same way as in the first embodiment, how much the image processing apparatus determines in which direction and how much from the change in the image captured by the image capturing unit 101. Whether it has moved or not is detected, and the movement direction and the movement amount are detected and stored in the work buffer separately for the movement amounts of the x-axis and y-axis. In this movement detection, when the average value of the differences between the images changed in all ranges is larger than a predetermined value, the movement amount is stored as 0 for both the x-axis and the y-axis.

  In step S405, the value of the movement amount in the x-axis and y-axis directions is confirmed. If the movement amount is 0 for both the x-axis and the y-axis, it is considered that the movement detection has failed (No in S405), and the process proceeds to step S409. Or you may return to step S404, without performing the process after step S409. If both the x-axis and y-axis are non-zero values, it is considered that the movement detection has succeeded (Yes in S405), and the process proceeds to step S406.

  In step S406, according to the image display program, the central control unit 104 reads the xy axis movement direction and the movement amount stored in the work buffer, and moves the captured image stored in the image buffer according to the movement direction and the movement amount. To display. FIG. 9B shows a display example when the captured image is moved in the lower left direction and FIG. 9C shows the captured image in the lower right direction.

  Next, in step S407, the central control unit 104 executes a process determination program, and calculates the distance between the coordinates of the center of the captured image calculated from the movement amount stored in the work buffer and the coordinates of the center of each icon. When this distance is equal to or smaller than a predetermined value (in this embodiment, it is set to 1/8 of the diagonal line of the display screen) (Yes in S407), the center portion of the captured image is determined to be near the icon. In step S408, the application that performs the process indicated by the icon is activated. If there is no icon whose distance is equal to or smaller than the predetermined value (No in S407), the process proceeds to step S409.

  In the present embodiment, the intersection of diagonal lines of a captured image displayed as a rectangle is defined as the center of the captured image. The icon image is a rectangular image with a transparent background, and the center of the diagonal of the rectangle is defined as the center of the icon.

  In step S408, the central control unit 104 reads the captured image stored in the image buffer, and starts and executes the application defined in the icon.

  For example, as shown in FIG. 9B, when the captured image is moved in the lower right direction and the distance between the center of the captured image and the center of the icon indicating mail transmission is equal to or less than a predetermined value, the central control unit 104 Then, the e-mail transmission application is started, and the e-mail transmission screen with the captured image attached is displayed. For example, as shown in FIG. 9C, when the distance from the lower left icon is a predetermined value or less, the captured image is additionally stored in the album.

  If movement detection fails in step S405 (No in S405), in step S409, the central control unit 104 determines whether or not a process for a captured image has been manually designated by pressing a predetermined operation, for example, a number key. . If it is determined that the image processing has been manually designated (Yes in S409), the central control unit 104 performs the designated processing on the captured image in step S410. If the designated process is cancelled, the confirmed captured image is deleted and the process is terminated. If it is determined that there is no manual instruction (No in S409), the process returns to step S404, and the central control unit 104 repeats the movement detection process and the image movement process.

  In steps S409 and S410, for example, at midnight when there is no light at all, it may be difficult to extract a change in the image captured by the imaging unit 101. In such a case, the user processes the image. It is effective to prevent the fact that it cannot be done.

  As described above, in the image processing apparatus according to the present embodiment, the movement direction is detected by the central control unit 104 that executes the movement detection program that is the movement detection apparatus, using the input from the imaging unit 101 that is the imaging apparatus. Then, it is possible to select the processing of the captured image defined by the icon in the detected moving direction. That is, it is possible to select processing for captured images defined for each moving direction.

[Fourth Embodiment]
FIG. 10 is a flowchart showing image processing in the image processing apparatus according to this embodiment. The processing shown in the flowchart of FIG. 10 is also realized by the central control unit 104 reading and executing each program stored in the ROM 107 and controlling the operation of each unit shown in FIG.

  In the present embodiment, a character string is extracted from an image captured by the imaging unit 101 and displayed, the display of the character string is moved based on the result of determining the movement of the image processing apparatus, and a predefined display position An example of an image processing apparatus that selects a process instructed by a user by moving the image processing apparatus from icons indicating applications for Internet access, telephone call, storing in a notepad, and mail transmission processing displayed on is there. FIG. 11 is a diagram showing the correspondence between the detected moving direction of the image processing apparatus and the processing content defined for each moving direction of the image processing apparatus in the present embodiment.

  Referring to FIG. 10, first, in step S <b> 501, a captured image is determined by the user operating the shutter button of operation unit 103, and the digital image data determined in step S <b> 502 is stored in an image buffer in RAM 105. Is done.

  Next, in step S503, the central control unit 104 executes the character string extraction program included in the other program of FIG. 1, performs character recognition on the image data of the captured image stored in the image buffer, and recognizes the result. Are recognized and stored in the work buffer in step S504. Here, the recognition result character string candidate is a codeable area extracted as character code string information as a result of character recognition from image data by the central control unit 104 according to a character string extraction program that is a character recognition means. Information. Encoding is a process of converting letters, numbers, and symbols into letters, numbers, and symbols assigned to them so that they can be handled by a computer.

  After the character string extraction, the central control unit 104 executes the image display program in step S505, displays the recognition result character string candidate stored in the work buffer at the center of the display screen, and further overlays icons indicating each operation on the display. Are displayed on the display unit 102. FIG. 12A shows this display example. The intersection of the diagonal lines of the circumscribed rectangle of the character string that is the recognition result character string candidate is defined as the center of the character string, and is displayed so that the center of the character string overlaps the center of the display screen. To do.

  Hereinafter, as in the second embodiment, the central control unit 104 executes the movement detection program in step S506, and determines whether the movement determination is successful in step S507.

  If the movement determination is successful (Yes in S507), it is determined in step S508 whether or not the movement amount is within the display screen range based on whether or not the center of the character string is within the display screen. If it is within the display range (Yes in S508), the central control unit 104 moves and displays the extracted character string using the movement direction and the movement amount detected in Step S506 as parameters in Step S509. FIG. 12B shows an example in which the recognition result character string candidate is moved and displayed in the lower left direction and FIG. 12C is displayed in the lower right direction.

  Next, in step S510, the central control unit 104 calculates the distance between the coordinates of the center of the character string and the coordinates of the center of each icon, and these distances are predetermined values (in this embodiment, on the display screen). If it is determined that it is not equal to or less than 1/8 of the diagonal (No in S510), the process returns to Step S506, and the moving direction detection process is repeated.

  When it is determined that the distance from the icon is equal to or smaller than the predetermined value (Yes in S510), in step S511, the central control unit 104 determines the coordinates of the center of the character string as shown in FIGS. 13B and 13C. The color of the icon whose distance is less than or equal to a predetermined value is changed so as to be easily understood by the user. In the example, the color is changed from blue to pink, but it is also possible to change the icon image, change to text, or change the color or tone of the image showing the display image. . When a character is displayed in advance, the display method can be changed, such as displaying the character in black and white reversed or changing the color.

  Next, in step S512, when the user presses the process determination key (Yes in S512), the central control unit 104 selects a process corresponding to the icon having the closest distance between the center of the character string and the center of the icon. Is done. If the process determination key is not pressed (No in S512), the process returns to step S506 and the movement direction detection process is repeated.

  When the user presses the process determination key in step S512, the central control unit 104 recognizes the process of saving the recognition result character string candidate in the memo pad in the example of FIG. 12B, and recognizes the process in the example of FIG. Select and execute the process to start the mail sending application addressed to the result character string candidate address.

  After the process is selected, in step S513, the central control unit 104 performs a character string shaping process that transforms the recognition result character string candidate into an appropriate character string candidate in accordance with the selected process. The character string shaping process is a process of extracting a necessary character string from the recognition result character string candidates as a character string candidate based on the selected application. A specific example is shown in FIG. The memo input process in FIG. 14 is a process according to an application that stores a character string that is a recognition result character string candidate as it is as a memo.

  As shown in the first line of FIG. 14, when the recognition result candidate character string is “090125345678@dokoka.co.jp”, when the user selects the telephone call, the character corresponding to the telephone number is only a number. The central control unit 104 extracts only numbers from the recognition result character string candidates and transmits them to “09012345678”. When mail transmission is selected, since the character used for the mail address is an alphanumeric symbol, the central control unit 104 performs activation processing of a mail transmission application addressed to “09012345678@dokoka.co.jp”.

  As shown in the second line of FIG. 14, when the recognition result candidate character string is “Tel: 06-1111-2222”, when the user selects a telephone call, the central control unit 104 extracts only the number corresponding to the telephone number. When mail transmission is selected, the central control unit 104 does not generate the mail address because the character string “@” (at mark) required for the mail address used on the Internet is not included in the character string, and therefore the character string “Tel: 06— 1111-2222 "is inserted in the text, and the mail transmission application is activated.

  As shown in the third line of FIG. 14, when the recognition result candidate character string example is “Email: foo @ aaa, co.jp”, when the user selects the telephone call, the central control unit 104 corresponds to the telephone number. Since there is no character to be recognized in the recognition result candidate character string, it is determined as an error, a display indicating that it is not a telephone number is displayed, and the telephone call processing is terminated. In addition, when mail transmission processing is selected, “,” (comma) after “aaa” is a character that is not generally used in an email address, so it is often similar to “,” (comma). The character used is converted to “.” (Period), and the mail transmission application is activated. Also. If the storage process is selected for Notepad, the above conversion is not performed and the character string is saved as it is.

  After the character string shaping process, in step S514, the central control unit 104 performs the process of saving the recognition result candidate character string in the memo pad in the example of FIG. 12B, and the address of the recognition result candidate character string in FIG. Select and execute the process that starts the e-mail sending application.

  In the present embodiment, the central control unit 104 executes the process determination program by pressing the process execution key. However, a movement determination key for executing the movement determination program can be installed, and the movement can be performed. It is also possible to execute the process determination program when the determination key is released.

  Although not mentioned in the present embodiment, a manual operation corresponding to steps S409 and S410 of the third embodiment is repeated from the determination of steps S507, S508, S510, and S512 to return to step S506. It is also possible to insert inside.

  As described above, the image processing apparatus according to the present embodiment uses the input from the imaging unit 101 that is the imaging apparatus, and the movement detected by the central control unit 104 that executes the movement detection program that is the movement detection apparatus. The processing of the recognition result candidate character string extracted from the captured image can be selected according to the direction or the like, the character string can be shaped as if it were in the form of each application, and the processing can be executed.

[Fifth Embodiment]
FIG. 15 is a flowchart showing image processing in the image processing apparatus according to the present embodiment, and will be described below with reference to this figure. The processing shown in the flowchart of FIG. 15 is also realized by the central control unit 104 reading and executing each program stored in the ROM 107 and controlling the operation of each unit shown in FIG.

  In the present embodiment, a plurality of character string candidates are extracted as character code strings from the image captured by the imaging unit 101 by a character recognition unit, and the candidates are displayed on the display screen of the image processing apparatus. This is an example of an image processing apparatus in which a user selects an appropriate result. FIG. 16 is an example of a display screen of the image processing apparatus according to the present embodiment, and FIG. 17 is a diagram showing the correspondence between the moving direction and the processing content.

  Referring to FIG. 15, first, in step S <b> 1501, an imaging operation for determining a captured image is performed by the user operating the shutter button of operation unit 103. In step S1502, the confirmed captured image is stored in the image buffer in the RAM 105 as digital image data.

  Next, in step S1503, the central control unit 104 executes a character recognition program, extracts character string candidates including one or more character string code information from the captured image stored in the image buffer, and stores them in the work buffer. In step S1504, character string candidates for character string code information are displayed.

  A screen A in FIG. 16 is a display example of character string candidates, and the central control unit 104 displays characters from the top in the most likely order as the character strings included in the captured image as a result of the character recognition means. At this time, the initial attention candidate is considered to be the most likely character string included in the captured image, and is the first candidate displayed at the top, making the current attention candidate easy to understand for the user For this reason, it is emphasized using a bold font and displayed with an underline. Highlight display and underline display are merely examples, and reverse display and display color may be changed.

  Next, in step S1505, the central control unit 104 executes a movement detection program, and detects how much the image processing apparatus has moved in which direction from the change in the image of the imaging unit, as in the third embodiment. , Divided into movement amounts in the x-axis and y-axis directions, and stored in the work buffer. In this movement detection, when the average value of the differences between the images changed in all ranges is larger than a predetermined value, the movement amounts of the x-axis and y-axis are stored as zero.

  In step S1506, the value of the movement amount for each of the x-axis and y-axis is confirmed. If the movement amount is 0 for both the x-axis and the y-axis (No in S1506), the movement detection is considered to have failed and the process returns to step S1505. Repeat the detection process. If both the x-axis and the y-axis are non-zero values (Yes in S1506), the process proceeds to step S1507.

  In step S1507, it is determined whether the absolute value of the movement amount in the y-axis direction (vertical direction) is larger than the absolute value in the x-axis direction and the x-axis has moved in the negative direction, that is, in the downward direction. In step S1508, the attention candidate is changed to one lower level and displayed. An example of the screen B shown in FIG. 16 that has been changed from the screen A in FIG. 16 to the lower level is the screen B in FIG. 16. Similarly, the screen B can be repeatedly changed to the lower level, such as the screen C.

  In step S1507, if false (No in S1507), similarly, the upward movement is determined in step S1509. If true (Yes in S1509), the attention candidate is changed to one higher in step S1510. The operation for changing the attention candidate to the next higher level is the change from screen C to screen B and from screen B to screen A in FIG.

  Next, if it is determined in step S1511 that it has been moved to the right (Yes in S1511), by selecting the current attention candidate in step S1512, the central control unit 104 determines the character string candidate of the recognition result and ends. To do. A display example at this time is a screen D in FIG. 16, and the confirmed character string is input to an application or the like.

  As described above, the image processing apparatus according to the present embodiment uses the input from the imaging unit 101 that is the imaging apparatus, the movement direction detected by the central control unit 104 that executes the movement detection program that is the movement detection apparatus, and the like. Thus, it is possible to select a character string candidate of a recognition result extracted from the captured image.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is a figure which shows the specific example of a structure of the image processing apparatus in embodiment of this invention. It is a figure explaining the definition of the moving direction of the said image process detected in the image processing apparatus concerning this Embodiment. It is a flowchart which shows the image processing in the image processing apparatus concerning the 1st Embodiment of this invention. It is a figure which shows a response | compatibility with the movement detection direction and the processing content in the image processing apparatus concerning 1st Embodiment. It is a figure which shows the example of the image used at the time of a movement detection. It is a flowchart which shows the image processing in the image processing apparatus concerning the 2nd Embodiment of this invention. It is a flowchart which shows the image process in the image processing apparatus concerning the 3rd Embodiment of this invention. It is a figure which shows a response | compatibility with the movement of an image, and the processing content in the image processing apparatus concerning 3rd Embodiment. It is a figure which shows the specific example of the display screen which moves the captured image in the image processing apparatus concerning 3rd Embodiment. It is a flowchart which shows the image processing in the image processing apparatus concerning the 4th Embodiment of this invention. It is a figure which shows a response | compatibility with the moving direction of the image processing apparatus, and the processing content in the image processing apparatus concerning 4th Embodiment. It is a figure which shows the specific example of the display screen which moves the recognition result character string in the image processing apparatus concerning 4th Embodiment. It is a figure which shows the specific example of the display screen which changes the color of the icon in the image processing apparatus concerning 4th Embodiment. It is a figure which shows the specific example of the character string shaping from a recognition result character string. It is a flowchart which shows the image processing in the image processing apparatus concerning the 5th Embodiment of this invention. It is a figure which shows the specific example of the display screen in the image processing apparatus concerning 5th Embodiment. It is a figure which shows the response | compatibility with the moving direction of an image, and the content of a process in the image processing apparatus concerning 5th Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 Image pick-up part, 102 Display part, 103 Operation part, 104 Central control part, 105 RAM, 106 External storage part apparatus, 107 ROM.

Claims (10)

An imaging unit for photographing a subject;
A captured image determining means for determining one captured image from among the captured images obtained from the imaging unit;
Movement detecting means for detecting the moving direction of the image from the image obtained by the imaging unit;
Having one or more captured image processing means for processing the captured image;
An image processing apparatus further comprising an image processing selection unit that selects processing of the captured image determined by the captured image determination unit according to a moving direction of the image detected by the movement detection unit. The movement detection means detects the movement amount of the image from the image obtained by the imaging unit,
The image processing apparatus according to claim 1, wherein the image processing selection unit selects processing of the captured image based on a movement amount of the image detected by the movement detection unit.   The image processing apparatus according to claim 1, wherein the process performed by the image process selection unit is a process of selecting a storage location of the captured image.   The image processing apparatus according to claim 1, wherein the processing performed by the image processing selection unit is processing for selecting an application for the captured image or selecting an application activation condition for the captured image. Define specific processing contents in a predetermined direction on the display screen in advance or a predetermined position on the periphery of the display screen,
The image processing selection unit selects processing of the captured image by moving an image indicating the captured image displayed on the display screen in the vicinity of the defined direction or position. An image processing apparatus according to 1.   The image processing apparatus according to claim 5, wherein an image or a character indicating the processing content is displayed at or near the predetermined position where the specific processing content is defined.   Based on the detection result of the movement detection means, an image or character indicating the captured image when the image indicating the captured image moves in the predetermined direction or the predetermined position that defines the specific processing content Or the image processing apparatus of Claim 5 which changes the display method of the image or character which shows the said processing content. The movement detection unit and the image processing selection unit continue to detect the movement direction or amount of the image and select the processing of the captured image even after the processing for the captured image is selected by the image processing selection unit. And
The image processing apparatus according to claim 1, wherein a plurality of types of processes or a plurality of processes can be selected for the captured image. Have more character recognition means,
The image processing according to claim 1, wherein the image processing selection unit selects a character string candidate obtained by the character recognition unit according to a moving direction or a movement amount obtained by the movement detection unit. apparatus.   The image processing apparatus according to claim 9, wherein processing for changing the character string candidate extracted by the character recognizing unit to a shape that matches the processing content selected by the processing selecting unit is performed.

Images