JP5989479B2 - Character recognition device, method for controlling character recognition device, control program, and computer-readable recording medium on which control program is recorded - Google Patents

Description

  The present invention relates to a character recognition device that recognizes characters included in an image.

  The character recognition device recognizes characters included in an image using a technique such as optical character recognition. By installing the camera in the character recognition device, the user can use the camera to capture images of characters written on store signs, documents, etc. that he sees while he is out and recognize the characters.

  However, in this case, the user needs to specify to the character recognition device which region in the captured image corresponds to the character region to be character-recognized.

  As a character recognition device equipped with a function for designating a character area, for example, a captured image is displayed on a display screen, and a part of the image is surrounded by a marker, or a user operation such as painting is performed on the operation unit. The character recognition device equipped with the function of designating the character area by accepting it is disclosed.

  In addition, a character recognition device equipped with a function for designating the character area by allowing the user to capture an image so that the character recognition target character is included in the frame displayed on the monitor screen of the camera is disclosed. ing.

  In addition, a character recognition device having a function of automatically extracting the character region by the character recognition device without specifying anything for the captured image is disclosed.

  Moreover, the following patent documents 1-3 are disclosed as a character recognition device equipped with a function for designating a character area.

  Japanese Patent Application Laid-Open No. 2004-228867 is a terminal that inputs a captured image from a CCD camera, displays the captured image on a display, specifies an image range with an input pen from a touch panel, and performs pattern recognition processing of characters from the display image in the specified range An apparatus is disclosed.

  In Patent Document 2, an image of a document captured by a camera and an image of a finger within the camera field of view indicating a document area are processed, and from the image, an area indicated in the document is displayed. An apparatus for discriminating a referenced web page is disclosed.

  In Patent Document 3, the character of a character string that represents the month is recognized to recognize the month of the paper calendar, schedule data corresponding to the recognized month is acquired, and the straight line is blocked at the tip of the finger region. A calendar apparatus that determines the display format and display period of schedule data by detecting a date frame is disclosed.

Japanese Patent Laid-Open No. 11-282836 (released on October 15, 1999) JP 2005-507526 A (published on March 17, 2005) JP 2012-43348 A (published March 1, 2012)

  In the character recognition device according to Patent Literature 1, after an image is captured by a camera, the user needs to specify a character region to be character-recognized by an input pen. Therefore, there is a problem that the procedure until the character recognition is executed becomes complicated.

  In the character recognition devices according to Patent Documents 2 and 3, the user's finger is placed on the subject of the camera, and the range specified by the finger is acquired as the character region. Therefore, the user specifies the character region after imaging. The operation of can be omitted. However, in the inventions according to Patent Documents 2 and 3, the content of processing performed using the character-recognized character is determined in advance. Therefore, it is inconvenient when the user wants to execute processing other than the above processing using the character-recognized character.

  In addition, even if a function that can select the type of processing that uses a character-recognized character is installed in the character recognition device, the user can specify an operation for specifying the character area and the type of the processing. It is necessary to continuously perform two operations of specifying “ Therefore, there is a problem that the procedure becomes complicated when it is desired to perform an operation using the character recognized character after the character recognition.

  The present invention has been made in view of the above problems, and an object thereof is to provide a character recognition device and the like that can execute character recognition and user-desired processing with a simple operation. .

  In order to solve the above problems, a character recognition device according to an aspect of the present invention is a character recognition device that recognizes characters included in an image, and an image acquisition unit that acquires an image including a finger and characters; Gesture recognition means for recognizing a gesture from position information of a finger part and a character part adjacent to the finger part from the image acquired by the image acquisition means, and a finger gesture recognized by the gesture recognition means are stored in advance. A finger gesture determination unit that determines which of the plurality of gestures matches, a character recognition unit that recognizes a character or a series of character groups at a position indicated by the finger recognized by the gesture recognition unit, and The character recognition means recognizes the process associated with the gesture determined to be the same as the finger gesture determination means. And a, a process execution means for executing relative character or stream of characters.

  A control method for a character recognition device according to an aspect of the present invention is a control method for a character recognition device that recognizes characters included in an image, the image acquisition step for acquiring an image including a finger and characters, A gesture recognition step for recognizing a gesture from position information of a finger portion and a character portion adjacent to the finger portion, and a finger gesture recognized by the gesture recognition step from an image acquired by the image acquisition step are stored in advance. A finger gesture determination step for determining which of the plurality of gestures matches, a character recognition step for recognizing a character or a series of characters at the position indicated by the finger recognized by the gesture recognition step, and the finger The process associated with the gesture determined to match the gesture determination step Including a processing execution step of executing to recognized characters or stream of characters by the character recognition step.

  The finger part recognized by the gesture recognition unit and the finger part recognized in the gesture recognition step may be a finger shape or a finger movement.

  According to an aspect of the present invention, there is an effect that a user can execute a desired process on a recognized character and recognized character only by a simple and intuitive operation of showing a desired character area with a gesture. .

It is a schematic block diagram which shows the structure of the character recognition apparatus which concerns on embodiment of this invention. It is an external view which shows an example of the gesture registered into the said character recognition apparatus, (a) is a figure which shows the case where a gesture points from the upper part of a screen, (b) is the figure which shows the case where a gesture points from the lower part of a screen (C) is a figure which shows the case where a gesture pinches | interposes an area | region with a finger | toe. It is a figure which shows an example of the gesture registered into the said character recognition apparatus. It is a table which shows the correspondence of the kind of said gesture, and the process which the said character recognition apparatus performs. It is the flowchart which showed an example of the flow of the process in the said character recognition apparatus.

  One embodiment of the present invention will be described below in detail with reference to FIGS.

(Configuration of character recognition device 1)
FIG. 1 is a schematic block diagram showing the configuration of the character recognition device 1 according to this embodiment. As shown in FIG. 1, the character recognition device 1 includes a control unit 2, a storage unit 3, a camera (image acquisition unit) 4, a communication unit (communication unit) 5, a display unit 6, and an operation unit 7. .

  The character recognition device 1 is an information processing device having a function of recognizing characters included in an image acquired by imaging. The character recognition device 1 may be an information processing device such as a personal computer, a mobile phone, a smartphone, a tablet PC, or a game device.

  The control unit 2 controls the entire character recognition device 1 in an integrated manner, and can be configured by, for example, a CPU (Central Processing Unit). The control unit 2 controls each of the storage unit 3, the camera 4, the communication unit 5, and the display unit 6. The detailed configuration of the control unit 2 will be described later.

  The storage unit 3 is executed by the control unit 2 (1) a control program for each unit, (2) an OS (Operation System) program, (3) an application program, and (4) various data read when executing these programs. Is memorized. The storage unit 3 is configured by a nonvolatile storage device such as a flash memory. In addition, the storage unit 3 is configured by a volatile storage device such as a RAM (Random Access Memory) as a work area for temporarily storing data in the process of executing the various programs described above by the control unit 2. Has an area. Note that the storage unit 3 is not necessarily provided in the character recognition device 1 and is an external storage device that can be attached to and detached from the character recognition device 1 or an external storage device on a network that can communicate via the communication unit 5. Further, a configuration connected to the character recognition device 1 may be possible.

  The storage unit 3 particularly stores a finger image pattern, a finger image gesture correspondence table, a dictionary database, a memo (character), and the like.

  The camera 4 has a function of a normal camera that captures an image of an object such as a character, background, or person as a subject based on a user operation received by the operation unit 7. In particular, the camera 4 captures an object including a finger and characters as a subject. Then, the camera 4 outputs the image acquired by the imaging to the control unit 2. In addition, the camera 4 may have a function of periodically capturing a plurality of images. Further, the camera 4 may have a function of capturing an image. In this case, the camera 4 captures a plurality of images or videos in accordance with a user operation received by the operation unit 7. Note that the camera 4 may have a function of switching a part of the processing between imaging a background, a person, and the like and imaging for capturing characters for the purpose of character recognition.

  The camera 4 may output the acquired video to the control unit 2 in real time.

  The communication unit 5 is a communication interface for the character recognition device 1 to perform Internet communication via a communication network. In the character recognition device 1, information is retrieved from the Internet using the character recognized character as a key via the communication unit 5.

  The display unit 6 is a display device that displays an image based on an instruction from the control unit 2. An LC (Liquid Crystal) display panel, an EL (Electro Luminescence) display panel, or the like can be applied as the display unit 6. Although not shown, the control unit 2 or the display unit 6 is appropriately provided with a configuration necessary for displaying an image such as a VDP (Video Display Processor) and a VRAM (Video RAM). The display unit 6 may be a touch panel having both functions of image display and operation input. In the character recognition device 1, the display unit 6 displays the result of executing the process using the character-recognized character. The display unit 6 also serves as a monitor screen for the camera 4.

  The operation unit 7 receives an operation of the user of the character recognition device 1 and is typically a physical key, a keyboard, a touch panel, or the like. When the display unit 6 is a touch panel, the display unit 6 also functions as the operation unit 7.

(Detailed configuration of the control unit 2)
The configuration of the control unit 2 will be described in detail. The control unit 2 includes a gesture determination unit (gesture recognition unit, finger gesture determination unit) 10, an image combination unit (connection unit) 11, a character cutout unit (character recognition unit) 12, and a character recognition processing unit (character recognition unit) 13. , A keyword search unit (process execution unit) 14, a dictionary search unit (process execution unit) 15, and a storage processing unit (process execution unit) 16.

  When an object including a finger and characters is imaged as a subject by the camera 4, the gesture determination unit 10 acquires an image showing the object as a subject from the camera 4, and the finger indicates from the image. It is determined whether or not the gesture is a gesture registered in advance in the character recognition device 1.

  Specifically, the gesture determination unit 10 cuts out a finger area that is an area along the contour of the finger from the image as an image, and determines the finger gesture indicated by the finger area using a technique such as pattern recognition. To do.

  For example, the storage unit 3 stores a finger image pattern indicating the appearance of a finger gesture. The storage unit 3 stores a finger image gesture correspondence table in which the finger image pattern is associated with the type of finger gesture corresponding to the finger image pattern. At this time, the character recognition processing unit 13 searches the finger image pattern for an image pattern having the highest degree of similarity with the cut image of the finger region. Then, the character recognition processing unit 13 refers to the finger image gesture correspondence table, and determines that the gesture corresponding to the image pattern determined to have the highest similarity is a finger gesture candidate indicated by the finger region. Then, when the similarity of the candidate is lower than a predetermined value, the gesture determination unit 10 determines that the candidate is a gesture that is not registered in the character recognition device 1 in advance. On the other hand, when the similarity of the candidate is a predetermined value or more, it is determined that the candidate is a gesture registered in advance in the character recognition device 1.

  The storage unit 3 may store the relative positions of the finger area and the character area in the storage unit 3. In this case, the gesture determination unit 10 determines a gesture from the finger gesture and the relative positions of the finger and the adjacent character area. According to said structure, the determination precision can be improved rather than the determination only by a finger, or the determination only by a character area.

  In the present embodiment, gestures registered in advance in the character recognition device 1 are gesture A (second gesture), gesture B (second gesture), gesture C (third gesture), and gesture D (first gesture). There are four types. Details of the appearance of the gestures A to D will be described later. In the present embodiment, the number of registered gestures is described as four. However, the number of gestures is not limited to four and may be any number.

  Note that the method by which the gesture determination unit 10 determines the type of gesture is not limited to the pattern recognition method described above.

  In addition, the image for the gesture determination unit 10 to determine the type of gesture is not limited to the image captured by the camera 4, and may be an image acquired from a web page.

  Further, the camera 4 may have a function that the gesture determination unit 10 has. In this case, the camera 4 may further include a function of capturing an image of an object including the finger and characters as a subject by using the state where the finger is stationary for a while as a trigger.

  When the gesture determined to be a gesture registered in advance in the character recognition device 1 by the gesture discriminating unit 10 is a gesture surrounding the character area, the image combining unit 11 is a plurality of images captured by the camera 4 or A video is obtained from the camera 4 and the images or videos are combined. Here, the gesture that starts the motion surrounding the character area corresponds to the gesture D. Details of specific processing executed by the image combining unit 11 will be described later.

  When the gesture discriminating unit 10 determines that the gesture indicated by the finger area is a gesture registered in advance in the character recognition device 1, the character cutting unit 12 uses different methods depending on the type of the gesture, A part of the area is cut out from the image. Details of the cutout process performed by the character cutout unit 12 will be described later.

  The character recognition processing unit 13 performs character recognition processing on the image including the character area acquired by the character cutout unit 12. Then, the character recognition processing unit 13 acquires a character or a series of characters recognized as a character code. The series of character groups includes English words included in the character region. The character recognition processing unit 13 uses a technique such as optical character recognition (hereinafter referred to as OCR (Optical Character Recognition)) as a character recognition method.

  The keyword search unit 14 corresponds to the character code when the gesture determined by the gesture determination unit 10 as a gesture registered in the character recognition device 1 in advance is a gesture that directs a character from above with a finger. Information is searched from the Internet via the communication unit 5 using a character or a series of characters as a key (hereinafter referred to as keyword search). Here, the gesture in which a character is directed from the top with a finger corresponds to the gesture A. Then, the keyword search unit 14 displays the obtained keyword search result on the display unit 6.

  The dictionary search unit 15 is stored in the storage unit 3 when the gesture determined to be a gesture registered in the character recognition device 1 in advance by the gesture determination unit 10 is a gesture for directing a character from below with a finger. A term is searched from a dictionary database created by associating a term with a description of the term, using a character or a series of characters corresponding to the character code as a key (hereinafter referred to as dictionary search). Here, the gesture for instructing a character from below with a finger corresponds to the gesture B. Then, the dictionary search unit 15 displays the obtained dictionary search result on the display unit 6. In addition, the kind of dictionary database stored in the memory | storage part 3 is not specifically limited, The database containing terms, such as a national language dictionary, an English-Japanese dictionary, a Japanese-English dictionary, an encyclopedia, or a technical term dictionary, may be sufficient.

  The memory processing unit 16 determines that the gesture determined in advance by the gesture determination unit 10 as a gesture registered in the character recognition device 1 is a gesture that sandwiches a character with two fingers, or uses the gesture D described above. If there is, a character or a series of character groups corresponding to the character code is stored in the storage unit 3 as a memo (hereinafter referred to as memo recording). Here, the gesture in which a character is sandwiched between two fingers corresponds to the gesture C. The storage processing unit 16 may read the content of the memo from the storage unit 3 and display it on the display unit 6 at an arbitrary timing designated by the user based on a user operation received by the operation unit 7.

(Character cutout 12)
Details of the cutout process executed by the character cutout unit 12 will be described.

  2A to 2C are external views showing examples of gestures registered in the character recognition device 1, respectively. Each gesture shown in (a) to (c) of FIG. 4 is displayed on the display unit 6 of the character recognition device 1.

  FIG. 2A shows a gesture for directing a character from above with a finger, and corresponds to the gesture A described above. In this case, the character cutout unit 12 outputs an image generated by cutting out a character or a series of character groups immediately below the tip of the finger to the character recognition processing unit 13 as a character area to be subjected to character recognition. To do.

  FIG. 2B shows a gesture for directing a character from below with a finger, and corresponds to the gesture B described above. In this case, the character cutout unit 12 outputs an image generated by cutting out a character or a series of character groups immediately above the tip of the finger to the character recognition processing unit 13 as a character area to be subjected to character recognition. To do.

  (C) of FIG. 2 shows a gesture for sandwiching a character with two fingers, and corresponds to the gesture C described above. In this case, the character cutout unit 12 uses the image generated by cutting out a character or a series of character groups sandwiched between the tips of two fingers as a character recognition target character region. Output to the processing unit 13.

(Image combiner 11)
Details of specific processing executed by the image combining unit 11 will be described.

  FIG. 3 is an external view showing an example of a gesture registered in the character recognition device 1, and shows a gesture surrounding a character area when a user moves a finger or a finger and the camera 4. is there.

  First, when capturing a character area by specifying an area, the user presents a gesture for starting the operation of surrounding the area with a finger to the camera 4. At this time, the gesture determination unit 10 determines that the gesture is a part of the gesture D registered in the character recognition device 1 in advance.

  Next, as shown in FIG. 3, the user moves the finger while moving the camera 4 on the newspaper 50 and surrounds the character area, thereby designating a character area to be subjected to character recognition. During the movement, the camera 4 periodically captures a part of the character area as an image or an image, thereby acquiring a plurality of images or videos each including a part of the character area. Then, the user presents a gesture to the camera 4 indicating that the operation of surrounding the area with the finger is to be ended. At this time, the camera 4 ends the acquisition of the image or video.

  On the other hand, the image combining unit 11 combines the plurality of images or videos. Specifically, the image combining unit 11 matches overlapping regions between the plurality of images or the plurality of images obtained by sampling the video, and based on the matched regions, the plurality of images Of images. When connecting the regions, the image combining unit 11 connects the images using the image including the gesture D as a starting point and the image including the gesture indicating that the region designation has been completed as an end point. Then, the image combining unit 11 outputs one image generated by combining the plurality of images or videos to the character cutting unit 12.

(Modification)
In the above configuration, when the object including a finger and a character is captured as a subject by the camera 4, the character recognition device 1 acquires an image in which the object is captured as a subject from the camera 4 and discriminates a gesture from the image. To do. Therefore, in order for each block included in the control unit 2 to discriminate a finger gesture, an image or a video needs to be captured by the camera 4 in advance.

  Therefore, the video captured by the camera 4 may be output to the control unit 2 in real time, and the control unit 2 may be configured to continuously determine the input video.

  According to the above configuration, for example, a word translated by the dictionary search unit 15 can be displayed in real time at the same time as holding a finger over a character while copying to the camera 4.

  Further, according to the above configuration, a fine finger movement pattern (finger vertical reciprocation, left-right reciprocation, etc.) can be determined as the type of gesture. For example, the gesture for starting the movement of surrounding the above area with a finger and the gesture for ending the movement of surrounding the above area with a finger are not only special finger shapes but also a fine finger movement (such as a finger And the like can be determined by the gesture determination unit 10. Further, although the shape of the finger does not change, the gesture determination unit 10 can determine that the surrounding operation is in progress by relatively changing the position of the surrounding character area.

  Furthermore, according to said structure, when the camera 4 recognizes a finger in real time, a part of user's operation received in the operation part 7 can be substituted.

(Correspondence between gesture and processing)
FIG. 4 is a table showing the correspondence between the types of gestures registered in the character recognition device 1 and the processing executed by the character recognition device 1 in accordance with the gestures.

  As illustrated in FIG. 4, when a gesture (corresponding to gesture A) that instructs a character from the top is captured by the camera 4, the keyword search unit 14 performs a keyword search. Further, when a gesture (corresponding to gesture B) that instructs a character from the lower part is captured by the camera 4, the dictionary search unit 15 performs a dictionary search. When the camera 4 captures a gesture that sandwiches a character with two fingers (corresponding to gesture C) or a gesture that surrounds an area with a finger (corresponding to gesture D), The character group is recorded in a memo.

  In addition, when recording a character as a memo, the character recognition device 1 may hold an image layout, for example, as a cutout of a newspaper, or may record only a text character after character recognition. In addition, the character recognition device 1 determines a special finger pattern after recognizing a gesture of sandwiching a character with two fingers or a gesture surrounding a region with a finger, and based on the determination result, as described above. Different recording methods may be selected.

  As described above, the character recognition device 1 performs different processing using the character recognized by the character recognition processing unit 13 according to the type of the gesture determined by the gesture determination means. .

  In the present embodiment, the gestures registered in advance in the character recognition device 1 are the gestures A to D. However, the types of gestures that can be registered in the character recognition device 1 are not limited to the gestures A to D. .

  Moreover, the process which the character recognition apparatus 1 performs is not limited to each above-mentioned process.

  The correspondence relationship between the types of gestures and the processing executed by the character recognition device 1 according to the gestures is an example, and the scope of application of the present invention is not limited to these correspondence relationships.

(Process flow)
FIG. 5 is a flowchart illustrating an example of a process flow in the character recognition device 1.

  First, an object including a finger and characters is imaged as a subject by the camera 4 (S1). At this time, the gesture determination unit 10 acquires an image in which the object including the finger and the character is captured as a subject.

  Next, the gesture discrimination | determination part 10 cuts out the finger area | region which is an area | region along the outline of a finger from the said image (S2).

  And the gesture discrimination | determination part 10 determines whether the gesture which the said finger area | region shows is the gesture previously registered into the character recognition apparatus 1 (S3). That is, whether the gesture indicated by the finger area corresponds to any one of the four types of gesture A, gesture B, gesture C, and gesture D, or does not correspond to any of the above four types of gestures Determine.

  If the gesture is a gesture registered in advance in the character recognition device 1 (Yes in S3), that is, if the gesture indicated by the finger area corresponds to one of the four types of gestures, Advances to step S4.

  On the other hand, if the gesture is not registered in the character recognition device 1 in advance (No in S3), that is, if the gesture indicated by the finger area does not correspond to any of the four types of gestures, Return to S1.

  Then, the image combining unit 11 determines whether or not the gesture determined by the gesture determining unit 10 is a gesture surrounding the character area (that is, gesture D) (S4).

  If the gesture is a gesture surrounding the character area (Yes in S4), the process proceeds to step S5.

  On the other hand, when the gesture is not a gesture surrounding the character area (No in S4), the process proceeds to step S6.

  Then, the image combining unit 11 acquires a plurality of captured images or videos from the camera 4 and combines the images or videos (S5).

  Then, the character cutout unit 12 cuts out a partial region from the image by a different method depending on the type of gesture determined by the gesture determination unit 10 (S6). For example, when the gesture indicated by the finger region is gesture A, the character cutout unit 12 cuts out a character or a series of character groups immediately below the tip of the finger included in the image as a character region.

  And the character recognition process part 13 performs character recognition with respect to the character area cut out by the character cutout part 12 (S7). Then, the character recognition processing unit 13 acquires a character or a series of characters recognized as a character code.

  Subsequently, the gesture determination unit 10 selects a process using the character code according to the determined type of gesture (S8).

  Specifically, when the gesture discriminated by the gesture discriminating unit 10 directs a character from the top with a finger (gesture A in S8), the gesture discriminating unit 10 instructs the keyword search unit 14 to enter the keyword Instructs the search to be performed.

  And the keyword search part 14 performs a keyword search using the said instruction | indication from the gesture discrimination | determination part 10 as a trigger using the character or a series of character groups corresponding to the said character code (S9).

  Further, when the gesture determined by the gesture determination unit 10 is to direct a character from below with a finger (gesture B in S8), the gesture determination unit 10 performs a dictionary search to the dictionary search unit 15. Instruct.

  Then, the dictionary search unit 15 uses the instruction from the gesture determination unit 10 as a trigger to perform a word search using a character or a series of characters corresponding to the character code and a dictionary database stored in the storage unit 3. Execute (S10).

  Also, when the gesture determined by the gesture determination unit 10 is a character that sandwiches a character with two fingers (gesture C in S8), or when the character region is surrounded by a finger (gesture in S8) D) The gesture determination unit 10 instructs the storage processing unit 16 to execute memo recording.

  And the memory | storage process part 16 stores the character or a series of character group corresponding to the said character code as a memo in the memory | storage part 3 by using the said instruction | indication from the gesture discrimination | determination part 10 as a trigger (S11).

(Summary)
A character recognition device according to an aspect of the present invention is a character recognition device that recognizes characters included in an image, and includes an image acquisition unit (camera 4) that acquires an image including a finger and characters, and the image acquisition unit includes: From the acquired image, a gesture recognition unit (gesture determination unit 10) that recognizes a gesture from position information of a finger part and a character part adjacent to the finger part, and a finger gesture recognized by the gesture recognition unit Recognizes a character or a series of characters at the position indicated by the finger recognized by the finger recognition unit (gesture determination unit 10) for determining which of the plurality of stored gestures matches, and the gesture recognition unit Character recognition means (character recognition processing unit 13) that performs the gesture determination that the finger gesture determination means matches. A process execution unit (keyword search unit 14, dictionary search unit 15, storage processing unit 16) that executes a process associated with a character on a character or a series of character groups recognized by the character recognition unit. ing.

A control method for a character recognition device according to an aspect of the present invention is a control method for a character recognition device that recognizes characters included in an image, and an image acquisition step (S1) for acquiring an image including a finger and characters. When,
A gesture recognition step (S2, S3) for recognizing a gesture from position information of a finger portion and a character portion adjacent to the finger portion from the image acquired by the image acquisition step, and the finger recognition recognized by the gesture recognition step. A finger gesture determination step (S3, S4) for determining which of the plurality of gestures is stored in advance, and a character or a series of characters at the position indicated by the finger recognized by the gesture recognition step The character recognition step (S7) for recognizing a group and the processing associated with the gesture determined to match the finger gesture determination step are performed on the character or series of characters recognized by the character recognition step. Process execution steps (S9, S10, S11) to be executed.

  The finger part recognized by the gesture recognition unit and the finger part recognized in the gesture recognition step may be a finger shape or a finger movement.

  According to said structure, the said character recognition apparatus recognizes the character (or series of character group) of the position shown with a finger | toe in the image containing a finger | toe and a character, The process matched with the gesture of the finger | toe, Execute on a recognized character (or set of characters). Here, the series of character groups is, for example, English words in English sentences.

  Thereby, the said character recognition apparatus can recognize a desired character only with a finger gesture, and can perform a desired process with respect to the recognized character.

  Therefore, the user can perform the character recognition and the desired processing for the recognized character continuously by simple and intuitive operation of showing the desired character area with the gesture.

  Further, the image acquisition means acquires a plurality of images having a common part in a part of each other image, and the character recognition device is configured such that a finger gesture included in each of the plurality of images is the one of the plurality of gestures. When the finger gesture determination unit determines that the first gesture (gesture D), which is one of them, matches, a combining unit (image combining unit) that combines the plurality of images by superimposing the common parts 11) may be provided.

  According to the above configuration, the character recognition device acquires a plurality of images having a common part in a part of each other image, and a finger gesture included in each of the plurality of images is the plurality of gestures. If it is one of them, the plurality of images are combined by superimposing the common parts.

  Therefore, the user can easily set the target range for character recognition even for an imaging target that does not fit in the imaging range by one imaging.

  Further, the character recognition device includes a communication unit (communication unit 5) that communicates with an external device via a communication network, and the processing execution unit (keyword search unit 14) detects the finger recognized by the gesture recognition unit. A character or a series of characters recognized by the character recognition means when the finger gesture determination means determines that a gesture matches a second gesture (gesture A) which is one of the plurality of gestures. Information may be retrieved from the communication network via the communication means using a group as a key.

  According to the above configuration, when the recognized finger gesture is one of the plurality of gestures, the character recognition device communicates using the recognized character (or a series of characters) as a key. Retrieve information from the network.

  Therefore, the user can retrieve information from the communication network using the recognized character and the recognized character as a key only by a simple and intuitive operation of showing a desired character with a gesture.

  Furthermore, the character recognition device includes a dictionary database created by associating terms with explanations of terms, and the processing execution means (dictionary search unit 15) is configured such that the finger gesture recognized by the gesture recognition means is When the finger gesture determination unit determines that the third gesture (gesture B), which is one of the plurality of gestures, matches a character or a series of characters recognized by the character recognition unit As an alternative, the dictionary database term may be searched.

  According to the above configuration, when the recognized finger gesture is one of the plurality of gestures, the character recognition device uses the recognized character (or a series of characters) as a key. A term is searched from a dictionary database created by associating a term with a description of the term.

  Therefore, the user can perform a dictionary search using the recognized character and the recognized character as a key only by a simple and intuitive operation of showing a desired character with a gesture.

  The character recognition device further includes a storage unit for storing characters, and the processing execution unit (storage processing unit 16) is configured such that the finger gesture recognized by the gesture recognition unit is one of the plurality of gestures. When the finger gesture determination unit determines that the fourth gesture (gesture C) is the same, the character or the series of characters recognized by the character recognition unit may be stored in the storage unit.

  According to the above configuration, when the recognized finger gesture is one of the plurality of gestures, the character recognition device stores the recognized character (or a series of character groups).

  Therefore, the user can recognize the character and memorize the recognized character only by a simple and intuitive operation of showing a desired character with a gesture.

  The character recognition device may be realized by a computer. In this case, a control program for the character recognition device that causes the character recognition device to be realized by the computer by causing the computer to operate as each of the means, and A computer-readable recording medium on which it is recorded also falls within the scope of the present invention.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

(Example of software implementation)
Finally, each block of the character recognition device 1, particularly the control unit 2, may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or a CPU (Central Processing Unit) is used. It may be realized by software.

  In the latter case, the character recognition device 1 includes a CPU that executes instructions of a program that realizes each function, a ROM (Read Only Memory) that stores the program, a RAM (Random Access Memory) that expands the program, the program, A storage device (recording medium) such as a memory for storing various data is provided. An object of the present invention is a recording medium on which a program code (execution format program, intermediate code program, source program) of a control program of the character recognition device 1 which is software for realizing the functions described above is recorded so as to be readable by a computer. This can also be achieved by supplying the character recognition apparatus 1 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include non-transitory tangible medium, such as magnetic tape and cassette tape, magnetic disk such as floppy (registered trademark) disk / hard disk, and CD-ROM / MO. Discs including optical discs such as / MD / DVD / CD-R, cards such as IC cards (including memory cards) / optical cards, semiconductor memories such as mask ROM / EPROM / EEPROM (registered trademark) / flash ROM Alternatively, logic circuits such as PLD (Programmable Logic Device) and FPGA (Field Programmable Gate Array) can be used.

  The character recognition device 1 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited as long as it can transmit the program code. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication network, and the like can be used. The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, even with wired lines such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, and ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA and remote control, Bluetooth (registered trademark), IEEE 802.11 wireless, HDR ( It can also be used by radio such as High Data Rate (NFC), Near Field Communication (NFC), Digital Living Network Alliance (DLNA), mobile phone network, satellite line, and digital terrestrial network. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention can be used for an information processing apparatus equipped with a character recognition function for recognizing characters included in an image. In particular, it can be widely applied to various information processing apparatuses such as a personal computer, a mobile phone, a smartphone, a tablet PC, and a game machine.

1 character recognition device 3 storage unit 4 camera (image acquisition means)
5 Communication part (communication means)
10 Gesture discriminating unit (gesture recognition means, finger gesture judgment means)
11 Image combiner (combination means)
12 Character extraction part (character recognition means)
13 Character recognition processing unit (character recognition means)
14 Keyword search part (process execution means)
15 Dictionary search unit (process execution means)
16 Storage processing unit (processing execution means)

Claims (8)

A character recognition device for recognizing characters included in an image,
Image acquisition means for acquiring an image including a finger and a character;
Gesture recognition means for recognizing a gesture from position information of a finger part and a character part adjacent to the finger part from the image acquired by the image acquisition means;
Finger gesture determination means for determining which of the plurality of gestures stored in advance is a finger gesture recognized by the gesture recognition means;
A character recognition means for recognizing a character or a series of characters at a position indicated by the finger recognized by the gesture recognition means;
Processing execution means for executing processing associated with the gesture determined to match the finger gesture determination means on the character or series of characters recognized by the character recognition means ,
The position indicated by the finger recognized by the gesture recognition means is
When directing a character from the top of the display unit with a finger, it is a character or a series of characters directly under the tip of the finger in the display unit,
When directing a character from the lower part of the display unit with a finger, a character or a series of character groups immediately above the tip of the finger in the display unit,
When a character is sandwiched between two fingers, it is a character or a series of characters in a portion sandwiched between the tips of two fingers.
A character recognition device. The image acquisition means acquires a plurality of images having a common part in a part of each other image,
When the finger gesture determination unit determines that a finger gesture included in each of the plurality of images matches a first gesture that is one of the plurality of gestures, the common portion is superimposed. The character recognition apparatus according to claim 1, further comprising a combining unit that combines the plurality of images. A communication means for communicating with an external device via a communication network;
When the finger gesture determination unit determines that the finger gesture recognized by the gesture recognition unit matches a second gesture that is one of the plurality of gestures, the processing execution unit 3. The character recognition apparatus according to claim 1, wherein information is retrieved from the communication network via the communication unit using a character or a series of character groups recognized by the recognition unit as a key. A dictionary database created by associating terms with term descriptions;
When the finger gesture determination unit determines that the finger gesture recognized by the gesture recognition unit matches a third gesture that is one of the plurality of gestures, the processing execution unit 4. The character recognition apparatus according to claim 1, wherein a term in the dictionary database is searched using a character or a series of character groups recognized by a recognition means as a key. A storage unit for storing characters;
When the finger gesture determination unit determines that the finger gesture recognized by the gesture recognition unit matches a fourth gesture that is one of the plurality of gestures, the processing execution unit The character recognition device according to any one of claims 1 to 4, wherein a character or a series of character groups recognized by a recognition unit is stored in the storage unit. A method for controlling a character recognition device that recognizes characters included in an image,
An image acquisition step of acquiring an image including a finger and a character;
A gesture recognition step for recognizing a gesture from position information of a finger part and a character part adjacent to the finger part from the image acquired by the image acquisition step;
A finger gesture determination step for determining which of the plurality of gestures stored in advance is the finger gesture recognized by the gesture recognition step;
A character recognition step for recognizing a character or a series of characters at the position indicated by the finger recognized by the gesture recognition step;
The process of the finger gesture determination step is associated with the gesture is determined that they coincide, see containing and a process executing step of executing to recognized characters or stream of characters by the character recognition step,
The position indicated by the finger recognized in the gesture recognition step is
When directing a character from the top of the display unit with a finger, it is a character or a series of characters directly under the tip of the finger in the display unit,
When directing a character from the lower part of the display unit with a finger, a character or a series of character groups immediately above the tip of the finger in the display unit,
When a character is sandwiched between two fingers, it is a character or a series of characters in a portion sandwiched between the tips of two fingers.
A control method for a character recognition device.   A control program for operating a computer included in the character recognition device according to any one of claims 1 to 5, wherein the control program causes the computer to function as each of the means.   A computer-readable recording medium on which the control program according to claim 7 is recorded.

Images