JP2015075750A - Image recognition device and image recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correct an imaging object such as sign language or a gesture with a simple operation.SOLUTION: In an image recognition device 100, a translation control part 103 performs translation to the operation of an imaging object, and a parameter generation part 104 generates a first operation parameter, and an image display part 109 outputs a translation result, and the translation control part 103 performs translation to the operation of the imaging object acquired for correction again, and a parameter generation part 104 generates the second operation parameter of the operation of the imaging object acquired again, and a correction control part 105 compares the second operation parameter with the first operation parameter determined for each operation, and when the comparison result satisfies predetermined conditions, replaces the translation result of the operation corresponding to the first operation parameter with the translation result of the operation of the second operation parameter, and corrects the translation result, and an image display part 109 outputs the corrected translation result.

Description

  The present invention relates to an image recognition apparatus and an image recognition method for recognizing an action of a person to be photographed.

  In general, an apparatus that translates gestures, sign language, and the like by image recognition is known. In such an apparatus, it may be possible to make a correction because it may be erroneously recognized. For example, as described in Patent Document 1, a device is known in which all candidates obtained as a result of translation of input sign language are displayed and an input person can select them using a mouse or the like. ing. According to this Patent Document 1, a correct translation result can be obtained.

JP-A-6-337627

  However, in the technique described in Patent Document 1, a device such as a mouse is required to correct the recognition result of sign language, and the operation is troublesome.

  Therefore, an object of the present invention is to provide an image recognition apparatus and an image recognition method that can correct a translation result of a photographing target such as a sign language or a gesture by a simple operation.

  In order to solve the above-described problem, an image recognition apparatus according to the present invention provides an image acquisition unit that acquires a shooting target as image data, and an operation of the shooting target included in the image data acquired by the image acquisition unit. An image translating unit that performs translation for each operation, a generation unit that generates, for each operation, a first operation parameter indicating the operation of the imaging target included in the image data acquired by the image acquisition unit, A result output unit that outputs a translation result of the motion of the object to be photographed translated by the image translation unit; and a correction control unit that corrects the translation result output by the result output unit, wherein the image acquisition unit includes: After the translation result is output by the result output means, the image data including the operation of the photographing target for correction is acquired again, and the image translation means is again acquired by the image acquisition means. Translation is performed on the obtained motion of the photographing target, the generation means generates a second motion parameter of the motion of the photographing target acquired again by the image acquisition means, and the correction control means is the second control parameter. The operation parameter is compared with each of the first operation parameters determined for each operation, and the translation result of the operation corresponding to the first operation parameter satisfying a predetermined condition as a result of the comparison is obtained as the second operation parameter. The result output means is configured to output the translation result corrected by the correction control means.

  Further, in the image recognition method of the present invention, in an image recognition device for recognizing the operation of a shooting target, an image acquisition step for acquiring the shooting target as image data, and a shooting included in the image data acquired by the image acquisition step An image translation step for performing translation for each operation, and a first operation parameter indicating the operation of the imaging target included in the image data acquired by the image acquisition step is generated for each operation. A generation step for outputting, a result output step for outputting the translation result of the motion of the object to be photographed translated by the image translation step, and a correction control step for correcting the translation result output by the result output step. The acquisition step includes the operation of the photographing target for correction after the translation result is output by the result output step. Image data is acquired again, the image translation step translates the operation of the imaging target acquired again by the image acquisition step, and the generation step acquires the operation of the imaging target acquired again by the image acquisition step. The second operation parameter is generated, and the correction control step compares the second operation parameter with each of the first operation parameters determined for each operation, and the result of the comparison is a predetermined condition. The translation result of the operation corresponding to the first operation parameter satisfying the above is corrected by replacing the translation result of the operation of the second operation parameter, and the result output step includes the translation result corrected by the correction control step Is output.

  According to the present invention, the translation of the motion of the photographing target is performed, the first motion parameter is generated, the translation result is output, and then the image data including the motion of the photographing target for correction is acquired again. To do. Then, the translation of the operation of the imaging target acquired again is performed, and the second operation parameter of the operation of the imaging target acquired again is generated. The second operation parameter is compared with each of the first operation parameters determined for each operation, and the translation result of the operation corresponding to the first operation parameter for which the result of the comparison satisfies a predetermined condition is The translation result of the operation of the second operation parameter is modified and corrected, and the translation result is output.

  As a result, the correction process can be performed without specifying the correction target. In particular, it is not necessary to provide a processing configuration for designating the correction target, and the configuration can be simplified and the cost can be reduced.

  In the image recognition apparatus of the present invention, when the image translation unit recognizes an operation indicating a correction instruction, the image acquisition unit executes an image data acquisition process for correction.

  According to the present invention, when the photographing target performs an operation indicating a correction instruction, the image data acquisition process for correction is executed by recognizing it. Thus, the correction instruction can be executed without requiring a physical configuration such as a button for the correction instruction. Furthermore, since the correction is instructed by the operation, it is not necessary for the subject to be photographed nearby, and the usability can be improved.

  In the image recognition apparatus of the present invention, the correction control unit compares the second operation parameter with each of the first operation parameters determined for each operation when a predetermined condition is satisfied. .

  In the image recognition apparatus according to the present invention, the correction control means may include, as the predetermined condition, after the output of the translation result by the result output means, from the image acquisition by the image acquisition means to the second by the image translation means. When the time until operation parameter generation is within a predetermined time, operation parameter comparison processing is performed.

  In the image recognition apparatus of the present invention, the correction control means performs an operation parameter comparison process when an operation for a correction instruction is received as the predetermined condition.

  According to the present invention, when a predetermined condition is satisfied, the second operation parameter is compared with each of the first operation parameters determined for each operation, thereby grasping the operation to be corrected. be able to. Therefore, when the condition is not satisfied, the comparison process is not performed, and as a result, the normal translation process is performed.

  For example, as a predetermined condition, it is conceivable that the time from image acquisition to second operation parameter generation is within a predetermined time after the translation result is output. In this case, if the processing based on the operation is within a predetermined time, that is, the operation is short, it can be determined that the correction of a certain operation is not a communication of a series of operations. . Therefore, in such a case, it is possible to automatically perform the process by determining the correction instruction and executing the comparison process of each operation parameter for the correction target.

  In addition to automatically performing all of the predetermined conditions as described above, the correction instruction can be an instruction using a physical button, a touch panel button, or the like.

  Further, the image recognition apparatus of the present invention further includes a measuring unit that measures the time from the start to the end of the operation for each operation of the photographing target of the image data acquired by the image acquisition unit, In addition to the operation parameter indicating the operation of the imaging target, the time for each operation is generated as the operation parameter.

  According to the present invention, the time from the start to the end of the operation is measured for each operation of the imaging target of the image data, and in addition to the operation parameter indicating the operation of the imaging target, the time for each operation is used as the operation parameter. Generate. Thus, the parameter can take into account not only the operation but also the time for configuring the operation, and can be performed more accurately when determining the degree of coincidence of the operation to be corrected.

  According to the present invention, correction processing can be performed without specifying a correction target. In particular, it is not necessary to provide a processing configuration for designating the correction target, and the configuration can be simplified and the cost can be reduced.

It is a block diagram which shows the function of the image recognition apparatus 100 of this embodiment. 2 is a hardware configuration diagram of the image recognition apparatus 100. FIG. It is a figure for demonstrating the operation parameter which shows a signer's operation | movement. It is explanatory drawing which showed typically a signer's operation | movement and its translation result. It is explanatory drawing which shows the parameter table which describes an operation parameter. 5 is a flowchart illustrating a sign language translation process of the image recognition apparatus 100. It is a flowchart which shows the translation process in the modification which considered operation | movement time as an operation parameter. It is a block diagram which shows the function structure of the image recognition apparatus 100a in a modification. It is a flowchart which shows the translation process in the modification which does not perform a correction instruction | indication explicitly. It is a block diagram which shows the function structure of the image recognition apparatus 100b and the server 200 which have the structure which accesses the database of a network destination.

  Embodiments of the present invention will be described with reference to the accompanying drawings. Where possible, the same parts are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a block diagram illustrating functions of the image recognition apparatus 100 according to the present embodiment. The image recognition apparatus 100 is an apparatus for translating a gesture of a sign language or the like into text or an image, and includes a gesture input unit 101 (image acquisition unit), a translation engine unit 102, and an image display unit 109 (result output unit). , And a communication control unit 110. Further, the translation engine unit 102 includes a translation control unit 103 (image translation unit), a parameter generation unit 104 (generation unit), a correction control unit 105 (correction control unit), a gesture recognition DB 106, an intention interpretation DB 107, and a character translation DB 108. It is configured to include. The image recognition apparatus 100 is preferably a mobile terminal such as a mobile terminal or a smartphone.

  FIG. 2 is a hardware configuration diagram of the image recognition apparatus 100. As shown in FIG. 2, the image recognition apparatus 100 shown in FIG. 1 physically includes one or a plurality of CPUs 11, a main memory RAM 12 and ROM 13, and input devices 14 such as a keyboard and a mouse. The computer system includes an output device 15 such as a display, a communication module 16 that is a data transmission / reception device such as a network card, an auxiliary storage device 17 such as a semiconductor memory, and the like. Each function in FIG. 1 operates the input device 14, the output device 15, and the communication module 16 under the control of the CPU 11 by loading predetermined computer software on the hardware such as the CPU 11 and the RAM 12 shown in FIG. 2. In addition, it is realized by reading and writing data in the RAM 12 and the auxiliary storage device 17. Hereinafter, each functional block will be described based on the functional blocks shown in FIG.

  The gesture input unit 101 is a part that acquires an action of a person to be photographed such as a sign language as an image, and is a camera, for example.

  The translation engine unit 102 is a part that translates text or images that are easy to understand for those who cannot understand sign language based on the action of the person to be photographed acquired by the gesture input unit 101. As described above, the translation engine unit 102 includes a translation control unit 103, a parameter generation unit 104, a correction control unit 105, a gesture recognition DB 106, an intention interpretation DB 107, and a character translation DB 108. This component will be further described in detail.

  The translation control unit 103 is a part that translates image data including the signer's action acquired by the gesture input unit 101 into text or an image with reference to the gesture recognition DB 106, the intention interpretation DB 107, and the character translation DB 108. is there. The translation control unit 103 can recognize whether or not the signer's operation is an operation for control such as a correction instruction in addition to the text translation process.

  The parameter generation unit 104 is a part that generates one or a plurality of motion parameters indicating the motion from the image data including the motion of the sign language acquired by the gesture input unit 101. Examples of the operating parameters include those shown in FIG.

FIG. 3 is a diagram for explaining the operation parameters indicating the signer's operation. FIG. 3A shows operation parameters and specific examples thereof. As apparent from FIG. 3 (a), the operation parameters are: finger motion, number of fingers, palm motion, left and right hand motion, vertical hand motion, presence / absence of hand rotation motion, and hand It consists of the position of. The contents of each operation parameter are as follows.
Finger movement: Whether the number of fingers has changed in one movement (whether there is a change, 0 or 1)
Number of fingers: How many fingers were in one movement (how many were the most, 0-5)
Palm movement: Whether there was a flip change in one movement (whether there is a change, 0 or 1)
Left / right hand movement: Has there been a change in left / right movement in one movement (right to left, right to left, right to left, none, 0 to 3)
Up and down hand movement 2: Has there been a movement change in the up and down direction in one movement (up to down, up to down, up to down, up to down, none, 0 to 3)
Rotation of the hand: Whether there was a change in movement in the direction of rotation (whether there was a change, 0 or 1)
Hand position: The main position of the hand in one movement (0-2 in front of the face, in front of the neck, in front of the body)

  FIG. 3B shows an operation in which “obedient” is expressed in sign language. Here, an operation is shown in which a finger character “mo” is created with both hands and the fingertip is applied to the chest and simultaneously pulled up and down. In this case, the finger motion, the number of fingers, etc. are zero. On the other hand, since the hand is moving in the vertical direction, the “motion of the hand in the vertical direction” is represented by 3. Moreover, since the position of the hand is in front of the body, it is represented by 2.

  Similarly, FIG. 3C shows an operation in which “anxiety” is expressed in sign language. Here, an operation of tapping twice with the fingertips of both hands is shown. In this case, since the number of fingers is recognized, it is represented by 5. Moreover, since the position of the hand is in front of the body, it is represented by 2.

  In this way, when the translation control unit 103 recognizes the motion / position of the finger or hand, the parameter generation unit 104 can generate each operation parameter based on the recognition.

  FIG. 4 is an explanatory diagram schematically showing the actions of the signer and the translation results thereof. 4 (a) to 4 (g) represent "enviable, I can only rest for 3 days" by a series of sign language actions. This is a translation result obtained based on the translation control by the translation control unit 103 described above.

  The parameter generation unit 104 can generate operation parameters for each operation from such a sequence of sign language operations. FIG. 5 is an explanatory diagram showing a parameter table describing the operation parameters generated by the parameter generation unit 104. As shown in FIG. 5, in the parameter set composed of operation parameters P1 to P7, association is made for each operation of each sign language. For example, “enviable” is assigned 1 in the operation parameter P2 and the operation parameter P5. The parameter generation unit 104 generates a parameter set including the operation parameters P1 to P7 based on each operation of the sign language recognized by the translation control unit 103 when the sign language performs a series of operations of the sign language. In FIG. 5, as an option, sign language operation time t may be associated with each operation as one operation parameter. The operation time of each operation in a series of operations is measured by the parameter generation unit 104 (measurement means), and is used in calculating the degree of coincidence using the operation parameters.

  The correction control unit 105 is a part that corrects the translation result translated by the translation control unit 103. More specifically, the correction control unit 105 generates the operation parameter (first operation parameter) generated by the parameter generation unit 104 before the correction instruction and the parameter generation after the correction instruction when a predetermined condition is satisfied. The operation parameter (second operation parameter) generated by the unit 104 is compared, and the result of comparison with the operation parameter after the correction instruction satisfies the predetermined condition satisfying the predetermined operation condition. This is a part for performing the correction to be replaced with the translation result of the operation parameter operation after the correction instruction. For example, the correction control unit 105 can perform the correction process when the degree of coincidence is a predetermined value or more.

This will be described with reference to FIG. In FIG. 3B and FIG. 3C, the sign language operation of the sign language is very similar. As shown in FIG. 3A, the finger motions and the like are the same in each motion parameter, but the number of fingers and the hand motions in the vertical direction are different. The correction control unit 105 can calculate a similarity of 71.4% (5/7) based on each of the operation parameters, and can determine the translation result of the operation with the highest similarity as a correction target. .
An application example of such a correction target determination method will be described with reference to FIG. FIG. 4 is an explanatory diagram showing sign language actions and translation results for each action. As shown in FIG. 4A to FIG. 4G, the sign language person is performing seven operations. Here, in FIG. 4C and FIG. 4D, there is a portion expressed as “3 days”. When it is desired to correct this “3” portion to “4”, for example, the user performs an operation indicating four fingers after giving a correction instruction. The translation control unit 103 recognizes this, and the parameter generation unit 104 generates an operation parameter (second operation parameter) indicating “4”. The correction control unit 105 searches for a motion parameter (first motion parameter) having a high degree of coincidence with the motion parameter indicating “4” from the series of motions illustrated in FIG. A process of replacing the translation result of the operation having the first operation parameter) with the corrected translation result is performed. Here, since “3” shown in FIG. 4C has a high degree of coincidence with “4” (only the number of fingers is different), the translation result of “3” is the translation result of “4”. Will be replaced.

  Returning to FIG. 1, the block diagram will be described. The gesture recognition DB 106 is a database describing an operation pattern for causing the translation control unit 103 to recognize a signer's operation. This is a general database for gesture recognition.

  The intention interpretation DB 107 is a database for interpreting the meaning when the translation control unit 103 recognizes a certain amount of motion with reference to the gesture recognition DB 106. Here, an operation pattern and its meaning (intention) are described in association with each other. This is also a database for general gesture analysis.

  The character translation DB 108 is a database for translating meanings interpreted by the intention interpretation DB 107 into general sentences. For example, if it is interpreted using the intention interpretation DB 107 to indicate “self” and the subject, it is a database for translation such as “I am”. This database is also a general database in gesture translation.

  The image display unit 109 is a part that displays the translation result translated by the translation control unit 103 and the translation result corrected by the correction control unit 105.

  If the image recognition apparatus 100 configured as described above is a tablet-type mobile terminal, a gesture input unit 101 that is a camera is disposed on the back side (the surface opposite to the image display unit 109). . Then, the person to be photographed is photographed by the gesture input unit 101, translated by the translation engine unit 102, and the translation result is displayed on the image display unit 109.

  A user holding this image recognition apparatus 100 (a person who is healthy and cannot understand sign language) checks the translation result toward the person to be photographed (sign language person). If the person to be photographed wants to correct, the user can again point the gesture input unit 101 side and give a predetermined correction instruction.

  In addition, by arranging the image display unit 109 on both sides (front and back) of the image recognition apparatus 100 and arranging the gesture input unit 101 on the back side, the user does not need to change the orientation of the image recognition apparatus 100. .

  Next, a sign language translation process performed by the image recognition apparatus 100 configured as described above will be described. FIG. 6 is a flowchart showing sign language translation processing of the image recognition apparatus 100.

  The sign language action of the sign language to be photographed is photographed and input by the gesture input unit 101 (S101). The photographed sign language motion is translated by the translation control unit 103 (S102), and the parameter generation unit 104 sets the motion parameter (first motion parameter) based on the sign language motion recognized by the translation control unit 103. Generated and temporarily stored (S103). A series of these sign language operations, that is, until one sentence is completed (S104). If the translation control unit 103 determines that one sentence ends, that is, there is no sign language operation for a predetermined time after the end of a series of sign language operations, the translation result is displayed on the image display unit 109 (S105).

  Here, when a correction instruction is accepted, a correction process is started (S106). For example, as a correction instruction, pressing of a predetermined correction button (not shown) by a sign language person or another user is detected by a control unit (not shown), or a predetermined gesture (a correction instruction is shown by a sign language person). It is conceivable that the translation control unit 103 recognizes the

  Then, when a correction instruction is given by a sign language person or another user, the gesture input unit 101 starts photographing the action of the sign language (S107). Similar to the above-described photographing process and translation process before the correction instruction, the translation control unit 103 performs a translation process (S108), and the parameter generation unit 104 generates an operation parameter (second operation parameter) for storage. (S109). Here, whether the operation parameter after the correction instruction and the operation parameter before the correction instruction match, or the operation (sign language phrase) having a predetermined degree of coincidence is stored in the parameter table is corrected control. The determination is made by the unit 105 (S110).

  When the operation parameter after the correction instruction matches the operation parameter before the correction instruction, or the operation having a predetermined degree of coincidence is stored in the parameter table, the correction control unit 105 causes the operation parameter after the correction instruction. A process of replacing the translation result of the action that constitutes the operation parameter before the correction instruction with the translation result of the action that constitutes (S111). Then, the replaced entire translation result is displayed on the image display unit 109 (S111). If the translation control unit 103 determines that there is a next input process, the imaging process and the translation process are repeated again (S112). In S110, the process of returning to S107 is performed. However, the present invention is not limited to this, and an error may be generated when there is no matching phrase.

  As a result, a person who cannot understand sign language can correctly understand the matters transmitted from the sign language. If there is the next input, the process returns to S101, and the sign language translation process and its correction process are repeated.

  In S110, the correction control unit 105 identifies the operation to be corrected by determining whether the respective operation parameters match or the matching degree is equal to or greater than a predetermined value. It is not limited. That is, priority may be given to each operation parameter, or weighting may be performed so that the degree of coincidence is determined for each operation parameter. For example, if the operation parameter P1 is important, even if other operation parameters match, if the operation parameter P1 does not match, a determination process such as not satisfying a predetermined condition of the matching degree May be performed.

  In addition, when the operation parameters are the same before and after the correction instruction, and when it is normal to add 1, if the operation parameter is normal, it is assumed that the important operation parameter is multiplied by a predetermined coefficient. Thus, a weighting process may be performed.

  Next, a modification of the translation process in FIG. 7 will be described. FIG. 7 is a flowchart showing translation processing in the modification. This modification differs from FIG. 6 in that the operation parameter includes the operation time.

  As shown in FIG. 7, when a sign language action is photographed by the gesture input unit 101 and translated by the translation control unit 103 (S101, S102), an operation parameter (first operation parameter) is generated by the parameter generation unit 104. At the same time, the parameter generation unit 104 measures the time for each sign language operation (so-called sign language phrase) and stores it in the parameter table (S103a). Then, the translation result is displayed on the image display unit 109 (S105), and when a correction instruction is given (S106), the sign language action is taken again and translated (S107, S108). Also here, the parameter generation unit 104 generates an operation parameter (second operation parameter) after the correction instruction, and measures the time for each operation (so-called sign language phrase) and stores it in the parameter table ( S109a). Then, when there is an operation in which the respective operation parameters match after the correction instruction and before the correction instruction, or there is an operation (sign language phrase) having a predetermined degree of coincidence (S110), the translation result corresponding to the operation is corrected. Processing to replace the translated result of the operation after the instruction is performed and displayed on the image display unit 109 (S111). Then, as long as there is a sign language input, these processes are repeated (S112). In S110, the process of returning to S107 is performed. However, the present invention is not limited to this, and an error may be generated when there is no matching phrase.

  In this way, by using the sign language operation time as one of the operation parameters, it is possible to determine the degree of coincidence more accurately.

  Next, another modification will be described. FIG. 8 is a block diagram showing a functional configuration of the image recognition apparatus 100a in this modification. In this modification, the signer can automatically determine whether or not there has been a correction instruction in the image recognition apparatus 100a without giving a correction instruction in advance. In order to realize the processing, the image recognition apparatus 100 a includes a correction control unit 105 a and a timer 105 b built therein instead of the correction control unit 105.

  The correction control unit 105a controls the gesture input unit 101 to take a photograph of the sign language after the image display unit 109 once displays the translation result, and starts the measurement of the timer 105b. Then, when the translation control unit 103 translates the signer's movement acquired in the gesture input unit 101 and the parameter generation unit 104 generates an operation parameter for the movement, the measurement process of the timer 105b is stopped. If the correction control unit 105a determines that the translation processing time and parameter generation time measured by the timer 105b have ended within a predetermined time, the correction control unit 105a corrects according to the degree of matching of the operation parameters with respect to the re-acquired operation. Processing can be executed. In the operation parameter comparison processing, each operation parameter may be weighted or prioritized as described above.

  As in the above-described embodiment, every time correction is performed, the signer on the side where the image display unit 109 of the image recognition apparatus 100a is arranged is checked for the translation result, and the gesture is again performed after the check. The input unit 101 side may be directed to the sign language to determine whether the sign language has been continuously translated or modified. Further, the image display units 109 may be arranged on both sides of the image recognition apparatus 100a.

  FIG. 9 is a flowchart showing the specific processing. The sign language action of the sign language to be photographed is photographed and input by the gesture input unit 101 (S201). The photographed sign language motion is translated by the translation control unit 103 (S202), and the parameter generation unit 104 sets the motion parameter (first motion parameter) based on the sign language motion recognized by the translation control unit 103. Generated and temporarily stored (S203). A series of these sign language operations, that is, until one sentence is completed (S204). Here, when one sentence ends, that is, when there is no sign language operation for a predetermined time, the translation control unit 103 determines that the translation result is displayed on the image display unit 109 (S205).

  Then, the gesture input unit 101 resumes shooting of the signer (S206), and measurement of the timer 105b for measuring the sign language operation time is started (S207). Here, the timer 105b is assumed to be built in the correction control unit 105a, but is not particularly limited thereto. The translation control unit 103 performs translation based on the signer's movement input by the gesture input unit 101 (S208). Along with this translation, the parameter generation unit 104 generates an operation parameter (second operation parameter) based on the signer's operation (S209). When the operation parameter is generated, the measurement process by the timer 105b for measuring the sign language operation time is stopped, and whether or not the measurement time is within a predetermined time is determined by the correction control unit 105a (S210).

  Here, the motion parameter generated based on the motion (sign language phrase) acquired again matches the motion parameter generated based on the motion acquired earlier, or the matching degree satisfies a predetermined condition. If it is determined by the correction control unit 105 (S211), it is possible to determine that the operation of the signer acquired again is not an operation for the next sign language but an operation for correction. . If there is no match in S211, the process returns to S206. However, the present invention is not limited to this, and an error may be generated if there is no matching phrase.

  Then, a process of replacing the translation result of the operation corresponding to the operation parameter that matches or satisfies the predetermined condition is performed by the correction control unit 105 and displayed on the image display unit 109 (S212). These processes are repeated as long as there is a gesture input (S213).

  Thus, it is possible to provide an image recognition apparatus for sign language translation that can be automatically corrected without a specific instruction such as a correction instruction being performed by a sign language person or another user, and is easy to use. it can.

  In the above-described embodiment and modification, the image recognition apparatus 100 includes the gesture recognition DB 106, the intention interpretation DB 107, and the character translation DB 108, and functions as a single apparatus, but is not limited thereto. These various DBs may be provided in a server connected via a network, and the image recognition apparatus 100b may perform translation processing by accessing the various DBs of the server.

  FIG. 10 is a block diagram showing a specific configuration thereof. As illustrated in FIG. 10, the image recognition apparatus 100 b includes a gesture input unit 101, a translation engine unit 102, an image display unit 109, and a communication control unit 110. The translation engine unit 102 includes a translation control unit 103, a parameter generation unit 104, and a correction control unit 105, as in the above embodiment.

  When the image data including the signer's action is input from the gesture input unit 101, the translation engine unit 102 receives the gesture recognition DB 206, the intention interpretation DB 207, and the character recognition DB 208 of the server 200 via the communication control unit 110. To access and translate.

  The image display unit 109 displays the translation result translated by the translation engine unit 102.

  When performing the correction process, the gesture input unit 101 inputs image data including the signer's action again, and the translation engine unit 102 performs a translation process on the image data.

  The server 200 has a gesture recognition DB 206, an intention interpretation DB 207, and a character recognition DB 208. When a translation request is received from the image recognition device 100b or other communication terminals 100x to 100z, the server 200 permits reference to each DB. It is configured as follows.

  Next, functions and effects of the image recognition devices 100, 100a, and 100b in the present embodiment and the modifications will be described.

  According to the image recognition apparatus 100 of the present embodiment, the translation control unit 103 translates the motion of the photographing target input by the gesture input unit 101, and the parameter generation unit 104 is a motion parameter before the correction instruction. The first operation parameter is generated, and the image display unit 109 outputs the translation result. After that, the gesture input unit 101 obtains again image data including the action of the photographing target for correction. Then, the translation control unit 103 translates the operation of the imaging target acquired again, and the parameter generation unit 104 is the operation parameter after the correction instruction, and the second of the operation of the imaging target acquired again. Generate operating parameters.

  The correction control unit 105 compares the second operation parameter with each of the first operation parameters determined for each operation, and the result of this comparison corresponds to the first operation parameter that satisfies a predetermined condition. The translation result of the motion is replaced with the translation result of the motion of the second motion parameter, and the image display unit 109 outputs the corrected translation result.

  As a result, the correction process can be performed without specifying the correction target. In particular, it is not necessary to provide a processing configuration for designating the correction target, and the configuration can be simplified and the cost can be reduced.

  Note that the image recognition apparatus 100a according to the modification also has the same effect.

  In addition, according to the image recognition device 100 of the present embodiment or the image recognition device 100a in the modification, the translation control unit 103 recognizes the gesture when the sign language person who is the photographing target performs an operation indicating the correction instruction, The input unit 101 executes image data acquisition processing for correction. Thus, the correction instruction can be executed without requiring a physical configuration such as a button for the correction instruction. Furthermore, since the correction is instructed by the operation, it is not necessary for the subject to be photographed nearby, and the usability can be improved.

  Further, according to the image recognition apparatus 100a of the modification, the correction control unit 105 compares the second operation parameter with each of the first operation parameters determined for each operation when a predetermined condition is satisfied. By doing so, it is possible to grasp the operation to be corrected. Therefore, when the condition is not satisfied, the comparison process is not performed, and as a result, the normal translation process is performed.

  For example, as a predetermined condition, it is conceivable that the time from image acquisition to second operation parameter generation is within a predetermined time after the translation result is output. In the image recognition apparatus 100a in the modification, the timer 105b measures the time, and when the correction control unit 105a determines that the measurement time by the timer 105b is within a predetermined time, the correction process is executed.

  That is, if the processing based on the operation is within a predetermined time, that is, the operation is short, it can be determined that a certain operation is corrected rather than a communication of a series of operations. Therefore, in such a case, it is possible to automatically perform the process by determining the correction instruction and executing the comparison process of each operation parameter for the correction target.

  In addition to automatically performing all of the predetermined conditions as described above, the correction instruction can be an instruction using a physical button, a touch panel button, or the like.

  In addition, according to the image recognition device 100 of the present embodiment or the image recognition device 100a in the modification, the parameter generation unit 104 measures the time from the start to the end of the operation for each operation of the image data capturing target. In addition to the operation parameter indicating the operation to be imaged, the time for each operation is generated as the operation parameter. As a result, the operation parameter can take into account not only the operation but also the time for configuring the operation, and can be performed more accurately when determining the degree of coincidence of the operation to be corrected.

  In the above description, the operational effects of the image recognition devices 100 and 100a have been described. However, the image recognition device 100b also exhibits the same operational effects. In the image recognition apparatus 100b, since the database for translation is provided on the network side, the burden on the inner side can be reduced.

DESCRIPTION OF SYMBOLS 100, 100a, 100b ... Image recognition apparatus, 101 ... Gesture input part, 102 ... Translation engine part, 103 ... Translation control part, 104 ... Parameter generation part, 105 ... Correction control part, 105a ... Correction control part, 105b ... Timer, DESCRIPTION OF SYMBOLS 109 ... Image display part, 110 ... Communication control part, 106 ... Gesture recognition DB, 107 ... Intention interpretation DB, 108 ... Character translation DB, 206 ... Gesture recognition DB, 207 ... Intention interpretation DB, 208 ... Character recognition DB.

Claims (7)

An image acquisition means for acquiring a shooting target as image data;
Image translation means for translating for each operation for the operation of the imaging target included in the image data acquired by the image acquisition means;
Generating means for generating, for each operation, a first operation parameter indicating an operation of an imaging target included in the image data acquired by the image acquisition unit;
A result output means for outputting a translation result of the movement of the photographing object translated by the image translation means;
Correction control means for correcting the translation result output by the result output means;
With
The image acquisition means, after the translation result is output by the result output means, acquires again the image data including the operation of the photographing target for correction,
The image translation means performs translation for the operation of the photographing target acquired again by the image acquisition means,
The generation unit generates a second operation parameter of the operation of the photographing target acquired again by the image acquisition unit,
The correction control means compares the second operation parameter with each of the first operation parameters determined for each operation, and the result of the comparison corresponds to the first operation parameter satisfying a predetermined condition. The translation result of the motion to be performed is replaced with the translation result of the motion of the second motion parameter, and corrected.
The result output means outputs the translation result corrected by the correction control means;
Image recognition device. When the image translation means recognizes an operation indicating a correction instruction, the image acquisition means executes an image data acquisition process for correction,
The image recognition apparatus according to claim 1. The image recognition device according to claim 1, wherein the correction control unit compares the second operation parameter with each of the first operation parameters determined for each operation when a predetermined condition is satisfied. The correction control means has, as the predetermined condition, a time from the output of the translation result by the result output means to the second operation parameter generation by the image translation means within a predetermined time after the output of the translation result by the image acquisition means In the case where the first operation parameter is compared with the second operation parameter,
The image recognition apparatus according to claim 3. The correction control means performs a comparison process between the first operation parameter and the second operation parameter when an operation for a correction instruction is received as the predetermined condition.
The image recognition apparatus according to claim 3. For each operation to be imaged of the image data acquired by the image acquisition means, further comprises a measuring means for measuring the time from the start to the end of the operation,
6. The image recognition apparatus according to claim 1, wherein the generation unit generates a time for each operation as an operation parameter in addition to an operation parameter indicating an operation to be photographed. In an image recognition device that recognizes the operation of a shooting target,
An image acquisition step of acquiring a shooting target as image data;
An image translation step for performing translation for each operation with respect to the operation of the imaging target included in the image data acquired by the image acquisition step;
A generation step of generating, for each operation, a first operation parameter indicating an operation of a photographing target included in the image data acquired by the image acquisition step;
A result output step for outputting the translation result of the motion of the object to be photographed translated by the image translation step;
A correction control step of correcting the translation result output by the result output step;
With
In the image acquisition step, after the translation result is output by the result output step, the image data including the operation of the photographing target for correction is acquired again,
The image translation step performs translation for the operation of the photographing target acquired again by the image acquisition step,
The generation step generates a second operation parameter of the operation of the imaging target acquired again by the image acquisition step,
The correction control step compares the second operation parameter with each of the first operation parameters determined for each operation, and the result of the comparison corresponds to the first operation parameter satisfying a predetermined condition. The translation result of the motion to be performed is replaced with the translation result of the motion of the second motion parameter, and corrected.
The result output step outputs the translation result corrected by the correction control step.
Image recognition method.

Images