WO2021015117A1 - データ処理システム、およびデータ処理方法 - Google Patents

データ処理システム、およびデータ処理方法 Download PDF

Info

Publication number
WO2021015117A1
WO2021015117A1 PCT/JP2020/027797 JP2020027797W WO2021015117A1 WO 2021015117 A1 WO2021015117 A1 WO 2021015117A1 JP 2020027797 W JP2020027797 W JP 2020027797W WO 2021015117 A1 WO2021015117 A1 WO 2021015117A1
Authority
WO
WIPO (PCT)
Prior art keywords
annotation
inference
image
data processing
teacher
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/027797
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
田中　一徳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kyocera Document Solutions Inc
Original Assignee
Kyocera Document Solutions Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kyocera Document Solutions Inc filed Critical Kyocera Document Solutions Inc
Priority to US17/597,804 priority Critical patent/US12002191B2/en
Priority to JP2021533997A priority patent/JP7238992B2/ja
Publication of WO2021015117A1 publication Critical patent/WO2021015117A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/12—Detection or correction of errors, e.g. by rescanning the pattern
    • G06V30/127—Detection or correction of errors, e.g. by rescanning the pattern with the intervention of an operator
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G06V10/987—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns with the intervention of an operator
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • the present invention relates to a data processing system and a data processing method.
  • the data processing system described in Patent Document 1 is different from a learning means for learning and modeling the relationship between a teacher label given to at least one area on a learning image and an image of the area, and a learning image. It is provided with a detection means that receives a detection image as an input and detects a region including an object based on a model from a detection image that has been subjected to predetermined image processing and a detection image that has not been subjected to image processing. ..
  • Patent Document 1 does not determine whether or not the detected region of the detection image includes an object based on the model. Therefore, there is a problem that the labor of assigning labels to the learning image cannot be suitably reduced.
  • the present invention has been made in view of the above problems, and an object of the present invention is to provide a data processing system and a data processing method capable of suitably reducing correction work for inference annotations.
  • the data processing system includes a storage unit, an inference unit, a support unit, and a correction unit.
  • the storage unit stores a teacher model in which a teacher annotation that identifies the target image is added to the target image.
  • the inference unit adds an inference annotation for inferring the inspection image to the inspection image belonging to the same category as the target image.
  • the support unit assists a human in determining whether or not the inference annotation identifies the inspection image.
  • the correction unit adds a correction annotation to the inference annotation to generate a correction model so that the inspection image can be specified.
  • the data processing method includes a step of inputting a target image, a step of adding a teacher annotation for specifying the target image to the target image, and the teacher annotation to the target image.
  • a step of giving, a step of assisting a human in determining whether or not the inference annotation identifies the inspection image, and a modification annotation added to the inference annotation so that the inspection image is identified. Includes steps to generate a modified model.
  • FIG. 1 It is a figure which shows the data processing system which concerns on this embodiment of this invention. It is a block diagram which shows the data processing apparatus provided in the data processing system which concerns on this embodiment. Is a diagram showing a target image of the data processing system according to the present embodiment. Is a diagram showing a teacher model of a data processing system. It is a figure which shows the list which a data processing apparatus has. Is a diagram showing an inspection image of the data processing system according to the present embodiment. Is a diagram showing an inference model of a data processing system. It is a figure which shows the list which a data processing apparatus has. Is a diagram showing an inspection image of the data processing system according to the present embodiment. Is a diagram showing inference annotations of a data processing system.
  • Is a diagram showing a modified model of a data processing system It is a figure which shows the list which a data processing apparatus has. It is a flowchart which shows the operation of the data processing system which concerns on this embodiment. It is a flowchart which shows the operation of the data processing system which concerns on this embodiment. Is a diagram showing an inspection image of the data processing system according to the present embodiment. Is a diagram showing an inference model of a data processing system. Is a diagram showing inference annotations of a data processing system. Is a diagram showing a modified model of a data processing system. Is a diagram showing a teacher model of the data processing system according to the present embodiment. Is a diagram showing an inference model of a data processing system. Is a diagram showing a modified model of a data processing system. It is a figure which shows the list which a data processing apparatus has.
  • FIG. 1 is a diagram showing a data processing system 100 according to the present embodiment.
  • FIG. 2 is a block diagram showing a data processing device 2 included in the data processing system 100 according to the present embodiment.
  • FIG. 3A is a diagram showing a target image 50 of the data processing system 100.
  • FIG. 3B is a diagram showing a teacher model 54 of the data processing system 100.
  • FIG. 4 is a diagram showing a list 60 included in the data processing device 2.
  • FIG. 5A is a diagram showing an inspection image 70 of the data processing system 100.
  • FIG. 5B is a diagram showing an inference model 74 of the data processing system 100.
  • FIG. 6 is a diagram showing a list 60 included in the data processing device 2.
  • FIG. 7A is a diagram showing an inspection image 80 of the data processing system 100.
  • FIG. 7B is a diagram showing the inference annotation 82 of the data processing system 100.
  • FIG. 7C is a diagram showing a modified model 86 of the data processing system 100.
  • FIG. 8 is a diagram showing a list 60 included in the data processing device 2.
  • the data processing system 100 includes a data processing device 2, a server 4, and a cloud 6.
  • the data processing device 2, the server 4, and the cloud 6 are connected to each other by a line L.
  • the line L is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network).
  • the line L is not limited to either a wireless line or a wired line.
  • FIG. 2 is a diagram showing a data processing device 2 according to the present embodiment.
  • the data processing device 2 includes a control unit 20, a data input unit 22, a storage unit 24, an image input unit 26, a processing unit 28, and an output unit 38.
  • the processing unit 28 includes a granting unit 30, an inference unit 32, a support unit 34, and a correction unit 36.
  • the storage unit 24 or the processing unit 28 may be arranged in the server 4 or the cloud 6.
  • the data processing device 2 is, for example, a personal computer.
  • a security camera is connected to the personal computer, for example.
  • the security camera captures a landscape or the like and outputs image data.
  • the security camera is used for crime prevention, for example.
  • the personal computer is used for purposes such as analyzing image data acquired from a security camera, extracting an image of a cat from landscape images, and discriminating a specific cat.
  • the control unit 20 controls the operation of the data processing device 2.
  • the control unit 20 includes a CPU (Central Processing Unit) as an example.
  • the data input unit 22 inputs the target image 50.
  • the data input unit 22 may acquire the target image 50 from the server 4 or the cloud 6. Further, the data input unit 22 may acquire the target image 50 from the outside via the Internet by operating the data processing device 2.
  • the target image 50 is an image of a cat as an example.
  • the target image 50 may be an image of a plurality of cats belonging to the same type of category.
  • the giving unit 30 gives the teacher annotation 52 to the target image 50.
  • the teacher annotation 52 identifies the target image 50 by trimming the target image 50 with a rectangular frame line, as shown by a broken line in FIG. 3B.
  • the teacher annotation 52 includes coordinate data (x 1 , y 1 ) of at least two points that specify the trimming range of the target image 50, and coordinate data (x 2 , y 2 ).
  • the teacher annotation 52 is given by trimming described as a bounding box.
  • All of the cat skeleton of the target image 50 is included in the trimming range specified by the coordinate data (x 1 , y 1 ) of the two points and the coordinate data (x 2 , y 2 ). Therefore, the user can visually recognize the cat from the landscape image.
  • the teacher annotation 52 is added to the target image 50, so that the data processing system 100 is used for, for example, identifying a criminal.
  • the storage unit 24 includes a storage device and stores data and computer programs. Specifically, the storage unit 24 includes a main storage device such as a semiconductor memory and an auxiliary storage device such as a semiconductor memory and / or a hard disk drive.
  • a main storage device such as a semiconductor memory
  • an auxiliary storage device such as a semiconductor memory and / or a hard disk drive.
  • the storage unit 24 may include removable media.
  • the processor of the control unit 20 executes a computer program stored in the storage device of the storage unit 24 to control each component of the data processing device 2.
  • the storage unit 24 stores the teacher model 54 to which the teacher annotation 52 that identifies the target image 50 is added to the target image 50.
  • the storage unit 24 further stores the list 60 as shown in FIG.
  • the storage unit 24 registers a plurality of teacher models 54 in the list 60.
  • a plurality of teacher models 54 such as the teacher model 54 (1) and the teacher model 54 (2) can be registered in the list 60.
  • the teacher model 54 (1) is associated with the two-point coordinate data (x 1 , y 1 ) and the teacher annotation 52 of the coordinate data (x 2 , y 2 ).
  • the teacher model 54 (2) is associated with the coordinate data (x 3 , y 3 ) of the two points and the teacher annotation 52 of the coordinate data (x 4 , y 4 ).
  • the inference unit 32 trims the inspection image 70 input from the image input unit 26, and plays a role of, for example, a sample for identifying a cat from a landscape image or a teacher.
  • the image input unit 26 inputs the inspection image 70.
  • the image input unit 26 may acquire the inspection image 70 from the server 4 or the cloud 6. Further, the user may operate the data processing device 2 so that the image input unit 26 may acquire the inspection image 70 from the outside via the Internet.
  • the user may operate the data processing device 2 so that the image input unit 26 may acquire the inspection image 70 from, for example, the security camera.
  • the security camera captures an image of the landscape.
  • the landscape image partially includes the inspection image 70.
  • the inspection image 70 may be an image of a cat belonging to the same category as the target image 50.
  • the inference unit 32 adds an inference annotation 72 that infers the inspection image 70 to the inspection image 70 belonging to the same category as the target image 50 based on the teacher model 54, as shown in FIG. 5B. Give.
  • the inference unit 32 refers to the list 60 shown in FIG.
  • the inference unit 32 infers the inference annotation 72 that trims the inspection image 70 with reference to the plurality of teacher models 54 registered in the list 60.
  • the inference unit 32 adds the inference annotation 72 to the inspection image 70.
  • the inference annotation 72 identifies the inspection image 70 by trimming the inspection image 70 with a rectangular frame as shown by a broken line in FIG. 5B.
  • the inference annotation 72 includes coordinate data (x 5 , y 5 ) of at least two points that specify the trimming range of the inspection image 70, and coordinate data (x 6 , y 6 ).
  • the inference annotation 72 is given by trimming described as a bounding box.
  • FIG. 5B is an example in which the reasoning unit 32 correctly infers and trims the range of the cat skeleton which is the inspection image 70.
  • All of the cat skeleton of the inspection image 70 is included in the trimming range specified by the coordinate data (x 5 , y 5 ) at two points and the coordinate data (x 6 , y 6 ). Therefore, the user can visually recognize the cat from the landscape image.
  • the storage unit 24 stores the inference model 74 to which the inference annotation 72 that identifies the inspection image 70 is added to the inspection image 70.
  • the storage unit 24 further registers a plurality of inference models 74 in the list 60, as shown in FIG. In addition to the teacher model 54 (1) and the teacher model 54 (2), the inference model 74 (1) and the inference model 74 (2) are registered in the list 60. The storage unit 24 can further register a plurality of inference models 74.
  • the inference model 74 (1) is associated with the two-point coordinate data (x 5 , y 5 ) and the inference annotation 72 of the coordinate data (x 6 , y 6 ).
  • the inference model 74 (2) is associated with the coordinate data (x 7 , y 7 ) of the two points and the inference annotation 72 of the coordinate data (x 8 , y 8 ).
  • FIG. 7A is a diagram showing an inspection image 80.
  • FIG. 7B is a diagram showing the inference annotation 82 of the inspection image 80.
  • FIG. 7C is a diagram showing the modified annotation 84 of the inspection image 80.
  • the inference unit 32 Based on the teacher model 54, the inference unit 32 adds an inference annotation 82 for inferring the inspection image 80 to the inspection image 80 belonging to the same category as the target image 50 shown in FIG. 7A, as shown in FIG. 7B. ..
  • the inference unit 32 refers to the list 60 shown in FIG.
  • the inference unit 32 infers the inference annotation 82 that trims the inspection image 80 with reference to the plurality of teacher models 54 registered in the list 60.
  • the inference unit 32 adds the inference annotation 82 to the inspection image 80.
  • the inference annotation 82 identifies the inspection image 80 by trimming the inspection image 80 with a rectangular frame as shown by a broken line in FIG. 7B.
  • the inference annotation 82 includes coordinate data (x 9 , y 9 ) at at least two points that specify the trimming range of the inspection image 80, and coordinate data (x 10 , y 10 ).
  • FIG. 7B is an example in which the reasoning unit 32 erroneously infers and trims the range of the cat skeleton which is the inspection image 70.
  • a part of the cat skeleton of the inspection image 70 is included in the trimming range specified by the coordinate data (x 9 , y 9 ) and the coordinate data (x 10 , y 10 ) at two points. Therefore, the user may not be able to recognize what the image specified in the cropping range is.
  • the support unit 34 shown in FIG. 2 supports a human being to determine whether or not the inference annotation 82 specifies the inspection image 80.
  • the support unit 34 acquires a user's instruction using a mouse or keyboard (not shown).
  • the correction unit 36 shown in FIG. 2 adds the correction annotation 84 to the inference annotation 82 so that the inspection image 80 is specified, and generates the correction model 86.
  • the correction annotation 84 includes coordinate data (x 11 , y 11 ) of at least two points that specify the trimming range of the inspection image 80, and coordinate data (x 12 , y 12 ).
  • the modification annotation 84 is given by trimming described as a bounding box.
  • the user recognizes that the inference annotation 82 shown in FIG. 7B is an erroneous inference.
  • the user modifies the trimming of the inspection image 80 included in the landscape image so that the inspection image 80 can be identified, for example, using a mouse or a keyboard, as shown in FIG. 7C.
  • the correction unit 36 generates the correction annotation 84 based on the trimming corrected by the user, and generates the correction model 86.
  • the storage unit 24 stores the modified model 86 in which the modified annotation 84 that identifies the inspection image 80 is added to the inspection image 80.
  • the storage unit 24 further registers a plurality of modified models 86 in the list 60, as shown in FIG. Listing 60 shows modified model 86 (1), modified model 86 (2), in addition to teacher model 54 (1), teacher model 54 (2), inference model 74 (1), and inference model 74 (2). , And the modified model 86 (3) are registered.
  • the storage unit 24 can further register a plurality of modified models 86.
  • Modified model 86 (1) for example, the user, the coordinate data of the two points contained in the inference annotation 82 (x 9, y 9) and the coordinate data (x 10, y 10) of the coordinate data (x 10, Recognize that the inference of y 10 ) is incorrect.
  • the user instructs the coordinate data (x 11 , y 11 ).
  • the modification annotation 84 is associated with the coordinate data (x 11 , y 11 ) in addition to the coordinate data (x 9 , y 9 ) and the coordinate data (x 10 , y 10 ) at the two points.
  • the modified model 86 (1) may include a difference annotation between the coordinate data (x 10 , y 10 ) and the coordinate data (x 11 , y 11 ).
  • Correction models 86 (2) for example, the user, the coordinate data of the two points contained in the inference annotation 82 (x 9, y 9) and the coordinate data (x 10, y 10) of the coordinate data (x 9, Recognize that the inference of y 9 ) is incorrect.
  • the user instructs the coordinate data (x 11 , y 11 ).
  • the modification annotation 84 is associated with the modification annotation 84 of the coordinate data (x 11 , y 11 ) in addition to the inference annotation 82 of the coordinate data (x 9 , y 9 ) and the coordinate data (x 10 , y 10 ) of the two points. Be done.
  • the modified model 86 (1) may include a difference annotation between the coordinate data (x 9 , y 9 ) and the coordinate data (x 11 , y 11 ).
  • the modified model 86 (3) has two-point coordinate data (x 15 , y 15 ), and (x 16 , y 16 ) has two-point coordinate data (x 15 , y 15 ), (x 16 , y 16 ). Both inferences are incorrect, meaning that both the two-point coordinate data (x 15 , y 15 ) and (x 16 , y 16 ) have been corrected.
  • the modified model 86 (3) in addition to the inference annotation 82 including the two-point coordinate data (x 15 , y 15 ) and (x 16 , y 16 ), the two-point coordinate data (x 17 , y 17 ), It is associated with the modification annotation 84 of (x 18 , y 18 ).
  • the modified model 86 (3) includes coordinate data (x 17 , y 17 ) or coordinate data (x 18 , y 18 ), coordinate data (x 15 , y 15 ), and coordinate data (x 16 , y 16 ). It may include a difference annotation.
  • the output unit 38 outputs the teacher model 54, the inference model 74, or the modified model 86.
  • the correction annotation 84 by adding the correction annotation 84 to the inference annotation 82 that does not preferably specify the inspection image 80, it is preferable to modify the inference annotation 82 that preferably specifies the inspection image 80. Can be reduced to.
  • FIGS. 9 and 10 are flowcharts showing the operation of the data processing system 100 according to the present embodiment. As shown in FIGS. 9 and 10, the process includes steps S10 to S32. Specifically, it is as follows.
  • step S10 the learning data including the target image 50 is input.
  • step S12 the learning data including the target image 50 is input. The process proceeds to step S12.
  • step S12 the granting unit 30 grants the teacher annotation 52 manually input by the user to the target image 50.
  • the granting unit 30 generates the teacher model 54.
  • the process proceeds to step S14.
  • step S14 the storage unit 24 stores the teacher model 54. The process proceeds to step S16.
  • step S16 the image input unit 26 inputs the inspection image 70.
  • step S18 the image input unit 26 inputs the inspection image 70.
  • step S18 the inference unit 32 compares the teacher model 54 with the inspection image 70. The process proceeds to step S20.
  • step S20 the inference unit 32 outputs the inference annotation 72 or the inference annotation 82 given to the inspection image 70 with reference to the teacher model 54. The process proceeds to step S22.
  • step S22 the user determines whether or not the inference annotation 72 or the inference annotation 82 is correct. The process proceeds to step S24.
  • step S22 If the inference annotation 72 is determined to be correct in step S22 (Yes in step S22), the process proceeds to step S24 shown in FIG. If it is determined in step S22 that the inference annotation 82 is incorrect (No in step S22), the process proceeds to step S28 shown in FIG.
  • step S24 the inference unit 32 outputs the inspection image 70 and the inference annotation 72.
  • the process proceeds to step S26.
  • step S26 the storage unit 24 adds the inference model 74 to the teacher model 54 and stores it.
  • the process proceeds to step S16 shown in FIG.
  • step S28 the user inputs the correction annotation 84 in addition to the inference annotation 82 by the support unit 34.
  • the process proceeds to step S30.
  • step S30 the correction unit 36 outputs the correction model 86.
  • the process proceeds to step S32.
  • step S32 the storage unit 24 additionally stores the modified model 86 in addition to the teacher model 54 and the inference model 74.
  • the process proceeds to step S16 shown in FIG.
  • FIG. 11A is a diagram showing an inspection image 90 of the data processing system 100 according to the second embodiment.
  • FIG. 11B is a diagram showing an inference model 93 of the data processing system 100.
  • FIG. 11C is a diagram showing the inference annotation 94 of the data processing system 100.
  • FIG. 11D is a diagram showing a modified model 98 of the data processing system 100.
  • the teacher annotation 52, the inference annotation 92, the inference annotation 94, and the modification annotation 96 are a set of pixel data that specifies the masking range of the target image 50 and the inspection image 90.
  • the teacher annotation 52, the inference annotation 92, the inference annotation 94, and the modification annotation 96 are given by masking with pixels described as semantic segmentation.
  • the image input unit 26 shown in FIG. 2 inputs the inspection image 90 shown in FIG. 11A.
  • the inference unit 32 adds the inference annotation 92 to the inspection image 90 as shown in FIG. 11B with reference to the teacher model 54 (FIG. 4) stored in the storage unit 24.
  • the inference unit 32 outputs an inference model 93 in which the inference annotation 92 is added to the inspection image 90.
  • the inference annotation 92 is a set of pixel data (pixel position data, pixel color data) arranged in the masking range.
  • FIG. 11B is an example in which the inference by the inference unit 32 is correct and the range of the cat's skeleton is accurately masked.
  • the inference unit 32 adds the inference annotation 94 to the inspection image 90 as shown in FIG. 11C with reference to the teacher model 54 stored in the storage unit 24.
  • FIG. 11C is an example in which the inference by the inference unit 32 is incorrect and the range of the cat's skeleton is not accurately masked.
  • the support unit 34 supports a human being to determine whether or not the inference annotation 94 specifies the range of the inspection image 90.
  • the correction unit 36 shown in FIG. 2 adds the correction annotation 96 to the inference annotation 94 so as to specify the range of the inspection image 90, and generates the correction model 98.
  • the correction annotation 96 includes a set of pixel data that specifies the trimming range of the inspection image 90.
  • the user recognizes that the inference annotation 94 shown in FIG. 11C is an erroneous inference.
  • the user modifies the trimming so that all the pixels included in the frame of the inspection image 90 included in the landscape image are selected, as shown in FIG. 11D.
  • the correction unit 36 generates the correction annotation 96 based on the trimming corrected by the user, and generates the correction model 98.
  • the storage unit 24 stores the inference model 93 and the modification model 98 to which the modification annotation 96 that identifies the inspection image 90 is added.
  • the storage unit 24 further registers a plurality of inference models 93 and a plurality of modified models 98 in the list 60, as shown in FIG. The duplicate description of Listing 60 will be omitted.
  • the output unit 38 outputs the teacher model 54, the inference model 93, or the modified model 98.
  • the inspection image 90 is preferably added by adding the correction annotation 96 to the inference annotation 94 that does not preferably specify the inspection image 90. It is possible to preferably reduce the modification work for the specified inference annotation 94.
  • FIG. 11A is a diagram showing a teacher model 107 of the data processing system 100 according to the present embodiment.
  • FIG. 11B is a diagram showing an inference model 111 of the data processing system 100.
  • FIG. 11C is a diagram showing a modified model 117 of the data processing system 100.
  • FIG. 12 is a diagram showing a list 130 included in the data processing device 2.
  • the third embodiment is an example of using the data processing system 100 as a translation system.
  • the target image 101 is the first sentence 102 created in the first language.
  • the teacher annotation 104 is a second phrase 106 that constitutes the second sentence 105 created in the second language, which is given by a human to each first phrase 103 that constitutes the first sentence 102.
  • the inspection image 108 is the third sentence 109 created in the first language.
  • the inference annotation 110 is the fourth phrase 114 constituting the fourth sentence 113, which is inferred for each third phrase 112 constituting the third sentence 109 based on the teacher model 107.
  • the correction annotation 115 is a human being by the correction unit 36 when the support unit 34 determines that the fourth phrase 114 inferred corresponding to the third phrase 112 is incorrect.
  • the fifth phrase 116 which is a translation of the third phrase 112 into a second language.
  • the first language is Japanese as an example.
  • the target image 101 is a character image of the first sentence 102 “Technology is rapidly developing in this country” created in Japanese.
  • the first sentence 102 is composed of the first phrase 103 "in this country”, “technology”, “rapidly”, and "developing.”
  • the second language is English as an example.
  • the second sentence 105 is a character image of "Technology is rapidly developing in this country.” Created by translating the first sentence 102 into English.
  • the second sentence 105 is composed of the second phrase 106 "Technology”, “rapidly”, “is developing”, and "in this country”.
  • the teacher annotation 104 is given by the giving unit 30.
  • the storage unit 24 stores the teacher model 107 to which the teacher annotation 104 that identifies the target image 101 is added to the target image 101.
  • the storage unit 24 further stores the list 130 as shown in FIG.
  • the storage unit 24 registers the teacher model 107 (1) in the list 130.
  • a plurality of teacher models 107 can be registered in the list 130.
  • the image input unit 26 inputs the inspection image 108.
  • the inspection image 108 is a character image of the third sentence 109 “Aging is rapidly developing in this country” prepared in Japanese.
  • the third sentence 109 is composed of the third phrase 112 "in this country”, "aging”, “rapidly”, and "developing.”
  • the inference unit 32 Based on the teacher model 107, the inference unit 32 translates the third phrase 112 "In this country, aging is progressing rapidly" into the fourth sentence 113 "Aging is rapidly advance in this country.” To do.
  • the third sentence 109 is in Japanese as an example.
  • the fourth sentence 113 is in English as an example.
  • the inference unit 32 refers to the list 130 shown in FIG.
  • the inference unit 32 adds the inference annotation 110 inferred from the inspection image 108 to the inspection image 108 with reference to the plurality of teacher models 107 registered in the list 130.
  • the inference annotation 110 is translated for each of the third phrases 112 "in this country”, “aging”, “rapidly”, and “developing”, which constitute the third sentence 109, based on the teacher model 107.
  • the fourth phrase 114 "Aging”, “rapidly”, “is advance”, and "in this country” constituting the fourth sentence 113.
  • the storage unit 24 stores the inference model 111 to which the inference annotation 110 that identifies the inspection image 108 is added to the inspection image 108.
  • the storage unit 24 further registers the inference model 111 in the list 130 as shown in FIG. In the list 130, the inference model 111 (1) is registered in addition to the teacher model 107 (1).
  • the storage unit 24 can further register a plurality of inference models 111.
  • the fact that the inference model 111 (1) is registered in the list 130 of the storage unit 24 means that the translation by the inference unit 32 was correct.
  • the inference unit 32 based on the teacher model 107, puts the third phrase 112 "In this country, the aging is progressing rapidly" in the fourth sentence 113. Translate to "Aging is phrasely developing in this country.”
  • the fourth phrase 114 "developing”, which is a translation of the third phrase 112 "advancing" into English, may be incorrect.
  • the support unit 34 shown in FIG. 2 assists a human being in determining whether or not the inference annotation 110 is correct as a translation of the inspection image 108 into English.
  • the correction unit 36 shown in FIG. 2 adds the correction annotation 115 to the inference annotation 110 to generate the correction model 117 so that the mistranslation of the inspection image 108 is corrected.
  • the modification annotation 115 includes the fifth phrase 116 “advancing” which is a modification of the fourth phrase 114 “developing”.
  • the user recognizes that the inference annotation 110 shown in FIG. 12C is an erroneous inference.
  • the user modifies the fourth phrase 114 "developing" to the fifth phrase 116 "advancing", for example, using a mouse or keyboard, as shown in FIG. 12C.
  • the correction unit 36 generates the correction annotation 115 and the correction model 117 based on the user's correction.
  • the storage unit 24 stores the correction model 117 in which the correction annotation 115 that corrects the mistranslation of the inference annotation 110 is added to the inspection image 108.
  • the storage unit 24 further registers a plurality of modified models 117 in the list 130, as shown in FIG. In Listing 130, a modified model 117 (1) is registered in addition to the teacher model 107 (1) and the inference model 111 (1).
  • the storage unit 24 can further register a plurality of modified models 117.
  • the correction unit 36 may generate a difference annotation 118 indicating a difference between the fourth phrase 114 “developing” and the fifth phrase 116 “advancing” based on the user's correction.
  • the storage unit 24 may include the difference annotation 118 in the modified model 117 registered in the list 130.
  • the output unit 38 outputs the teacher model 107, the inference model 111, or the modified model 117.
  • the present invention can be used in the fields of data processing systems and data processing methods.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
PCT/JP2020/027797 2019-07-25 2020-07-17 データ処理システム、およびデータ処理方法 Ceased WO2021015117A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/597,804 US12002191B2 (en) 2019-07-25 2020-07-17 Data processing system and data processing method
JP2021533997A JP7238992B2 (ja) 2019-07-25 2020-07-17 データ処理システム、およびデータ処理方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019136942 2019-07-25
JP2019-136942 2019-07-25

Publications (1)

Publication Number Publication Date
WO2021015117A1 true WO2021015117A1 (ja) 2021-01-28

Family

ID=74192860

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/027797 Ceased WO2021015117A1 (ja) 2019-07-25 2020-07-17 データ処理システム、およびデータ処理方法

Country Status (3)

Country Link
US (1) US12002191B2 (https=)
JP (1) JP7238992B2 (https=)
WO (1) WO2021015117A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024006945A (ja) * 2022-06-30 2024-01-17 ベイジン バイドゥ ネットコム サイエンス テクノロジー カンパニー リミテッド モデル更新方法、装置、電子デバイス及び記憶媒体
JP7601293B1 (ja) * 2023-12-15 2024-12-17 コニカミノルタ株式会社 アノテーション支援装置、および制御プログラム

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113282548A (zh) * 2021-05-20 2021-08-20 Oppo广东移动通信有限公司 文件保存方法、装置、电子设备以及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09223146A (ja) * 1996-02-16 1997-08-26 Toshiba Corp 翻訳方法
JP2006135796A (ja) * 2004-11-08 2006-05-25 Fuji Photo Film Co Ltd 画像処理装置及び画像処理方法
JP2009245404A (ja) * 2008-04-01 2009-10-22 Fujifilm Corp 画像処理装置および方法並びにプログラム
JP2009268085A (ja) * 2008-03-31 2009-11-12 Fujifilm Corp 画像トリミング装置およびプログラム
JP2018195107A (ja) * 2017-05-18 2018-12-06 ファナック株式会社 画像処理システム

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6435740B2 (ja) 2014-09-22 2018-12-12 日本電気株式会社 データ処理システム、データ処理方法およびデータ処理プログラム
US11449686B1 (en) * 2019-07-09 2022-09-20 Amazon Technologies, Inc. Automated evaluation and selection of machine translation protocols

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09223146A (ja) * 1996-02-16 1997-08-26 Toshiba Corp 翻訳方法
JP2006135796A (ja) * 2004-11-08 2006-05-25 Fuji Photo Film Co Ltd 画像処理装置及び画像処理方法
JP2009268085A (ja) * 2008-03-31 2009-11-12 Fujifilm Corp 画像トリミング装置およびプログラム
JP2009245404A (ja) * 2008-04-01 2009-10-22 Fujifilm Corp 画像処理装置および方法並びにプログラム
JP2018195107A (ja) * 2017-05-18 2018-12-06 ファナック株式会社 画像処理システム

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024006945A (ja) * 2022-06-30 2024-01-17 ベイジン バイドゥ ネットコム サイエンス テクノロジー カンパニー リミテッド モデル更新方法、装置、電子デバイス及び記憶媒体
JP7443649B2 (ja) 2022-06-30 2024-03-06 ベイジン バイドゥ ネットコム サイエンス テクノロジー カンパニー リミテッド モデル更新方法、装置、電子デバイス及び記憶媒体
JP7601293B1 (ja) * 2023-12-15 2024-12-17 コニカミノルタ株式会社 アノテーション支援装置、および制御プログラム
WO2025126445A1 (ja) * 2023-12-15 2025-06-19 コニカミノルタ株式会社 アノテーション支援装置、および制御プログラム

Also Published As

Publication number Publication date
JPWO2021015117A1 (https=) 2021-01-28
US20220254003A1 (en) 2022-08-11
JP7238992B2 (ja) 2023-03-14
US12002191B2 (en) 2024-06-04

Similar Documents

Publication Publication Date Title
Bull et al. Automatic segmentation of sign language into subtitle-units
JP6596511B2 (ja) デジタルグラフィックノベルの自動翻訳
Qi et al. Cogcom: Train large vision-language models diving into details through chain of manipulations
US12248794B2 (en) Self-supervised system for learning a user interface language
CN111656453B (zh) 用于信息提取的层次实体识别和语义建模框架
WO2021015117A1 (ja) データ処理システム、およびデータ処理方法
JP2006276915A (ja) 翻訳処理方法、文書翻訳装置およびプログラム
CN111767883A (zh) 一种题目批改方法及装置
US20220392127A1 (en) Image annotation method
Arbaz et al. GenFlowchart: parsing and understanding flowchart using generative AI
CN119398063A (zh) 文本翻译方法、装置、电子设备以及存储介质
Xia et al. Bootstrapping grounded chain-of-thought in multimodal llms for data-efficient model adaptation
JPWO2021015117A5 (https=)
CN114580346B (zh) 结合rpa和ai的信息生成方法、装置、电子设备及存储介质
CN111813978A (zh) 一种图像描述模型的生成方法、生成装置和存储介质
JP2020198023A (ja) 情報処理装置、方法及びプログラム
CN117910483A (zh) 翻译方法、装置、电子设备和存储介质
US11335108B2 (en) System and method to recognise characters from an image
JP6235373B2 (ja) 言語解析方法及びシステム
WO2021131127A1 (ja) 識別情報付与装置、識別情報付与方法、及びプログラム
JPWO2021107142A5 (https=)
JPH09325673A (ja) 翻訳文添削支援装置
JP7732518B2 (ja) 画像処理装置、方法およびプログラム
US20260064949A1 (en) Augmented language model for artifact extraction
CN119904853A (zh) 一种准确抽取医疗诊断材料信息方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20843150

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021533997

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20843150

Country of ref document: EP

Kind code of ref document: A1