US11308352B2 - Apparatus for processing labeled data to be used in learning of discriminator, method of controlling the apparatus, and non-transitory computer-readable recording medium - Google Patents

Apparatus for processing labeled data to be used in learning of discriminator, method of controlling the apparatus, and non-transitory computer-readable recording medium Download PDF

Info

Publication number
US11308352B2
US11308352B2 US16/798,602 US202016798602A US11308352B2 US 11308352 B2 US11308352 B2 US 11308352B2 US 202016798602 A US202016798602 A US 202016798602A US 11308352 B2 US11308352 B2 US 11308352B2
Authority
US
United States
Prior art keywords
partial region
data
local data
discriminator
labeling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US16/798,602
Other languages
English (en)
Other versions
US20200279132A1 (en
Inventor
Atsushi Nogami
Masakazu Matsugu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Assigned to CANON KABUSHIKI KAISHA reassignment CANON KABUSHIKI KAISHA ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MATSUGU, MASAKAZU, NOGAMI, ATSUSHI
Publication of US20200279132A1 publication Critical patent/US20200279132A1/en
Priority to US17/699,305 priority Critical patent/US11741683B2/en
Application granted granted Critical
Publication of US11308352B2 publication Critical patent/US11308352B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G06K9/6234—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G06V10/235—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition based on user input or interaction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2132—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on discrimination criteria, e.g. discriminant analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G06F18/2178—Validation; Performance evaluation; Active pattern learning techniques based on feedback of a supervisor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06K9/00671—
    • G06K9/4604—
    • G06K9/6218—
    • G06K9/6256—
    • G06K9/6263—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/778—Active pattern-learning, e.g. online learning of image or video features
    • G06V10/7784—Active pattern-learning, e.g. online learning of image or video features based on feedback from supervisors
    • G06V10/7788—Active pattern-learning, e.g. online learning of image or video features based on feedback from supervisors the supervisor being a human, e.g. interactive learning with a human teacher
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes

Definitions

  • the present invention relates to the generation of labeled data to be used in learning of a discriminator.
  • Labeled data is necessary to perform learning of a discriminator by machine learning.
  • This labeled data can also be called correct answer data indicating a correct answer of a discrimination result obtained by a discriminator, or teacher data for learning.
  • a work for preparing the labeled data requires a high work cost, so a technique for reducing this cost has been examined.
  • Japanese Patent Laid-Open No. 2017-167834 discloses a technique that selects data effective for learning from unlabeled data by using a discriminator having learned with the existing labeled data. This method discriminates unlabeled data by using a learned discriminator, and selects data discriminated near the discrimination boundary as data effective for learning. This data near the discrimination boundary is data which the existing discriminators cannot easily discriminate. Effective learning can be performed by labeling this data near the discrimination boundary.
  • an apparatus comprising: an obtaining unit configured to obtain target data as a result of discrimination of each portion of input data performed by a discriminator having learned in advance by using existing labeled data; a setting unit configured to set each portion of the target data, which is effective for additional learning of the discriminator, as local data; a determining unit configured to determine not less than one partial region of the target data, which accepts labeling by a user, based on a distribution of the set local data in the target data; and a display control unit configured to cause a display unit to display the determined not less than one partial region.
  • FIG. 1 is a view showing an arrangement example of an information processing apparatus according to an embodiment
  • FIG. 2 is a view for explaining labeling target data according to the first embodiment
  • FIG. 3 is a flowchart showing a procedure to be performed by the information processing apparatus according to the embodiment.
  • FIG. 4A is a view for explaining local data and partial regions according to the first embodiment
  • FIG. 4B is a view for explaining local data and partial regions according to the first embodiment
  • FIG. 4C is a view for explaining local data and partial regions according to the first embodiment
  • FIG. 5A is a view for explaining a method of setting partial regions by clustering of local data according to the first embodiment
  • FIG. 5B is a view for explaining a method of setting partial regions by clustering of local data according to the first embodiment
  • FIG. 6 is a view for explaining a display unit and a label input unit according to the first embodiment
  • FIG. 7 is a view for explaining input of a crack label according to the first embodiment
  • FIG. 8 is a view for explaining automatic range extension of a partial region according to the first embodiment
  • FIG. 9A is a view for explaining local data and partial regions according to the second embodiment.
  • FIG. 9B is a view for explaining local data and partial regions according to the second embodiment.
  • FIG. 9C is a view for explaining local data and partial regions according to the second embodiment.
  • FIG. 10 is a view for explaining local data and partial regions according to the third embodiment.
  • FIG. 11A is a view for explaining local data and a partial region according to the fourth embodiment.
  • FIG. 11B is a view for explaining local data and a partial region according to the fourth embodiment.
  • FIG. 12 is a view for explaining local data and partial regions according to the fifth embodiment.
  • a partial region effective for learning is selected from unlabeled data and presented as labeling target data.
  • local data requiring labeling is determined based on a given criterion, a partial region containing the local data and having a range wider than the local data is determined, and labeling is requested for this partial region.
  • data as a target of labeling is an image.
  • local data is a pixel of the image
  • a partial region is a partial image region containing the pixel as the local data.
  • a partial region like this is selected as labeling target data, and a label inputter (to be referred to as a user hereinafter) labels each pixel in the partial region.
  • This embodiment forms labeled data (teacher data) for performing learning of a discriminator that discriminates whether a target exists for each pixel of an image. For example, in an appearance test to be conducted on a product in a production process, learning is performed on a discriminator for detecting a scratch or a defect of the product. Such a discriminator is created by learning a discriminator for detecting a scratch or a defect for each pixel by using a label indicating the position of a scratch or defect having an arbitrary shape. As another example, in inspection of a concrete structure such as a bridge, a dam, a tunnel, or a building, a defect on a concrete wall surface is detected.
  • a discriminator for inspecting a concrete wall surface performs learning by using data labeled to a pixel of a detect in an image of the concrete wall surface. When detecting a defect, the discriminator discriminates whether a defect exists for each pixel.
  • An information processing apparatus 100 and a related arrangement can be implemented by executing software (a program) obtained from a network or various information recording media by using a computer including a CPU, a memory, a storage device, an input/output device, a bus, a display device, and the like.
  • a computer including a CPU, a memory, a storage device, an input/output device, a bus, a display device, and the like.
  • the computer it is possible to use a general-purpose computer or hardware designed to be optimum for the software.
  • the information processing apparatus 100 includes a data storage unit 101 , a local portion setting unit 102 , a partial region determination unit 103 , a display control unit 104 , a display unit 105 , a label input unit 106 , a discriminator storage unit 107 , and a learning unit 108 .
  • a user 110 labels a presented partial region by using the display unit 105 and the label input unit 106 .
  • these constituent elements are not always essential elements of this embodiment.
  • the display unit 105 and the label input unit 106 may also be installed outside the information processing apparatus 100 .
  • the data storage unit 101 is a storage for storing an unlabeled image and a labeled image, and is a storage device such as a hard disk. Images of concrete wall surfaces are stored in the data storage unit 101 according to this embodiment.
  • a high-resolution image in which 1 mm of a concrete wall surface is recorded in one pixel is necessary to observe a crack having a width of 1 mm or less.
  • an inspection image of a 10 m ⁇ 10 m concrete wall surface is a large-sized image having 10,000 pixels ⁇ 10,000 pixels. The number of pixels of an image having this size exceeds that of an ordinary camera.
  • a high-resolution image for inspection can be formed by a method of, for example, connecting images obtained by imaging different positions of a concrete wall surface.
  • FIG. 2 shows data obtained by pasting a captured image on a plan view of a bridge.
  • This plan view of FIG. 2 shows a slab 201 and piers 202 and 203 of the bridge, and it is assumed that images obtained by imaging the concrete wall surfaces of these members are pasted on them.
  • an image having 15,000 pixels ⁇ 5,000 pixels is pasted on the slab 201 having a size of 15 m ⁇ 5 m.
  • To perform learning of a discriminator for detecting a crack it is necessary to give a label indicating whether each pixel of the image as shown in FIG. 2 is a crack.
  • giving crack labels to all pixels in the large image as shown in FIG. 2 requires a very high work cost.
  • a partial image effective for learning is selected as a labeling target image from an image like this.
  • the local portion setting unit 102 loads a labeling target image from the data storage unit 101 , determines a portion effective for learning in the labeling target image data, and sets the portion as local data.
  • the partial region determination unit 103 forms partial region candidates based on the spatial distribution of the set local data. Then, the partial region determination unit 103 calculates the labeling priority of each partial region candidate, and selectively determines a partial region from the partial region candidates based on the priorities.
  • the display control unit 104 controls the display unit 105 to display the set partial region.
  • the display unit 105 is a display such as an LCD.
  • the label input unit 106 is an input device such as a keyboard, a mouse, or a touch panel display, and accepts input of labeling with respect to the partial region displayed on the display unit 105 .
  • the discriminator storage unit 107 stores a learned discriminator.
  • the learning unit 108 performs learning of a discriminator for crack detection or the like by using an image and label data stored in the data storage unit 101 , and stores the discriminator as a result of learning in the discriminator storage unit 107 .
  • FIG. 3 is a flowchart showing the procedure of processing to be performed by the information processing apparatus according to this embodiment.
  • step S 301 the local portion setting unit 102 loads a labeling target image from the data storage unit 101 .
  • image data to be loaded in step S 301 is not limited to one structure.
  • images of a plurality of structures may also be loaded. It is also possible to load a partial image of a structure, for example, load only an image of the slab 201 of the bridge shown in FIG. 2 .
  • the local portion setting unit 102 determines a portion effective for learning in the labeling target image data, and sets the portion as local data.
  • the portion effective for learning in the labeling target image data is a portion having a high labeling work efficiency when improving the performance of a discriminator by performing additional learning on the discriminator by using data labeled to that portion. That is, a portion that potentially improves the discrimination ability of a discriminator more with a less labeling work is the portion effective for learning.
  • An example is a partial image having a feature different from the existing labeled data (already learned learning data).
  • a portion of labeling target image data, which has low similarity to the existing labeled data is the portion effective for learning.
  • the first method a method of selecting a pixel that is hardly discriminated by a prelearned discriminator will be explained. Assume that in this method, a discriminator for detecting a crack has prelearned. This prelearned discriminator learns by the existing labeled data.
  • the existing labeled data is data obtained by giving a crack label to a pixel of an image different from the image data loaded in step S 301 , and is data obtained by giving a crack label to an image of another structure in advance.
  • This learned discriminator having learned by the existing labeled data is stored in the discriminator storage unit 107 .
  • a pixel which this learned discriminator hardly discriminates is determined as a pixel effective for learning, and set as local data.
  • the local portion setting unit 102 first loads the learned discriminator from the discriminator storage unit 107 , and executes crack detection on image data by using the learned discriminator. Then, the learned discriminator determines a hard-to-discriminate pixel by using the detection result. Since the discriminator outputs a crack detection result for each pixel of the image, a hard-to-discriminate pixel can be determined by selecting a pixel having a discrimination result near the discrimination boundary. More specifically, assuming that the discriminator outputs the crack probability of each pixel within the range of 0 to 1, a pixel for which the output is close to 0.5 is a hard-to-discriminate pixel.
  • the second method a method of using a plurality of learned discriminators will be explained.
  • a plurality of different discriminators have prelearned and are stored in the discriminator storage unit 107 .
  • the plurality of different discriminators are, for example, discriminators having learned by using different learning data sets.
  • the different learning data sets include concrete wall surface images of different structures and crack labels.
  • the discriminators having learned by using the different learning data sets have different detection characteristics.
  • the local portion setting unit 102 loads the plurality of learned models described above from the discriminator storage unit 107 , and performs a crack detection process on the image data by using the plurality of learned models. Consequently, a plurality of detection process results are obtained.
  • a pixel having a large detection result variation is determined as a portion that is hard to discriminate and effective for learning, and set as local data.
  • Various methods can be used to determine a detection result variation. For example, it is possible to select a pixel having a discrimination result variation by obtaining an entropy h n of a pixel n as follows:
  • h n - ⁇ i ⁇ V ⁇ ( y i ) C ⁇ log ⁇ ( V ⁇ ( y i ) C ) ( 1 )
  • i represents the number of discrimination classes.
  • V(y i ) indicates the number of discriminators having discriminated that the pixel n has a label y i
  • C is the number of discriminators.
  • the discrimination result variation of the plurality of discriminators is large.
  • a pixel for which the entropy h n calculated as described above has a predetermined value or more is set as local data.
  • the local data can be set by the following method.
  • FIG. 4A shows the way a pixel in an image 401 is selected as local data by the above method.
  • the image 401 is an image of the slab 201 shown in FIG. 2 , and points such as a pixel 411 indicate pixels selected as local data.
  • a plurality of local data can be selected from the image 401 . This is so because in the first method, a plurality of pixels exist within a predetermined range from the discrimination plane of the discrimination results, and, in the second method, a plurality of pixels for which the entropy has a predetermined value or more exist.
  • if there is no pixel determined as local data if there is no hard-to-discriminate pixel), it is possible to determine that there is no labeling target data, and terminate the whole process.
  • these local data setting methods can set importance to each local data.
  • the importance of each local data can be set based on the distance from the discrimination plane. In this case, the setting is so performed that the importance increases as the distance from the discrimination plane decreases.
  • the value of the entropy of each local data can directly be set as the importance. A method of using the importance of local data will be described later.
  • G ⁇ g 1 ,g 2 , . . . ,g n , . . . ,g N ⁇ (3)
  • (u n , v n ) indicates the image coordinates of the nth local data
  • g n is the importance of the nth local data.
  • step S 303 the partial region determination unit 103 forms partial region candidates based on the spatial distribution of local data.
  • FIG. 4B shows the way partial region candidates 421 , 422 , 423 , and 424 are formed from the local data in FIG. 4A .
  • a method of forming partial region candidates from the spatial distribution of the local data as described above will be explained below. Note that two types of methods will be explained as the method of forming partial region candidates in step S 303 .
  • the first method of forming partial region candidates is a method of forming partial region candidates based on the density of local data in a predetermined range.
  • an image is searched by a window having a preset size, and a region where the density (or the number) of local data contained in the window is equal to or larger than a predetermined value is extracted as a partial region candidate.
  • this process can extract a partial region where the density of local data is high.
  • the search window a plurality of windows having different sizes and aspect ratios are prepared, and a region where the density of local data is high is searched for by sequentially applying these windows. By thus performing searching by using a plurality of windows, it is possible to extract partial region candidates of rectangular regions having various sizes and aspect ratios.
  • the shape of the search window is not limited to a rectangular shape, and may also be another shape such as a circular shape.
  • a minimum search window size can be an image size necessary for learning.
  • a minimum search window size is a size having 256 ⁇ 256 pixels or more.
  • a search window having a size of 256 ⁇ 256 pixels or more a partial region including 256 ⁇ 256 pixels or more is extracted.
  • an image having a size of 256 ⁇ 256 pixels can be cut out from the partial region and used as learning data.
  • a region where the density of local data in the window is equal to or larger than a predetermined value is extracted as a partial region candidate
  • windows having different sizes may overlap each other, or adjacent windows may overlap each other depending on the search step widths of the windows. If overlapped regions are extracted as partial regions, inputting a label to the same image region a plurality of times is a waste in terms of labeling work. In this case, therefore, a partial region candidate is formed by leaving a representative window behind from the overlapped windows.
  • the second method of forming partial region candidates is a method of forming partial region candidates by clustering the distribution of local data.
  • a method such as mean shift clustering can be used.
  • the clustering method is not limited to mean shifting clustering and can be any clustering method.
  • Well-known techniques can be used as these clustering methods, so a detailed explanation thereof will be omitted.
  • FIG. 5A shows the result of clustering of the local data distribution shown in FIG. 4A .
  • a method of forming partial region candidates from the clustering result is performed as follows. First, a cluster in which the number of local data is equal to or smaller than a predetermined value is removed from clusters. Then, the range of partial region candidates is determined in each of the remaining clusters. The range of partial region candidates can be determined by setting a rectangular region having a range surrounding local data belonging to the cluster. As another method, a region having an arbitrary shape can be set as a partial region candidate based on the positions of local data belonging to the cluster.
  • FIG. 5B shows the way partial region candidates 521 , 522 , 523 , and 524 having arbitrary shapes are formed from the clusters shown in FIG. 5A based on the positions of local data.
  • step S 303 partial region candidates are formed by the methods as described above.
  • Processes in subsequent steps S 304 and S 305 are processes in which the partial region determination unit 103 calculates the priorities of partial region candidates, and selects M upper partial region candidates having high labeling priorities as partial regions. If the number of partial region candidates formed in step S 303 is 1 or very small, it is also possible to omit the processes in steps S 304 and S 305 , determine all partial region candidates as partial regions, and perform a labeling process (to be described later).
  • partial regions to be actually labeled are set by decreasing the number of partial region candidates, in order to perform efficient labeling.
  • One method of decreasing the number of partial region candidates is a method of selecting partial regions at random from the partial region candidates.
  • the processes in steps S 304 and S 305 are performed in order to select more effective labeled data by selecting a partial region candidate that is potentially highly effective for learning. Details of each process will be explained below.
  • step S 304 the partial region determination unit 103 calculates the priority of labeling for each of the M partial region candidates determined in step S 303 . Assume that a partial region candidate having a higher priority value is a region more effective for learning.
  • the priority can be calculated by, for example, the following method.
  • the priority can be set based on the density or number of local data contained in a partial region candidate. Letting s 1 be the priority of a partial region candidate r m based on the density of local data, s 1 can be represented by equation 4 below:
  • N(r m ) is the number of local data contained in the mth partial region candidate r m
  • A(r m ) is the area of the partial region candidate r m .
  • the importance is calculated for each local data.
  • the priority of a partial region candidate can also be calculated by using this importance. More specifically, a statistic such as the sum total or average of the importances of local data contained in the partial region candidate r m is calculated as the priority of the partial region candidate r m . Equation 5 indicates an example of calculating priority s 2 by using the sum total of importances g of local data contained in the partial region candidate r m .
  • Partial regions as labeling targets are desirably collected from various regions in order to form various kinds of learning data.
  • a method of acquiring various partial regions there is a method of preferentially selecting partial regions spaced apart from each other. This is based on the assumption that a concrete wall surface image of a structure is an appearance image in which near wall surfaces are similar and distant wall surfaces are different.
  • the appearances of the slab 201 and the piers 202 and 203 are different.
  • a partial region is selected from the slab 201
  • a next partial region is preferably selected from the pier 202 or 203 .
  • partial regions to be selected are combined such that their positions are spaced apart from each other. Note that this positional relationship can be a positional relationship on an image plane, and can also be a three-dimensional positional relationship in a structure.
  • the method of setting the priority based on the positional relationship can be performed as follows. First, a partial region candidate found to be most important by the priority s 1 or s 2 is selected as a first partial region from partial region candidates. Then, the distances between the first partial region and the remaining partial region candidates are calculated, and a partial region candidate farthest from the first partial region is selected as a second partial region. Furthermore, the distances between the remaining partial region candidates and the first and second partial regions are calculated, and a partial region candidate having the largest sum total of the distances from the first and second partial regions is selected as a third partial region. Fourth and subsequent partial regions are selected by repeating this processing. Consequently, the partial region candidates are sorted in the descending order of the distance between the positions.
  • Priority s 3 based on the positional relationship is set in each partial region candidate in accordance with the order of sorting (so that the numerical value decreases in the order of sorting). Note that when selecting a partial region from partial region candidates based on only the positional relationship, the M upper partial region candidates can also be selected as partial regions in the order of sorting.
  • the priority can be calculated by the method as described above, but the priority calculation method is not limited to this, and another calculation method can also be used as long as the labeling priority of a partial region candidate can be evaluated by the value.
  • the method of calculating the three kinds of priorities s 1 to s 3 has been explained, but one of these priorities can be calculated as priority S, or the priority S can be calculated as indicated by equation 6 below by combining priorities calculated by a plurality of methods:
  • the priority S can be obtained by integrating the weighted priorities.
  • the method of evaluating the diversity of partial region candidates based on their positional relationships has been explained, but the diversity evaluation method is not limited to this. For example, it is also possible to evaluate the similarity between image feature amounts of partial region candidates, and calculate the priorities or sort the partial region candidates so as to select various partial regions.
  • step S 305 the partial region determination unit 103 determines partial regions by selecting them from partial region candidates based on the priorities calculated in step S 304 .
  • the method of determining partial regions from partial region candidates based on the priorities it is possible to sort the partial region candidates in the order of priorities, and select the M upper partial region candidates as partial regions. It is also possible to select a partial region candidate representing priority equal to or larger than a predetermined threshold as a partial region without limiting the number of partial regions to be selected to M.
  • FIG. 4C shows the way partial regions 431 , 433 , and 434 are selected from the partial region candidates 421 , 422 , 423 , and 424 in FIG. 4B by the process in step S 305 .
  • a partial region candidate 432 indicated by the dotted lines is not selected as a partial region because the priority is low.
  • the display control unit 104 displays the determined partial regions on the display unit 105 .
  • the user labels the partial region by using the label input unit 106 .
  • the display unit 105 is a display and the label input unit 106 includes a keyboard and a mouse will be explained.
  • the present invention is not limited to this, and another arrangement may also be adopted.
  • the label input unit 106 can also be a pen tablet, or the display unit 105 and the label input unit 106 can also be integrated into a tablet with a touch panel. The operations of the display unit 105 and the label input unit 106 will be explained below.
  • FIG. 6 is a view for explaining a Graphical User Interface (GUI) to be displayed on the display unit 105 in order to perform labeling.
  • GUI Graphical User Interface
  • a window 600 of a GUI application displays an image 601 as a labeling target.
  • the image 601 is an image showing a part of the slab 201 of FIG. 2 in an enlarged scale.
  • the application window 600 of the GUI displays an image 602 of the whole slab 201 , and also displays a display range 603 of the image 601 by the dotted lines. This allows the user to easily recognize a part of the whole image of the slab 201 which he or she is viewing.
  • the image 602 of the whole slab 201 displays a partial region (for example, a solid-line rectangle 634 ) set by the above-described process.
  • the image 601 as a labeling target also shows a partial region 633 , and the user performs labeling in this image range.
  • the image 601 displays an image range including the partial region.
  • the image 601 shown in FIG. 6 displays the partial region 633 in the center.
  • the partial region 633 is displayed by the rectangular line in the image 601 of FIG. 6 , but the partial region display method is not limited to this. For example, it is also possible to recognizably display the partial region by covering an image region outside the rectangular range with a semitransparent mask. In addition, the partial region is presented as a range in which labeling is recommended, but it is also possible to allow the user to intentionally change the position and size of the partial region.
  • FIG. 6 shows triangular icons 611 or the like for changing the size of the partial region. The user can, for example, extend the partial region size to the right side by dragging the icon 611 to the right.
  • the user can reduce the input work by decreasing the partial region size.
  • the user can also form more labeled data by increasing the size and extending the labeling range to the periphery of the partial region.
  • FIG. 6 can also emphatically display the position of local data such as a pixel 612 in addition to the display of the partial region 633 . Since local data is a portion important for learning of a discriminator, the user desirably performs labeling very carefully. By thus emphatically displaying the pixel of local data, therefore, it is possible to draw user's attention when he or she performs labeling.
  • the method of emphatically displaying local data can be any method.
  • the pixel of local data can be displayed with a conspicuous color such as red. Since, however, a labeling target pixel cannot be seen in this emphatic display, it is also possible to use, for example, a display method of blinking the display at a predetermined interval, a display method that allows the user to switch “display” and “non-display” of the emphatic display, or a semitransparent display method. If the visibility is insufficient when only the pixel of local data is emphatically displayed, it is also possible to emphatically display a predetermined range containing the pixel of local data in the center. Furthermore, local data to be labeled with care can emphatically be displayed by a display method such as a method of enclosing the local data portion with a circle, or a method of indicating the local data portion by an arrow.
  • FIG. 7 shows the way a crack label is input in the partial region 633 .
  • the user inputs a crack label 711 to a crack pixel in the image by moving a mouse cursor 710 . More specifically, the crack label is input to the position of a crack by dragging the mouse (moving the mouse while pushing it) in the position of a pixel supposed to be a crack. Also, the user must confirm that a pixel to which no crack label is input is not a crack but a normal concrete wall surface. This is equivalent to inputting a non-crack label to an image to which a crack label is not attached. When all cracks in the partial region are labeled, labeling to the partial region is complete.
  • the emphatic display of local data may also be erased in accordance with the label input state.
  • the emphatic display of local data of the pixel to which the label is input, or of a pixel near the label is terminated.
  • local data 712 or the like is local data for which emphatic display is terminated when the crack label 711 is input.
  • the user can terminate the emphatic display of individual local data by using a tool for erasing (terminating) the emphatic display of local data in a predetermined range, like an eraser tool of paint software.
  • a tool for erasing (terminating) the emphatic display of local data in a predetermined range like an eraser tool of paint software.
  • the user terminates the emphatic display of local data if he or she thinks that the local data is not a crack. This allows the user to confirm the progress of determination of the presence/absence of a crack, even for an important portion to which no crack label is input.
  • FIG. 8 shows the way the user inputs a crack label 811 so that the label extends from the range of the partial region 633 . If an input crack label extends from the range of the partial region 633 , the display control unit 104 displays an extended range 810 of the partial region 633 on the display unit 105 .
  • the extended range 810 shown in FIG. 8 is displayed by the dotted lines, and is a range obtained by extending the rectangular shape of the partial region 633 to a range including the input crack label 811 .
  • the method of determining the extended range is not limited to this method.
  • a range surrounding only the vicinity of the crack label 811 extending from the partial region 633 can also be set as the extended range.
  • the display method may also be a display method other than the method of displaying the extended range by the dotted lines.
  • the extended range can also be displayed by a transparent color.
  • the extended range of a partial region When displaying the extended range of a partial region, it is also possible to display the extended range immediately after the user inputs the crack label 811 (immediately after the user releases mouse click at the position of a mouse pointer 801 ), and inquire the user about whether to extend the partial region.
  • the display unit 105 displays the extended range 810 , and also displays a dialogue box 820 by which the user gives an instruction to extend the partial region. If the user selects “YES”, the range of the partial region to be labeled is extended to the extended range 810 . Note that when the partial region is extended, a crack may be missed in an extended portion spaced apart from the input crack.
  • the dialogue box 820 displays a message that alerts missing of a crack in the extended range caused by the extension of the partial region by the user.
  • the method of alert like this is not limited to text display in the dialogue box, and it is also possible to directly display a region to be noted in the image 601 .
  • a partial region may also be extended at an arbitrary timing in accordance with a user's instruction.
  • an extended range is determined based on an input crack label extending from the partial region, and the extended range and an inquiry about region extension are displayed.
  • Partial region display switching will be explained below by referring to FIG. 6 again.
  • the image 601 displays an image range containing partial regions.
  • a partial region having the highest priority can be displayed.
  • partial regions can be displayed in order based on the priority of each partial region.
  • M regions are set as partial regions, and the user desirably labels all the partial regions from the viewpoint of learning data expansion.
  • the user it is also possible to allow the user to stop labeling partial regions halfway by clicking an “END” button 642 .
  • partial regions to which labels are input are data usable in learning.
  • the display order of partial regions is not limited to the order of priority, and can also be determined by another criterion.
  • partial regions can be displayed from the closest one. More specifically, a partial region having the highest priority is displayed first, and subsequent partial regions are displayed from the one having the shortest distance in an image, or from the one having the shortest three-dimensional distance in a structure. This increases the possibility that similar images are displayed in succession, and makes it possible to reduce the input load of the user.
  • partial regions can also be displayed from the farthest one. When displaying partial regions from the farthest one, labels may be input to various images in the early stages of the partial region display order. Accordingly, diverse labeled data can be formed even if the user terminates label input before completing label input to all partial regions.
  • the partial region display order can also be determined based on the positional relationships between the partial regions.
  • the user may also select a partial region as a label input target.
  • a partial region as a label input target.
  • a crack label can be given to a partial region by the operations of the display control unit 104 , the display unit 105 , and the label input unit 106 .
  • the input labeled data is stored in the data storage unit 101 in association with the image.
  • a new discriminator for crack detection can learn by using the image and the labeled data.
  • the learning unit 108 shown in FIG. 1 performs learning of a crack detection discriminator by using the image and the label data stored in the data storage unit 101 , and stores the learned discriminator in the discriminator storage unit 107 . Note that the learning unit 108 can perform discriminator learning by using only newly formed labeled data, or by using a data set including the existing labeled data.
  • a hard-to-discriminate pixel is selected as local data by using the new discriminator.
  • a partial region is set based on the local data distribution and labeling is requested for the partial region, in the same manner as above.
  • Learning of a discriminator and selection of local data can also be repeated a plurality of times. This repetition of labeling and learning can be performed until there is no pixel which a discriminator hardly discriminates (until there is no more local data to be selected), and can also be terminated by user's decision.
  • learning of a discriminator and update of a partial region can automatically be performed in the background while the user is labeling partial regions.
  • the labeling target image of the first embodiment is a structure image pasted on a plan view as shown in FIG. 2 , but the labeling target image is not limited to this.
  • an image data set including a plurality of snap images may also be used as labeling target data.
  • the processes of the local portion setting unit 102 and the partial region determination unit 103 are performed on each image of the image data set, and a partial region as a labeling target is determined from each of the plurality of images.
  • the display control unit 104 , the display unit 105 , and the label input unit 106 sequentially present the partial regions set in the plurality of images to the user, thereby performing label input to the partial regions.
  • the discrimination target of the first embodiment is a crack on the concrete wall surface, but the discrimination target is not limited to this.
  • this embodiment is also applicable to a case of a defective region in inspection of parts, and a case requiring a label for each pixel, such as semantic segmentation of an image or object contour extraction.
  • the label of the first embodiment is a 2-class label indicating whether each pixel is a crack, but the label may also be a multi-class label.
  • semantic segmentation for example, pixels are discriminated to multiple classes such as a sky region, a tree region, a ground region, and a person region. Therefore, a multi-class label must be used as label data.
  • This embodiment is also applicable to multi-class labeling like this.
  • local data is not limited to a pixel.
  • local data is a region including at least a plurality of pixels.
  • local data will be called a local region. This local region is narrower than a partial region to be presented as a labeling target region to the user.
  • a method of selecting local data (a local region) by a method different from that of the first embodiment will be explained.
  • Other processes can be executed by the same processes as in the first embodiment, so an explanation thereof will be omitted.
  • an apparatus arrangement is the same as that of the first embodiment, so an explanation thereof will be omitted.
  • an image 900 is a labeling target image. Assume that the image 900 is segmented into regions each having a size of 256 ⁇ 256 pixels like a patch 901 . In the second embodiment, whether to select each patch as a labeling target local region is determined. Note that in order to simplify the explanation, an embodiment that determines whether to select a patch partitioned by a grid as a local region will be explained. However, a region for local region determination is not limited to this. For example, a patch for local region determination can also be obtained from an image so that a plurality of patches overlap each other. In addition, the patch size is not limited to a size of 256 ⁇ 256 pixels, and may also be another size or another aspect ratio.
  • a local portion setting unit 102 determines whether an image of each patch is an unknown image of an unknown type not contained in the existing learning data, and, if the image is an unknown image, determines that the image requires labeling, and selects the patch as a local region. This is based on the idea that an image similar to an image of the existing learning data can be processed by learning using the existing data, but an unknown image not contained in the existing learning data should be labeled anew and added to the learning data.
  • an unknown image discriminator is prepared by using the existing learning data. When a patch image is input, this discriminator calculates the similarity to or the divergence from the existing image data set.
  • a patch having similarity equal to or smaller than a predetermined value, or a patch having a divergence equal to or larger than a predetermined value, is regarded as an unknown image, and selected as a local region.
  • the method of obtaining the similarity or divergence of given data with respect to a data set can be implemented by various well-known methods, so any method can be used as the discriminator.
  • a discriminator for calculating the divergence will be explained as an example of the discriminator.
  • a patch is obtained from an image of the existing learning data set, and an image of the patch is converted into a predetermined feature amount.
  • This feature amount is, for example, a feature representing the atmosphere or feature of the image, and a method such as an auto encoder can be applied.
  • This feature extraction is performed on a group of patches obtained from an image of the existing learning data set, thereby obtaining the feature amount of the existing learning data set.
  • the divergence can be calculated based on the distribution of this feature amount in a feature space.
  • a feature amount is extracted from a patch image as a determination target, and the distance between the feature amount of the patch image and the feature amount distribution of an image of the existing learning data set is calculated as the divergence.
  • the distance from the feature amount distribution can simply be a Euclidean distance from the distribution center, and can also be a Mahalanobis distance by taking the distribution into consideration.
  • the divergence from the existing learning data set is selected as a local region.
  • hatched patches such as a patch 902 are patches set as local regions.
  • the next processing after the local regions are set is a process of determining a partial region based on the distribution of the local regions by a partial region determination unit 103 , and this process can be performed by the same method as that of the first embodiment.
  • FIG. 9C shows a state in which partial regions 903 and 904 are set by a method of setting, as a partial region, a range in which the density of local regions (local data) is equal to or larger than a predetermined value.
  • this partial region setting process it is possible to form partial region candidates and select a partial region from the partial region candidates, in the same manner as in the first embodiment.
  • this importance can be used as a criterion for selecting a partial region from partial region candidates including local regions.
  • the user In processing after the partial region is set, the user is requested to label the partial region by a display unit 105 and a label input unit 106 in the same manner as in the first embodiment.
  • Labeling target data is an image in the first and second embodiments, but labeling target data is not limited to an image.
  • labeling target data is a moving image.
  • labeling target data is a moving image
  • one frame of a moving image is local data
  • a plurality of consecutive frames containing the local data are partial regions of labeling target data.
  • labeled data for performing learning of a discriminator for person detection is formed from moving image data of labeling target data.
  • a discriminator, however, to be applied to a moving image is not limited to this, and any discriminator can be a target.
  • any discriminator can be a target.
  • labeled data for performing learning of a discriminator for detecting abnormality in a moving image can be formed.
  • a discriminator for discriminating a plurality of consecutive frames such as a discriminator for discriminating a human action can be a target.
  • consecutive short frames indicating a human action are local data
  • consecutive frames containing the local data and longer than the frames of the local data are consecutive regions.
  • FIG. 10 shows a moving image 1000 as labeling target data.
  • the moving image 1000 includes frame images such as a frame 1001 .
  • a local portion setting unit 102 applies a learned discriminator for person detection to the moving image 1000 as labeling target data.
  • the existing discriminator determines hard-to-discriminate local data, in the same manner as in the first embodiment.
  • an image 1113 indicates a person detection result 1030 in an image of a frame 1013 .
  • the person detection result 1030 indicates a result near the discrimination plane of the existing discriminator and is a result which the existing discriminator hardly discriminates.
  • a frame including a result which the existing discriminator hardly discriminates as described above is local data.
  • the moving image 1000 indicates a frame of local data containing a hard-to-discriminate detection result, such as frames 1011 , 1013 , and 1014 , by the thick lines.
  • the local portion setting unit 102 sets a frame of a moving image of labeling target data as local data.
  • a partial region determination unit 103 selects a labeling target partial region based on the temporal distribution of the local data. More specifically, consecutive frames in which the density of the local data is high on the time axis of the moving image are set as partial regions.
  • FIG. 10 shows the way consecutive frames 1020 and 1021 are selected as partial regions.
  • the user inputs a label indicating the position of a person to each of the consecutive frames by using a display unit 105 and a label input unit 106 .
  • the person position label is, for example, rectangular information indicating a person position range in a frame image. By this operation, data for performing learning of a discriminator for person detection can be formed.
  • the way a person looks does not largely change between the frames. Accordingly, there is a case in which when a person position label is input to a given frame, the same label can be applied to consecutive frames.
  • person position labels are input to, for example, the frames 1011 and 1013 , a person position label of a frame between them can be estimated from the person position labels of the frames 1011 and 1013 .
  • a function of supporting label input of consecutive frames can be given to the display unit 105 and the label input unit 106 .
  • a function of supporting label input of consecutive frames can be given to the display unit 105 and the label input unit 106 .
  • labeling target data is not limited to moving image data and may also be time-series data such as sensor data.
  • labeling can be performed on unlabeled time-series sensor data in order to perform learning of a discriminator for detecting abnormality from a plurality of sensor data.
  • sensor data at the timing which a discriminator hardly discriminates is selected as local data from among the plurality of sensor data.
  • a partial region can be set based on the distribution of local data in a multi-dimensional space or a feature space of the data.
  • This partial region contains a group of similar data near the local data, so the local data supposed to be effective for learning and the group of similar data can be selected as labeling target data.
  • a partial region is set based on one local data.
  • a range is extended based on the position of one given local data, and a peripheral region containing the local data is set as labeling target data.
  • the local data setting method can be performed by the same methods as those of the above-described embodiments. Therefore, a process by which a partial region determination unit 103 determines a partial region in a state in which local data is selected will be explained below.
  • the method of determining a peripheral region by extending a range based on local data can be a method by which a predetermined range centering around local data is set as a peripheral region.
  • FIG. 11A shows the way a peripheral region having a predetermined size is set as a partial region 1111 around local data 1110 as the center.
  • the user performs labeling in the partial region 1111 in the same manner as in the first embodiment.
  • a partial region set by the partial region determination unit 103 is an initial value, so the user can adjust the position and size of the partial region 1111 by using a display unit 105 and a label input unit 106 .
  • FIG. 11B is a view for explaining processing when moving image data is labeling target data.
  • a frame 1120 is local data
  • the range of a predetermined number of frames before and after the frame 1120 is a peripheral region.
  • a range 1121 of the peripheral region is a partial region.
  • a partial region may also be set by combining one local data with another information. For example, it is also possible to evaluate the continuity or the similarity between a pixel of one local data and an image around the pixel, and set a range similar to the pixel of the local data as a partial region. Likewise, when using a moving image, the range of consecutive frames having images similar to that of a frame of local data, or the range of consecutive frames supposed to be the range of the same scene as that of the local data frame, can be set as a partial region.
  • a region where the distribution density of local data is high or a region clustered based on the distribution of local data is set as a partial region.
  • a method of setting a partial region based on the distribution of local data a method of setting the existing range of local data as a partial region will be explained. Note that this embodiment will be explained by taking crack detection of a concrete wall surface image as an example as in the first embodiment.
  • Local data is obtained by determining a data portion effective for learning by a given criterion.
  • a discriminator for crack detection performs learning for discriminating whether an edge portion in an image is a crack or an edge other than a crack, so an edge portion of an image is a portion important for learning in crack detection.
  • a portion effective for learning can be determined by extracting an edge portion of an image by image processing or the like.
  • image processing for edge extraction well-known methods such as differential filter processing and threshold processing can be used. It is also possible to convert an image into frequency components, and extract an edge by a process of leaving only a high-frequency-component region behind.
  • an edge pixel extracted as described above is set as local data.
  • the setting of local data by edge extraction is a local data setting method effective when performing learning of a discriminator whose task is crack detection or contour extraction.
  • a local data setting method like this can be set in accordance with the task of a discriminator as a target of learning. For example, to form labeled data of a sky region in image region segmentation, a blue pixel can be extracted as local data.
  • a versatile local data setting method there is a method by which the user designates a data portion to be determined as effective for learning from labeling target data, and data portions similar to the designated data portion are collected. Specific examples are a method of collecting pixels having a pixel value similar to that of a pixel designated by the user, and a method of performing pattern matching based on an image patch designated by the user and collecting similar image patch portions.
  • a partial region is a region surrounding all positions in each of which local data exists.
  • a partial region may also be a range surrounding all local data set by the above-described processing.
  • a shape surrounding local data can be a rectangular shape, and can also be an arbitrary shape as shown in FIG. 5B of the first embodiment.
  • local data setting methods such as the local data setting method using the differential filter processing and the threshold processing and the local data setting method of extracting a high frequency component have been explained.
  • the local data setting methods to be used together can include the local data setting method using a discriminator in the first embodiment and the like.
  • FIG. 12 shows the way partial regions are set by using a plurality of local data setting methods together.
  • local data setting methods A and B are used together.
  • Circles 1210 shown in FIG. 12 are the pixels of local data set by the local data setting method A.
  • a range 1211 surrounding these local data is a partial region set based on the distribution of the local data of the local data setting method A.
  • x marks 1220 are the pixels of local data set by the local data setting method B.
  • a range 1221 surrounding these local data is a partial region set based on the distribution of the local data of the local data setting method B. Partial regions having different characteristics can be set by thus using a plurality of local data setting methods together. Note that in FIG. 12 , the user can freely adjust the positions and sizes of the presented partial regions 1211 and 1221 in the same manner as in the above-described embodiments.
  • Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s).
  • computer executable instructions e.g., one or more programs
  • a storage medium which may also be referred to more fully as a
  • the computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions.
  • the computer executable instructions may be provided to the computer, for example, from a network or the storage medium.
  • the storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)TM), a flash memory device, a memory card, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
US16/798,602 2019-02-28 2020-02-24 Apparatus for processing labeled data to be used in learning of discriminator, method of controlling the apparatus, and non-transitory computer-readable recording medium Active 2040-05-20 US11308352B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/699,305 US11741683B2 (en) 2019-02-28 2022-03-21 Apparatus for processing labeled data to be used in learning of discriminator, method of controlling the apparatus, and non-transitory computer-readable recording medium

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2019036394A JP7233251B2 (ja) 2019-02-28 2019-02-28 情報処理装置、情報処理装置の制御方法及びプログラム
JPJP2019-036394 2019-02-28
JP2019-036394 2019-02-28

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/699,305 Continuation US11741683B2 (en) 2019-02-28 2022-03-21 Apparatus for processing labeled data to be used in learning of discriminator, method of controlling the apparatus, and non-transitory computer-readable recording medium

Publications (2)

Publication Number Publication Date
US20200279132A1 US20200279132A1 (en) 2020-09-03
US11308352B2 true US11308352B2 (en) 2022-04-19

Family

ID=72236135

Family Applications (2)

Application Number Title Priority Date Filing Date
US16/798,602 Active 2040-05-20 US11308352B2 (en) 2019-02-28 2020-02-24 Apparatus for processing labeled data to be used in learning of discriminator, method of controlling the apparatus, and non-transitory computer-readable recording medium
US17/699,305 Active US11741683B2 (en) 2019-02-28 2022-03-21 Apparatus for processing labeled data to be used in learning of discriminator, method of controlling the apparatus, and non-transitory computer-readable recording medium

Family Applications After (1)

Application Number Title Priority Date Filing Date
US17/699,305 Active US11741683B2 (en) 2019-02-28 2022-03-21 Apparatus for processing labeled data to be used in learning of discriminator, method of controlling the apparatus, and non-transitory computer-readable recording medium

Country Status (2)

Country Link
US (2) US11308352B2 (ja)
JP (1) JP7233251B2 (ja)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220130140A1 (en) * 2019-03-14 2022-04-28 Nec Corporation Information presentation method, information presentation device and program

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114945801A (zh) 2020-01-10 2022-08-26 塔斯米特株式会社 图案边缘检测方法、图案边缘检测装置、记录有用于让计算机执行图案边缘检测的程序的记录介质

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5671263A (en) * 1996-03-13 1997-09-23 Analogic Corporation Motion artifact suppression filter for use in computed tomography systems
US7050618B2 (en) * 2002-02-08 2006-05-23 Eastman Kodak Company Method for antiscatter stationary grid artifacts detection and attenuation in digital radiographic images
US7150559B1 (en) * 2002-09-25 2006-12-19 Illinois Tool Works Inc. Hot melt adhesive detection methods and systems
US20070208678A1 (en) 2004-03-17 2007-09-06 Canon Kabushiki Kaisha Parallel Pulse Signal Processing Apparatus, Pattern Recognition Apparatus, And Image Input Apparatus
US20070270664A1 (en) 2004-10-27 2007-11-22 Canon Kabushiki Kaisha Estimation Apparatus and Its Control Method
US20080211904A1 (en) 2004-06-04 2008-09-04 Canon Kabushiki Kaisha Situation Monitoring Device and Situation Monitoring System
US7434986B2 (en) * 2004-06-24 2008-10-14 Fluke Corporation Method and apparatus for monitoring and detecting defects in plastic package sealing
US20100079263A1 (en) 2008-10-01 2010-04-01 Canon Kabushiki Kaisha Information processing apparatus and information processing method
US20110037984A1 (en) 2008-02-01 2011-02-17 Canon Kabushiki Kaisha Information processing apparatus and method
US20110206237A1 (en) 2010-02-25 2011-08-25 Canon Kabushiki Kaisha Recognition apparatus and method thereof, and computer program
US8285056B2 (en) * 2008-03-13 2012-10-09 Hitachi High-Technologies Corporation Method and apparatus for computing degree of matching
US8405836B2 (en) * 2010-03-19 2013-03-26 Interfiber Analysis, LLC System and method for measuring an optical fiber
US20130286047A1 (en) 2012-04-25 2013-10-31 Canon Kabushiki Kaisha Mirror system and control method therefor
US8825131B2 (en) * 2009-10-14 2014-09-02 Nocimed, Llc MR spectroscopy system and method for diagnosing painful and non-painful intervertebral discs
US9589210B1 (en) * 2015-08-26 2017-03-07 Digitalglobe, Inc. Broad area geospatial object detection using autogenerated deep learning models
US9668699B2 (en) * 2013-10-17 2017-06-06 Siemens Healthcare Gmbh Method and system for anatomical object detection using marginal space deep neural networks
JP2017167834A (ja) 2016-03-16 2017-09-21 セコム株式会社 学習データ選択装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8239380B2 (en) * 2003-06-20 2012-08-07 Microsoft Corporation Systems and methods to tune a general-purpose search engine for a search entry point
US7831087B2 (en) * 2003-10-31 2010-11-09 Hewlett-Packard Development Company, L.P. Method for visual-based recognition of an object
US7188050B2 (en) * 2004-08-25 2007-03-06 Siemens Corporate Research, Inc. Method and apparatus for detecting out-of-range conditions in power generation equipment operations
US7840511B2 (en) * 2006-09-06 2010-11-23 Siemens Medical Solutions Usa, Inc. Learning or inferring medical concepts from medical transcripts using probabilistic models with words or phrases identification
US8092388B2 (en) * 2007-09-25 2012-01-10 Siemens Medical Solutions Usa, Inc. Automated view classification with echocardiographic data for gate localization or other purposes
US8090676B2 (en) * 2008-09-11 2012-01-03 Honeywell International Inc. Systems and methods for real time classification and performance monitoring of batch processes
US9119573B2 (en) * 2009-12-10 2015-09-01 Siemens Aktiengesellschaft Stent marker detection using a learning based classifier in medical imaging
US10339422B2 (en) 2015-03-19 2019-07-02 Nec Corporation Object detection device, object detection method, and recording medium
WO2018168515A1 (ja) 2017-03-14 2018-09-20 日本電気株式会社 画像処理装置、画像処理方法及び記録媒体

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5671263A (en) * 1996-03-13 1997-09-23 Analogic Corporation Motion artifact suppression filter for use in computed tomography systems
US7050618B2 (en) * 2002-02-08 2006-05-23 Eastman Kodak Company Method for antiscatter stationary grid artifacts detection and attenuation in digital radiographic images
US7174038B2 (en) * 2002-02-08 2007-02-06 Eastman Kodak Company Method for antiscatter stationary grid artifacts detection and attenuation in digital radiographic images
US7150559B1 (en) * 2002-09-25 2006-12-19 Illinois Tool Works Inc. Hot melt adhesive detection methods and systems
US20070208678A1 (en) 2004-03-17 2007-09-06 Canon Kabushiki Kaisha Parallel Pulse Signal Processing Apparatus, Pattern Recognition Apparatus, And Image Input Apparatus
US20080211904A1 (en) 2004-06-04 2008-09-04 Canon Kabushiki Kaisha Situation Monitoring Device and Situation Monitoring System
US7434986B2 (en) * 2004-06-24 2008-10-14 Fluke Corporation Method and apparatus for monitoring and detecting defects in plastic package sealing
US20070270664A1 (en) 2004-10-27 2007-11-22 Canon Kabushiki Kaisha Estimation Apparatus and Its Control Method
US20110037984A1 (en) 2008-02-01 2011-02-17 Canon Kabushiki Kaisha Information processing apparatus and method
US8285056B2 (en) * 2008-03-13 2012-10-09 Hitachi High-Technologies Corporation Method and apparatus for computing degree of matching
US20100079263A1 (en) 2008-10-01 2010-04-01 Canon Kabushiki Kaisha Information processing apparatus and information processing method
US20130135087A1 (en) 2008-10-01 2013-05-30 Canon Kabushiki Kaisha Information processing apparatus and information processing method
US8825131B2 (en) * 2009-10-14 2014-09-02 Nocimed, Llc MR spectroscopy system and method for diagnosing painful and non-painful intervertebral discs
US20110206237A1 (en) 2010-02-25 2011-08-25 Canon Kabushiki Kaisha Recognition apparatus and method thereof, and computer program
US8405836B2 (en) * 2010-03-19 2013-03-26 Interfiber Analysis, LLC System and method for measuring an optical fiber
US20130286047A1 (en) 2012-04-25 2013-10-31 Canon Kabushiki Kaisha Mirror system and control method therefor
US9668699B2 (en) * 2013-10-17 2017-06-06 Siemens Healthcare Gmbh Method and system for anatomical object detection using marginal space deep neural networks
US9589210B1 (en) * 2015-08-26 2017-03-07 Digitalglobe, Inc. Broad area geospatial object detection using autogenerated deep learning models
JP2017167834A (ja) 2016-03-16 2017-09-21 セコム株式会社 学習データ選択装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220130140A1 (en) * 2019-03-14 2022-04-28 Nec Corporation Information presentation method, information presentation device and program

Also Published As

Publication number Publication date
US20200279132A1 (en) 2020-09-03
JP2023054206A (ja) 2023-04-13
JP7233251B2 (ja) 2023-03-06
US20220207290A1 (en) 2022-06-30
US11741683B2 (en) 2023-08-29
JP2020140518A (ja) 2020-09-03

Similar Documents

Publication Publication Date Title
US11741683B2 (en) Apparatus for processing labeled data to be used in learning of discriminator, method of controlling the apparatus, and non-transitory computer-readable recording medium
US10769473B2 (en) Image processing apparatus, image processing method, and non-transitory computer-readable storage medium
US8437534B2 (en) Defect classification method and apparatus, and defect inspection apparatus
US8625906B2 (en) Image classification standard update method, program, and image classification device
JP6089886B2 (ja) 領域分割方法および検査装置
JP5022174B2 (ja) 欠陥分類方法及びその装置
CN110837403B (zh) 机器人过程自动化
US9040937B2 (en) Charged particle beam apparatus
CN111768381A (zh) 零部件缺陷检测方法、装置及电子设备
CN111368600A (zh) 遥感图像目标检测识别方法、装置、可读存储介质及设备
JP2014059875A (ja) 画像内のロゴの存在を検出する装置、及び方法
US11170501B2 (en) Image analysis device
JP5439543B2 (ja) 欠陥分類方法及びその装置
JP2021165888A (ja) 情報処理装置、情報処理装置の情報処理方法およびプログラム
CN110737785A (zh) 一种图片标注的方法及装置
Stommel et al. Segmentation-free detection of comic panels
CN115136209A (zh) 缺陷检测系统
US20120237118A1 (en) Image processing device, image processing method, and image processing program
CN110866931A (zh) 图像分割模型训练方法及基于分类的强化图像分割方法
JP2013171309A (ja) 文字切り出し方法、およびこの方法を用いた文字認識装置およびプログラム
JP2006292615A (ja) 外観検査装置、外観検査方法、コンピュータを外観検査装置として機能させるためのプログラムおよび記録媒体
CN116559177A (zh) 一种缺陷检测方法、装置、设备以及存储介质
JP7496008B2 (ja) 情報処理装置、情報処理装置の制御方法及びプログラム
JP2012238401A (ja) 欠陥レビュー方法および装置
JP2018005505A (ja) 画像認識パラメータ設定装置、画像認識パラメータ設定方法、および画像認識パラメータ設定プログラム

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED

AS Assignment

Owner name: CANON KABUSHIKI KAISHA, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NOGAMI, ATSUSHI;MATSUGU, MASAKAZU;SIGNING DATES FROM 20200304 TO 20200330;REEL/FRAME:053065/0121

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE