WO2022091478A1 - 処理方法およびそれを利用した処理装置 - Google Patents

処理方法およびそれを利用した処理装置 Download PDF

Info

Publication number
WO2022091478A1
WO2022091478A1 PCT/JP2021/024226 JP2021024226W WO2022091478A1 WO 2022091478 A1 WO2022091478 A1 WO 2022091478A1 JP 2021024226 W JP2021024226 W JP 2021024226W WO 2022091478 A1 WO2022091478 A1 WO 2022091478A1
Authority
WO
WIPO (PCT)
Prior art keywords
processing unit
image
processing
neural network
layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/024226
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
真也 家治
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to JP2022558850A priority Critical patent/JP7557796B2/ja
Publication of WO2022091478A1 publication Critical patent/WO2022091478A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present disclosure relates to a processing technique, particularly a processing method for executing processing on an image, and a processing apparatus using the same.
  • Deep Learning is used for the image recognition process.
  • Deep Learning is known as a machine learning methodology using a multi-layered neural network, and for example, a convolutional neural network is used for the multi-layered neural network.
  • a convolutional neural network is formed by a multi-layered neural network that repeats convolution and pooling of local regions. Further, a structure of a completely convolutional neural network in which a fully connected layer constituting a convolutional neural network is made into a convolutional layer has been proposed (see, for example, Patent Document 1).
  • a fully convolutional neural network is a stack of local region processing.
  • it is difficult to distinguish between the solder for fixing the component and the solder hole that is a defective product. May be.
  • the present disclosure has been made in view of such a situation, and an object thereof is to provide a technique for suppressing deterioration of processing accuracy of image recognition processing.
  • the processing apparatus of one aspect of the present invention has an input unit for inputting an image of a processing target including a plurality of elements, and segmentation for each element with respect to the image of the processing target input to the input unit.
  • the first processing unit that generates the segmented image
  • the second processing unit that executes the classification by executing the neural network processing on the segmentation image generated in the first processing unit, and the second processing unit. It is provided with an output unit that outputs the result of the classification in.
  • Another aspect of the present invention is a processing method.
  • an image to be processed containing multiple elements is input, a segmentation image segmented for each element is generated for the input image to be processed, and neural network processing is performed on the generated segmentation image.
  • the classification is executed and the classification result by the classification is output.
  • any combination of the above components and the conversion of the expression of the present disclosure between a method, an apparatus, a system, a computer program, a recording medium on which a computer program is recorded, or the like is also effective as an aspect of the present disclosure. be.
  • FIG. It is a figure which shows the structure of the processing apparatus which concerns on Example 1.
  • FIG. It is a figure which shows the outline of the processing in the 1st processing part of FIG. 3A and 3D are diagrams showing an outline of processing in the processing apparatus of FIG. 1.
  • (a)-(d) are diagrams showing an outline of processing in the processing apparatus according to the second embodiment.
  • 7 (a)-(e) are diagrams showing an outline of another process in the processing apparatus according to the second embodiment.
  • 8 (a)-(e) are diagrams showing an outline of processing in the processing apparatus according to the third embodiment.
  • 9 (a)-(f) are diagrams showing an outline of another process in the processing apparatus according to the third embodiment.
  • An embodiment relates to a processing device that performs a visual inspection of a product by performing an image recognition process on an image captured by the product.
  • An example of a product is a board to which parts are soldered, and an example of a visual inspection is an inspection of whether the solder on the board is a good product or a defective product. Good solder is solder for fixing parts, and defective solder is solder holes.
  • the use of convolutional neural networks is promising for visual inspection.
  • the convolutional neural network uses a convolutional layer that extracts features by convolutional processing by multiple filters, a pooling layer that acquires local data invariance by pooling processing that summarizes reactions in a certain region, and a probability by the Softmax function. Includes a fully connected layer for recognition.
  • the processing of the convolutional layer and the pooling layer is repeated, and an image in which the contained object is segmented by color painting (hereinafter referred to as "segmentation image”) is output.
  • segmentation image an image in which the contained object is segmented by color painting
  • the complete convolutional neural network is a stack of processing in the local region, it outputs a segmentation image in which the solder is colored, but it becomes difficult to distinguish whether the solder is a good product or a defective product. In some cases. In order to make these distinctions possible, it is desirable to utilize the positional relationship on the substrate.
  • the processing apparatus executes processing by a complete convolutional neural network on the image to be processed to generate a segmentation image in which solder is colored.
  • the processing device executes processing by a convolutional neural network on the segmentation image, and obtains the result of classification of whether the solder is good or bad.
  • the positional relationship on the substrate is utilized.
  • the segmentation image is generated by the complete convolutional neural network and then the processing by the convolutional neural network is executed, so that the increase in the processing amount is suppressed.
  • FIG. 1 shows the configuration of the processing device 100.
  • the processing device 100 includes an input unit 112, a first processing unit 114, an intermediate processing unit 116, a second processing unit 118, and an output unit 120.
  • (1) configuration, (2) learning, and (3) classification will be described in this order.
  • FIG. 2 shows an outline of processing in the first processing unit 114.
  • the first processing unit 114 includes a first convolution layer 142a, a second convolution layer 142b, a third convolution layer 142c, a fourth convolution layer 142d, a fifth convolution layer 142e, a sixth convolution layer 142f, and pooling, which are collectively referred to as a convolution layer 142.
  • the input image 140 is an image to be processed. Since the neural network in the first processing unit 114 does not include the fully connected layer, there is no limitation on the size of the input image 140.
  • the input image 140 is input to the first convolution layer 142a.
  • Each convolution layer 142 is represented by a straight hexahedron.
  • the size of the square surface having the depth direction and the height direction of the convolution layer 142 indicates the size of the spatial dimension of the image processed by the convolution layer 142, that is, the size of the image.
  • the convolution layer 142 performs spatial filtering on the image while shifting a spatial filter having a size smaller than the size of the image. Since spatial filtering is a known technique, the description thereof is omitted here, but this spatial filtering corresponds to the convolution process, and the feature amount of the image is extracted by the convolution process. Padding or the like may be performed on the convolution layer 142.
  • the convolution layer 142 may perform a plurality of spatial filters in parallel by using a plurality of spatial filters in parallel with respect to the image.
  • the parallel use of such multiple spatial filters increases the number of images.
  • the number of spatial filters used in parallel in the convolution layer 142 is called the number of channels, which is indicated by the lateral length of the convolution layer 142.
  • Each pooling layer 144 is also configured in the same manner as the convolution layer 142.
  • the pooling layer 144 reduces the size of the image by combining a plurality of pixels contained in an arbitrary region in the image into one pixel.
  • average pooling or maximum pooling is performed in order to combine the plurality of pixels into one pixel.
  • average pooling the average value of multiple pixel values in the area is used for one pixel
  • maximum pooling the maximum value of multiple pixel values in the area is used for one pixel. ..
  • the pooling process is performed in order to enhance the robustness against the translational movement of the representative value or the average value in the region of interest.
  • the fifth convolution layer 142e, the fifth pooling layer 144e, and the sixth convolution layer 142f are arranged in this order. That is, the convolution process and the pooling process are repeatedly executed. Further, by repeating the convolution process and the pooling process, the size of the image is sequentially reduced, and the sixth pooling layer 144f outputs a segmentation image having one or more channels.
  • the second processing unit 118 has a configuration of a convolutional neural network including the above-mentioned fully connected layer.
  • the second processing unit 118 includes a fully connected layer in addition to the convolution layer and the pooling layer.
  • the fully connected layer receives the image from which the feature amount is extracted from the intermediate processing unit 116.
  • An example of an image from which features have been extracted is a segmentation image.
  • the fully connected layer identifies the image by performing classification into multiple classes based on the features. Since a known technique may be used for the treatment in the fully bonded layer, the description thereof is omitted here, but the classification result in the fully bonded layer is output from the second processing unit 118 to the output unit 120.
  • the first processing unit 114 receives teacher data that is a known segmentation image, and also receives a learning image corresponding to the received teacher data.
  • the first processing unit 114 learns the coefficients of the spatial filter of each convolution layer 142 based on the received teacher data and the received learning image.
  • the size of the teacher data has a spatial dimension of 1 ⁇ 1. Therefore, the teacher data does not have a positional relationship of the objects included in the learning image, and the teacher data for one channel only indicates the existence of the objects included in the segmentation image. For example, the teacher data for the "solder" channel only indicates the presence of solder. The same applies to other channels. That is, the teacher data for one channel need only indicate whether or not one object exists, and does not have to be an image in which the object is segmented by color painting.
  • the learning image is an original image in which the teacher data is output when the image recognition process is executed accurately, and its size is determined so that the teacher data has a spatial dimension of 1 ⁇ 1. ..
  • the learning image and the input image 140 may have different sizes. Since a known technique may be used for learning the coefficients of the spatial filter in the situation where such teacher data and the image for learning are used, the description thereof will be omitted here.
  • the second processing unit 118 inputs the teacher data which is a known output and the learning image corresponding to the teacher data, and trains the coefficient in the fully connected layer and the coefficient of the spatial filter of each convolution layer.
  • the learning image has the same size as the image input from the intermediate processing unit 116, and is an original image such that teacher data is output when the image recognition processing is accurately executed.
  • a known technique may be used for such a learning process.
  • the class classification input unit 112 inputs an image to be processed.
  • the image input to the input unit 112 corresponds to the above-mentioned input image 140.
  • An example of the input image 140 is an image of a substrate to which components are soldered, which is an image containing a plurality of elements.
  • 3A and 3D show an outline of processing in the processing apparatus 100.
  • FIG. 3A shows an input image 140.
  • the substrate 200 is shown in the input image 140, and the first component 202a and the second component 202b, which are collectively referred to as the component 202, are mounted on the substrate 200 by the fixing solder 204.
  • the component 202 is, for example, an IC (Integrated Circuit).
  • a solder hole 206 is also arranged on the substrate 200.
  • the fixing solder 204 is a good product, and the solder hole 206 is a defective product. 3 (b)-(d) will be described later, and the process returns to FIG.
  • the coefficient of the spatial filter derived by learning is set in the convolution layer 142 of the first processing unit 114.
  • the first processing unit 114 processes the input image 140 input to the input unit 112 by a convolutional neural network in which the fully connected layer is excluded and the convolutional layer 142 and the pooling layer 144 are included. Generate a segmented image for each segmentation.
  • FIG. 3B shows the segmentation image 150 generated by the first processing unit 114.
  • the segmentation image 150 shows the fixing solder 214 and the solder holes 216.
  • the fixing solder 214 is the result of extracting the fixing solder 204 by the first processing unit 114
  • the solder hole 216 is the result of extracting the solder hole 206 by the first processing unit 114.
  • it is not classified whether the fixing solder 214 and the solder hole 216 are non-defective products or defective products. 3 (c)-(d) will be described later, and the process returns to FIG.
  • the intermediate processing unit 116 is arranged between the first processing unit 114 and the second processing unit 118, and serves as an interface between the first processing unit 114 and the second processing unit 118.
  • the intermediate processing unit 116 receives the segmentation image from the first processing unit 114.
  • the intermediate processing unit 116 outputs the received segmentation image as it is to the second processing unit 118.
  • the second processing unit 118 executes the classification of the segmentation image received from the intermediate processing unit 116 by executing the processing of the neural network including the fully connected layer.
  • FIG. 3C shows teacher data of a class determined to be non-defective, and four fixing solders 214 are arranged at the positions of the vertices of the rectangle. That is, the positional relationship between the four fixing solders 214 is learned by the second processing unit 118 as a class of non-defective products.
  • FIG. 3D shows teacher data for a class determined to be defective, showing solder holes 216 in addition to the four fixing solders 214 located at the vertices of the rectangle.
  • the second processing unit 118 executes class classification into a non-defective product class or a defective product class according to the positional relationship of the fixing solder 214 and the positional relationship between the fixing solder 214 and the solder hole 216. As a result, it is inspected whether it is a good product or a defective product.
  • the output unit 120 outputs the classification result of the second processing unit 118.
  • the intermediate processing unit 116 has output the segmentation image received from the first processing unit 114 to the second processing unit 118 as it is. However, the intermediate processing unit 116 may execute some processing on the segmentation image and output the processed segmentation image to the second processing unit 118.
  • an inspection is performed in which a solder hole 206 exists in the vicinity of the fixing solder 204 as a defective product. In that case, the portion of the input image 140 that is separated from the fixing solder 204 is an unnecessary portion for processing by the second processing unit 118.
  • the user operates the operation unit (not shown) of the processing device 100 to select a portion of the input image 140 to be inspected.
  • the portion to be inspected includes four fixing solders 204.
  • the processing device 100 stores in advance the correspondence between the coordinates in the input image 140 and the coordinates in the segmentation image 150, and converts the coordinates of the portion of the input image 140 to be inspected into the coordinates of the segmentation image 150.
  • the intermediate processing unit 116 extracts a part of the region indicated by the converted coordinates from the segmentation image 150 generated by the first processing unit 114.
  • the intermediate processing unit 116 outputs a part of the extracted region to the second processing unit 118 instead of the segmentation image 150.
  • the second processing unit 118 executes neural network processing on a part of the region extracted by the intermediate processing unit 116 as a segmentation image. At that time, the size of the teacher data used in the learning process in the second processing unit 118 is the same as the size of a part of the area.
  • the subject of the device, system, or method in this disclosure is equipped with a computer. By executing the program by this computer, the function of the subject of the device, system, or method in the present disclosure is realized.
  • a computer has a processor that operates according to a program as a main hardware configuration. The type of processor does not matter as long as the function can be realized by executing the program.
  • the processor is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or an LSI (Large Scale Integration).
  • IC semiconductor integrated circuit
  • LSI Large Scale Integration
  • the program is recorded on a non-temporary recording medium such as a computer-readable ROM, optical disc, or hard disk drive.
  • the program may be stored in a recording medium in advance, or may be supplied to the recording medium via a wide area communication network including the Internet or the like.
  • FIG. 4 is a sequence diagram showing a processing procedure by the processing apparatus 100.
  • the input unit 112 acquires an image (S10) and outputs the image to the first processing unit 114 (S12).
  • the first processing unit 114 executes segmentation on the image (S14), and outputs the segmentation image to the second processing unit 118 (S16).
  • the second processing unit 118 executes a classification process on the segmentation image (S18), and outputs the classification result to the output unit 120 (S20).
  • the output unit 120 outputs the classification result (S22).
  • FIG. 5 is a flowchart showing a processing procedure by the second processing unit 118.
  • the second processing unit 118 acquires a segmentation image (S100).
  • S100 segmentation image
  • the second processing unit 118 classifies the segmentation image into the non-defective product class (S104).
  • S106 the second processing unit 118 classifies the segmentation image into the defective product class (S106).
  • the segmentation image is generated from the image to be processed and the classification is executed by executing the neural network processing on the segmentation image
  • the increase in the processing amount of the image recognition processing is suppressed.
  • deterioration of processing accuracy can be suppressed.
  • the processing of the convolutional neural network excluding the fully connected layer is executed for the image to be processed and the processing of the neural network including the fully connected layer is executed for the segmentation image
  • the image recognition process is performed. It is possible to suppress an increase in the amount of processing.
  • the processing of the convolutional neural network excluding the fully connected layer is executed for the image to be processed and the processing of the neural network including the fully connected layer is executed for the segmentation image, the positional relationship is included. Features can be extracted.
  • the processing apparatus (100) of a certain aspect of the present disclosure has an input unit (112) for inputting an image to be processed including a plurality of elements, and an input unit (112) for each element with respect to the image to be processed.
  • a first processing unit (114) that generates a segmented segmentation image, and a second processing that executes classification by executing neural network processing on the segmentation image generated by the first processing unit (114).
  • a unit (118) and an output unit (120) for outputting the result of class classification in the second processing unit are provided.
  • the first processing unit (114) is a neural network in which the fully connected layer is excluded from the image to be processed input to the input unit (112), and the convolution layer (142) and the pooling layer (144) are included.
  • the segmentation image may be generated by executing the processing, and the second processing unit (118) may execute the processing of the neural network including the fully connected layer.
  • An intermediate processing unit (116) that extracts a part of the region from the segmentation image generated by the first processing unit (114) may be further provided.
  • the second processing unit (118) may execute neural network processing on a part of the region extracted by the intermediate processing unit (116) as a segmentation image.
  • Another aspect of the present disclosure is a processing method.
  • an image to be processed containing multiple elements is input, a segmentation image segmented for each element is generated for the input image to be processed, and neural network processing is performed on the generated segmentation image.
  • the classification is executed and the classification result by the classification is output.
  • the feature including the positional relationship of one element of the segmentation image is extracted, and the classification result may be a better product or a defective product than the feature including the positional relationship of one element.
  • Example 2 Next, Example 2 will be described.
  • the image to be processed is processed by the complete convolutional neural network to generate a segmentation image, and then the segmentation image is processed by the convolutional neural network.
  • the processing equipment to be used Regarding the processing equipment to be used.
  • the processing of the intermediate processing unit 116 arranged between the first processing unit 114 and the second processing unit 118 is different from that of the first embodiment.
  • the processing apparatus 100 according to the second embodiment is the same type as that of FIG. Here, the difference from the first embodiment will be mainly described.
  • the input unit 112 in FIG. 1 inputs the input image 140.
  • 6 (a)-(d) show the outline of the processing in the processing apparatus 100.
  • FIG. 6A shows an input image 140.
  • the input image 140 is the same as in FIG. 3 (a). 6 (b)-(d) will be described later, and the process returns to FIG.
  • learning is performed in advance in the first processing unit 114.
  • the first embodiment only one channel for the "solder” was included, but in the second embodiment, three channels, a channel for the "board", a channel for the "parts", and a channel for the "solder", are included.
  • the number of channels is not limited to "3".
  • the teacher data in the channel for the "board” only indicates the presence of the board
  • the teacher data in the channel for the "part” only indicates the presence of the component
  • the teacher data in the channel for the "solder” indicates the presence of the solder. Only.
  • the first processing unit 114 after such learning executes the processing of the convolutional neural network in which the fully connected layer is excluded and the convolutional layer 142 and the pooling layer 144 are included in the input image 140.
  • the first processing unit 114 generates a segmentation image including an intermediate image corresponding to each channel.
  • FIG. 6 (b) shows the substrate layer intermediate image 152 corresponding to the channel for the “substrate”.
  • the substrate 210 is shown in the substrate layer intermediate image 152.
  • the substrate 210 is the result of extracting the substrate 200 by the first processing unit 114.
  • FIG. 6 (c) shows a component layer intermediate image 154 corresponding to a channel for a “component”.
  • the component layer intermediate image 154 shows the first component 212a and the second component 212b, which are collectively referred to as the component 212.
  • the first component 212a and the second component 212b are the results of extracting the first component 202a and the second component 202b by the first processing unit 114.
  • solder layer intermediate image 156 shows a solder layer intermediate image 156 corresponding to a channel for “solder”. Since the solder layer intermediate image 156 is the same as FIG. 3B, the description thereof is omitted here.
  • the board layer intermediate image 152, the component layer intermediate image 154, and the solder layer intermediate image 156 are collectively referred to as an intermediate image.
  • the intermediate processing unit 116 selects at least one of a combination of a plurality of intermediate images. For example, the component layer intermediate image 154 and the solder layer intermediate image 156 are selected.
  • the intermediate processing unit 116 outputs the selected intermediate image to the second processing unit 118.
  • the second processing unit 118 inputs the intermediate image selected by the intermediate processing unit 116, and executes the neural network processing on the selected intermediate image.
  • the two or more selected intermediate images are included in different channels and processed by the neural network. Since the class classification in the second processing unit 118 may be the same as that in the first embodiment, the description thereof will be omitted here.
  • FIG. 7A shows an input image 140.
  • the input image 140 is an image included in an image captured in the front direction from the vehicle, and is, for example, an image used in automatic driving. 7 (b)-(e) will be described later, and the process returns to FIG.
  • the first processing unit 114 executes the processing of the convolutional neural network in which the fully connected layer is excluded and the convolutional layer 142 and the pooling layer 144 are included in the input image 140.
  • the first processing unit 114 generates a segmentation image including an intermediate image corresponding to each channel.
  • FIG. 7B is an empty layer intermediate image 160 corresponding to the channel for the “sky”
  • FIG. 7C is a concrete location layer intermediate image 162 corresponding to the channel for the “concrete location”.
  • FIG. 7 (d) is a field layer intermediate image 164 corresponding to the channel for the “field”
  • FIG. 7 (e) is a white line layer intermediate image 166 corresponding to the channel for the “white line”.
  • the sky layer intermediate image 160, the concrete part layer intermediate image 162, the field layer intermediate image 164, and the white line layer intermediate image 166 are collectively referred to as an intermediate image.
  • the intermediate processing unit 116 selects at least one of a combination of a plurality of intermediate images. For example, the concrete portion layer intermediate image 162 and the white line layer intermediate image 166 are selected.
  • the intermediate processing unit 116 outputs the selected intermediate image to the second processing unit 118.
  • the second processing unit 118 inputs the intermediate image selected by the intermediate processing unit 116, and executes the neural network processing on the selected intermediate image.
  • the two or more selected intermediate images are included in different channels and processed by the neural network.
  • a class classification in the second processing unit 118 whether or not the running may be continued is classified.
  • the "positional relationship" of each element and the classification output from the result may be, for example, a class separation regarding where you are now. For example, if the sidewalk layer, the tree layer, and the store layer have a positional relationship in which the lower ends are substantially the same and are adjacent to each other, the classification is regarded as an arcade output. Also, if white lines and guardrails are detected but the signal layer is not at any position, the classification is classified as an expressway. Other examples of class separation include where you are now, noise reduction (output as false positive noise if a white line is detected at the position of the empty layer), and the risk of detected pedestrians. You may.
  • At least one of the combinations of the plurality of intermediate images is selected, and the neural network processing is executed on the selected at least one intermediate image, so that the processing amount of the image recognition processing is increased. Can be suppressed. Further, since at least one of the combinations of the plurality of intermediate images is selected and the neural network processing is executed on the selected at least one intermediate image, deterioration of the processing accuracy can be suppressed.
  • the outline of one aspect of this disclosure is as follows.
  • the segmentation image generated in the first processing unit (114) may include a plurality of intermediate images, and may further include an intermediate processing unit (116) for selecting at least one of a combination of the plurality of intermediate images.
  • the second processing unit (118) may execute the processing of the neural network on at least one intermediate image selected in the intermediate processing unit (116) as the segmentation image.
  • Example 3 Next, Example 3 will be described.
  • the image to be processed is processed by the completely convolutional neural network to generate a segmentation image, and then the segmentation image is processed by the convolutional neural network.
  • the processing of the intermediate processing unit 116 arranged between the first processing unit 114 and the second processing unit 118 is different from that of the second embodiment.
  • the processing apparatus 100 according to the third embodiment is the same type as that of FIG. Here, the differences from the past will be mainly explained.
  • the input unit 112 in FIG. 1 inputs the input image 140.
  • 8 (a)-(e) show the outline of the processing in the processing apparatus 100.
  • FIG. 8A shows an input image 140.
  • the input image 140 is the same as that in FIGS. 3 (a) and 6 (a). 8 (b)-(e) will be described later, and the process returns to FIG.
  • the trained first processing unit 114 executes the processing of the convolutional neural network in which the fully connected layer is excluded and the convolutional layer 142 and the pooling layer 144 are included in the input image 140.
  • the first processing unit 114 generates a segmentation image including an intermediate image corresponding to each channel. 8 (b)-(d) are the same as FIGS. 6 (b)-(d).
  • the intermediate processing unit 116 sets the transmittance for each of the plurality of intermediate images.
  • the transmittance corresponds to a weighting coefficient and is set in advance.
  • the intermediate processing unit 116 generates a composite image in which a plurality of intermediate images are combined while using the transmittance.
  • FIG. 8E shows a composite image 158 generated by the intermediate processing unit 116.
  • the composite image 158 includes a substrate 210, a first component 212a, a second component 212b, a fixing solder 214, and a solder hole 216.
  • the intermediate processing unit 116 outputs the composite image 158 to the second processing unit 118.
  • the second processing unit 118 executes neural network processing on the composite image 158 generated by the intermediate processing unit 116. Since the class classification in the second processing unit 118 may be the same as that in the first embodiment, the description thereof will be omitted here.
  • FIG. 9A shows an input image 140.
  • the input image 140 is an image of a person.
  • 9 (b)-(f) will be described later, and the process returns to FIG.
  • the first processing unit 114 executes the processing of the convolutional neural network in which the fully connected layer is excluded and the convolutional layer 142 and the pooling layer 144 are included in the input image 140.
  • the first processing unit 114 generates a segmentation image including an intermediate image corresponding to each channel.
  • FIG. 9B is a hair layer intermediate image 170 corresponding to the channel for “hair”
  • FIG. 9C is an eye layer intermediate image 172 corresponding to the channel for “eyes”.
  • 9 (d) is a mouth layer intermediate image 174 corresponding to the channel for the “mouth”
  • FIG. 9 (e) is an ear layer intermediate image 176 corresponding to the channel for the “ear”.
  • the hair layer intermediate image 170, the eye layer intermediate image 172, the mouth layer intermediate image 174, and the ear layer intermediate image 176 are collectively referred to as intermediate images.
  • the intermediate processing unit 116 sets the transmittance for each of the plurality of intermediate images.
  • the intermediate processing unit 116 generates a composite image in which a plurality of intermediate images are combined while using the transmittance.
  • FIG. 9 (f) shows a composite image 178 generated by the intermediate processing unit 116.
  • the composite image 178 can be said to be an exaggerated image.
  • the intermediate processing unit 116 outputs the composite image 178 to the second processing unit 118.
  • the second processing unit 118 executes neural network processing on the composite image 178 generated by the intermediate processing unit 116. As a class classification in the second processing unit 118, a person is discriminated.
  • each part including the decorative part
  • the positional relationship of each part may be used.
  • the judgment of the human condition health or not, emotion category, and as a channel, makeup components (lipstick, cheek, mascara, eye shadow, eyebrow shape) are detected, and then the beauty and ugliness of the makeup. Judgment, person recognition) may be included.
  • the neural network processing is executed on the composite image obtained by synthesizing a plurality of intermediate images while using the transmittance, it is possible to suppress an increase in the processing amount of the image recognition processing. Further, since the neural network processing is executed on the composite image obtained by synthesizing a plurality of intermediate images while using the transmittance, deterioration of the processing accuracy can be suppressed.
  • the outline of one aspect of this disclosure is as follows.
  • the segmentation image generated in the first processing unit (114) includes a plurality of intermediate images, a transmittance is set for each of the plurality of intermediate images, and a composite image obtained by synthesizing the plurality of intermediate images while using the transmittance is obtained.
  • An intermediate processing unit (116) to be generated may be further provided.
  • the second processing unit (118) may execute the neural network processing on the composite image generated in the intermediate processing unit (116) as the segmentation image.
  • the first processing unit 114 In the configuration of the first processing unit 114 in Examples 1 to 3, a plurality of convolution layers 142 and a plurality of pooling layers 144 are alternately arranged.
  • the present invention is not limited to this, and for example, the first processing unit 114 may have a configuration such as a GoodLeNet system or a DenseNet system. According to this modification, the degree of freedom of configuration can be improved.
  • the first processing unit 114 in this embodiment a convolutional neural network excluding the fully connected layer, that is, a completely convolutional neural network is used.
  • the present invention is not limited to this, and for example, the first processing unit 114 may manually manufacture a segmentation image by accepting a user's operation. Further, the first processing unit 114 may manufacture a segmentation image by a known image processing other than the neural network. According to this modification, the degree of freedom of configuration can be improved.
  • processing equipment 112 input unit, 114 first processing unit, 116 intermediate processing unit, 118 second processing unit, 120 output unit, 140 input image, 142 convolution layer, 144 pooling layer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
PCT/JP2021/024226 2020-10-29 2021-06-25 処理方法およびそれを利用した処理装置 Ceased WO2022091478A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2022558850A JP7557796B2 (ja) 2020-10-29 2021-06-25 処理方法およびそれを利用した処理装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-181068 2020-10-29
JP2020181068 2020-10-29

Publications (1)

Publication Number Publication Date
WO2022091478A1 true WO2022091478A1 (ja) 2022-05-05

Family

ID=81383856

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/024226 Ceased WO2022091478A1 (ja) 2020-10-29 2021-06-25 処理方法およびそれを利用した処理装置

Country Status (2)

Country Link
JP (1) JP7557796B2 (https=)
WO (1) WO2022091478A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025177908A1 (ja) * 2024-02-19 2025-08-28 マクセル株式会社 異常検出装置及び異常検出方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019067364A (ja) * 2017-09-28 2019-04-25 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． 走行車路を識別する方法及び装置
JP2019125207A (ja) * 2018-01-17 2019-07-25 株式会社東芝 ラベルデータ生成装置、ラベルデータ生成方法及びプログラム
JP2019139762A (ja) * 2018-02-12 2019-08-22 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． 車両走行のための情報を提供する方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019067364A (ja) * 2017-09-28 2019-04-25 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． 走行車路を識別する方法及び装置
JP2019125207A (ja) * 2018-01-17 2019-07-25 株式会社東芝 ラベルデータ生成装置、ラベルデータ生成方法及びプログラム
JP2019139762A (ja) * 2018-02-12 2019-08-22 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． 車両走行のための情報を提供する方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025177908A1 (ja) * 2024-02-19 2025-08-28 マクセル株式会社 異常検出装置及び異常検出方法

Also Published As

Publication number Publication date
JPWO2022091478A1 (https=) 2022-05-05
JP7557796B2 (ja) 2024-09-30

Similar Documents

Publication Publication Date Title
JP6952268B2 (ja) 処理方法およびそれを利用した処理装置
CN112489037B (zh) 缺陷检测及相关模型的训练方法、电子设备和存储装置
US20060115153A1 (en) Page background estimation using color, texture and edge features
KR102160690B1 (ko) 다초점 이미지 융합 방법
EP4189638B1 (en) METHOD AND ELECTRONIC DEVICE FOR MANAGING IMAGE ARTIFACTS
CN103185730A (zh) 缺陷分类法则建立方法、缺陷分类与致命缺陷判断方法
JP7463186B2 (ja) 情報処理装置、情報処理方法及びプログラム
Mousavi et al. Analysis of a robust edge detection system in different color spaces using color and depth images
JPWO2008093835A1 (ja) 画像処理方法
JP2023535084A (ja) 施設平面図に含まれた記号分析装置及び方法
JP7557796B2 (ja) 処理方法およびそれを利用した処理装置
CN119048939A (zh) 基于6g网络和dgs-yolo的输电线塔部件检测方法、系统、终端、介质及程序产品
CN117036905A (zh) 一种基于hsv色彩空间颜色注意力的胶囊内窥镜图像病灶识别方法
JP7627564B2 (ja) 画像処理装置、画像処理方法、学習モデル生成装置、学習モデル製造方法、学習済みモデル、プログラム
US11423635B2 (en) Method for processing image using fully connected convolutional neural network and circuit system
JP2840347B2 (ja) 基板実装検査装置
JP2009217606A (ja) 塗り絵用線画の生成方法および生成プログラム
TWI849972B (zh) 學習裝置、學習方法及學習程式
JP7407407B2 (ja) 処理方法およびそれを利用した処理装置
CN115482452A (zh) 一种基于深度卷积神经网络的pcb焊点识别方法
JP5265058B1 (ja) 商品画像処理装置、商品画像処理方法、情報記録媒体、ならびに、プログラム
CN112488940A (zh) 一种铁路机车部件图像边缘增强方法
JP6939186B2 (ja) 判定装置、判定方法および判定プログラム
JP7774231B2 (ja) 処理方法およびそれを利用した処理装置
WO2022074877A1 (ja) 処理方法およびそれを利用した処理装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21885601

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022558850

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21885601

Country of ref document: EP

Kind code of ref document: A1