WO2021179565A1 - 用于获取信息的方法及装置 - Google Patents

用于获取信息的方法及装置 Download PDF

Info

Publication number
WO2021179565A1
WO2021179565A1 PCT/CN2020/116527 CN2020116527W WO2021179565A1 WO 2021179565 A1 WO2021179565 A1 WO 2021179565A1 CN 2020116527 W CN2020116527 W CN 2020116527W WO 2021179565 A1 WO2021179565 A1 WO 2021179565A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
sample
detected
defect
feature
Prior art date
Application number
PCT/CN2020/116527
Other languages
English (en)
French (fr)
Inventor
苏业
任思可
聂磊
黄锋
Original Assignee
北京百度网讯科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京百度网讯科技有限公司 filed Critical 北京百度网讯科技有限公司
Priority to KR1020217024357A priority Critical patent/KR20210102458A/ko
Priority to JP2021541617A priority patent/JP2022526473A/ja
Priority to EP20913051.7A priority patent/EP3907697A4/en
Priority to US17/443,013 priority patent/US20220270228A1/en
Publication of WO2021179565A1 publication Critical patent/WO2021179565A1/zh

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • G06T7/001Industrial image inspection using an image reference approach
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30121CRT, LCD or plasma display
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30141Printed circuit board [PCB]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30148Semiconductor; IC; Wafer
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Definitions

  • the embodiments of the present disclosure relate to the field of computer technology, and in particular to methods and devices for obtaining information.
  • the first is to inspect the quality of items by technicians to check possible defects; the second is to inspect the quality of items through equipment, but the equipment is also set up based on the experience of the technicians to detect defects.
  • the effect is basically the same as manual. And there is no need to rest, and a large number of items can be tested for a long time.
  • the embodiments of the present disclosure propose methods and devices for obtaining information.
  • an embodiment of the present disclosure provides a method for acquiring information, the method comprising: acquiring at least one image feature from an image to be detected, wherein the image to be detected includes an image of the object to be detected, and the image feature is Used to characterize the surface feature information of the object to be inspected; import the image to be inspected and the at least one image feature into a pre-trained defect detection model to obtain defect information corresponding to the object to be inspected, wherein the defect detection model passes through the sample image , Sample image features and sample defect information are obtained through training, and are used to characterize the corresponding relationship between the above-mentioned image to be detected and at least one image feature.
  • acquiring at least one image feature from the image to be detected includes: acquiring a reference feature of the object to be detected, and the reference feature includes at least one of the following: color feature, structural feature, planar feature; based on the reference
  • the feature performs image processing on the above-mentioned image to be detected through a setting method to obtain a corresponding image feature, wherein the above-mentioned setting method includes at least one of the following: color contrast enhancement, filtering, and texture acquisition.
  • importing the image to be inspected and the at least one image feature into a pre-trained defect detection model to obtain defect information corresponding to the object to be inspected includes: importing the image to be inspected and the at least one image feature The corresponding input channel of the defect detection model obtains defect information corresponding to the object to be detected.
  • the above-mentioned defect detection model is obtained by training in the following steps: acquiring a plurality of sample information groups and sample defect information corresponding to each of the above-mentioned sample information groups, wherein the above-mentioned sample information group includes A sample image and at least one sample image feature corresponding to the sample image; each sample information group in the multiple sample information groups is used as input, and the sample defect corresponding to each sample information group in the multiple sample information groups The information is used as output, and the defect detection model is trained.
  • each sample information group in the multiple sample information groups is used as input, and the sample defect information corresponding to each sample information group in the multiple sample information groups is used as output, and the training obtains
  • the defect detection model includes: performing the following training steps: sequentially inputting each sample information group in the multiple sample information groups to the initialization defect detection model to obtain the corresponding sample information group in the multiple sample information groups To predict the defect information, compare the predicted defect information corresponding to each sample information group in the multiple sample information groups with the sample defect information corresponding to the sample information group to obtain the prediction accuracy rate of the initialized defect detection model, It is determined whether the prediction accuracy is greater than the preset accuracy threshold, and if it is greater than the preset accuracy threshold, the initialized defect detection model is used as the trained defect detection model.
  • each sample information group in the multiple sample information groups is used as input, and the sample defect information corresponding to each sample information group in the multiple sample information groups is used as output, and the training obtains
  • the defect detection model includes: in response to being not greater than the preset accuracy threshold, adjusting the parameters of the initialization defect detection model, and continuing to execute the training step.
  • an embodiment of the present disclosure provides a device for acquiring information, the device comprising: a feature acquiring unit configured to acquire at least one image feature from an image to be detected, wherein the image to be detected contains the image to be detected.
  • An image of an object to be detected, and the image feature is used to characterize the surface feature information of the object to be detected;
  • the defect information acquisition unit is configured to import the image to be detected and the at least one image feature into a pre-trained defect detection model to obtain a corresponding The defect information of an object is detected, wherein the defect detection model is obtained through training of sample images, sample image features, and sample defect information, and is used to characterize the corresponding relationship between the image to be detected and at least one image feature.
  • the feature acquisition unit includes: a reference feature acquisition subunit configured to acquire a reference feature of the object to be detected, the reference feature includes at least one of the following: color feature, structural feature, plane feature; image feature
  • the acquiring sub-unit is configured to perform image processing on the image to be detected in a setting manner based on the reference feature to obtain corresponding image features, wherein the setting method includes at least one of the following: color contrast enhancement, filtering, and texture acquisition .
  • the defect information acquiring unit includes: an information input subunit configured to import the image to be detected and the at least one image feature into the corresponding input channel of the defect detection model to obtain the defect corresponding to the object to be detected information.
  • the aforementioned device includes a defect detection model training unit configured to train a defect detection model
  • the aforementioned defect detection model training unit includes: a sample acquisition subunit configured to acquire a plurality of sample information groups and the aforementioned plurality of samples The sample defect information corresponding to each sample information group in the information group, wherein the sample information group includes a sample image and at least one sample image feature corresponding to the sample image; the model training subunit is configured to combine the multiple sample information
  • Each sample information group in the group is used as an input, and the sample defect information corresponding to each sample information group in the multiple sample information groups is used as an output, and a defect detection model is obtained by training.
  • the aforementioned model training subunit includes: a model training module configured to sequentially input each sample information group of the aforementioned multiple sample information groups into the initialization defect detection model to obtain the aforementioned multiple sample information groups The predicted defect information corresponding to each sample information group in the above multiple sample information groups is compared with the predicted defect information corresponding to each sample information group in the plurality of sample information groups and the sample defect information corresponding to the sample information group to obtain the above Initialize the prediction accuracy of the defect detection model to determine whether the prediction accuracy is greater than the preset accuracy threshold, and if it is greater than the preset accuracy threshold, the initialized defect detection model is used as the trained defect detection model.
  • the model training subunit includes: a parameter adjustment module configured to adjust the parameters of the initialization defect detection model in response to not greater than the preset accuracy threshold, and return to the model training module.
  • an embodiment of the present disclosure provides an electronic device, including: one or more processors; a memory, on which one or more programs are stored, when the above one or more programs are used by one or more of the above When the processor is executed, the one or more processors are caused to execute the method for obtaining information in the first aspect.
  • an embodiment of the present disclosure provides a computer-readable medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the method for obtaining information in the first aspect is implemented.
  • the method and device for obtaining information provided by the embodiments of the present disclosure first obtain at least one image feature from an image to be detected, and then import the image to be detected and at least one image feature into a pre-trained defect detection model to obtain a corresponding Detect the defect information of the object. In this way, it is beneficial to improve the accuracy of obtaining defect information of the detected object.
  • Fig. 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure can be applied;
  • Fig. 2 is a flowchart of an embodiment of a method for obtaining information according to the present disclosure
  • Fig. 3 is a schematic diagram of an application scenario of the method for obtaining information according to the present disclosure
  • Fig. 4 is a flowchart of an embodiment of a defect detection model training method according to the present disclosure
  • Fig. 5 is a schematic structural diagram of an embodiment of an apparatus for acquiring information according to the present disclosure
  • Fig. 6 is a schematic structural diagram of an electronic device suitable for implementing embodiments of the present disclosure.
  • FIG. 1 shows an exemplary system architecture 100 of a method for acquiring information or an apparatus for acquiring information to which an embodiment of the present disclosure can be applied.
  • the system architecture 100 may include image acquisition devices 101, 102, and 103, a network 104 and a server 105.
  • the network 104 is used to provide a medium for communication links between the image acquisition devices 101, 102, 103 and the server 105.
  • the network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, and so on.
  • the image acquisition devices 101, 102, and 103 interact with the server 105 through the network 104 to receive or send messages and so on.
  • Various image processing applications may be installed on the image capture devices 101, 102, 103, such as image capture applications, image adjustment applications, image compression applications, and image encryption applications.
  • the image acquisition devices 101, 102, 103 may be various electronic devices that have a display screen and support image acquisition, including remote cameras, digital cameras, surveillance cameras, and the like. There is no specific limitation here.
  • the server 105 may be a server that provides various services, for example, a defect detection server that provides support for the images to be detected from the image acquisition devices 101, 102, and 103.
  • the defect detection server can analyze and process the received data such as the image to be detected, and obtain defect information corresponding to the image to be detected.
  • the method for obtaining information provided by the embodiments of the present disclosure is generally executed by the server 105, and accordingly, the device for obtaining information is generally set in the server 105.
  • the server can be hardware or software.
  • the server can be implemented as a distributed server cluster composed of multiple servers, or as a single server.
  • the server is software, it can be implemented as multiple software or software modules (for example, to provide distributed services), or as a single software or software module, which is not specifically limited here.
  • the method for obtaining information includes the following steps:
  • Step 201 Obtain at least one image feature from the image to be detected.
  • the execution subject of the method for acquiring information may receive the image to be detected from the image acquisition devices 101, 102, 103 through a wired connection or a wireless connection.
  • the image acquisition devices 101, 102, and 103 of the present application can be set in multiple positions of the product production line to obtain the images to be inspected in each link of the product production. That is, the image to be detected in this application includes the image of the object to be detected.
  • wireless connection methods can include, but are not limited to, 3G/4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (Ultra Wideband) connection, and other wireless connection methods that are currently known or developed in the future .
  • this application does not directly inspect the image to be inspected to identify possible defects on the surface of the object to be inspected. Instead, it performs related processing on the image to be inspected.
  • the aforementioned image feature can be used to characterize the surface feature information of the aforementioned object to be detected.
  • acquiring at least one image feature from the image to be detected may include the following steps:
  • the first step is to obtain the reference feature of the object to be detected.
  • the execution subject may first identify the object to be detected from the image to be detected, and then obtain the reference feature corresponding to the object to be detected from a data storage device such as a database by way of query.
  • the objects to be inspected can be different objects such as notebook computers, motherboards, displays, silicon chips and so on. Therefore, different objects to be detected have their corresponding reference features.
  • the aforementioned reference feature may include at least one of the following: color feature, structural feature, planar feature (that is, multiple faces of the object to be detected), material feature, light sensing feature, and the like.
  • the reference feature can also be other features, depending on the actual object to be detected.
  • the second step is to perform image processing on the above-mentioned image to be detected through a setting method based on the above-mentioned reference feature to obtain the corresponding image feature.
  • the image acquisition devices 101, 102, and 103 acquire images to be detected, they can acquire images of reference features related to the object to be detected. That is, the image to be detected contains the image features of the reference feature related to the object to be detected.
  • the execution subject can process the image to be detected in different ways to extract corresponding image features from the image to be detected.
  • the above setting method may include at least one of the following: color contrast enhancement, filtering, and texture collection.
  • image features can be extracted in different ways based on the reference features of the object to be detected, and defects that are not easy to find with the naked eye (for example, the texture feature of the object to be detected) can be collected, which is beneficial to improve the accuracy and accuracy of obtaining defect information. Effectiveness.
  • Step 202 Import the image to be detected and the at least one image feature into a pre-trained defect detection model to obtain defect information corresponding to the object to be detected.
  • the execution subject can import the image to be detected and at least one image feature into the pre-trained defect detection model.
  • the defect detection model here is a model that can process the image to be inspected and the corresponding image features.
  • the defect detection model detects the defect information of the object to be inspected in the image to be inspected through the image to be inspected and image features. In this way, the accuracy of obtaining defect information is improved.
  • the above-mentioned defect detection model may be obtained through training of sample images, sample image features, and sample defect information, and is used to characterize the corresponding relationship between the above-mentioned image to be detected and at least one image feature.
  • the above importing the above-mentioned image to be detected and the above-mentioned at least one image feature into a pre-trained defect detection model to obtain defect information corresponding to the object to be detected may include: The image and the at least one image feature are imported into the corresponding input channel of the defect detection model to obtain defect information corresponding to the object to be detected.
  • the defect detection model of the present application can have multiple input channels, and each input channel can input different input information. After different input information is input to the defect detection model, it can be processed in the network structure or calculation unit corresponding to the defect detection model to achieve the fusion of multiple image features and the image to be detected, which is beneficial to improve the accuracy and accuracy of obtaining defect information. Effectiveness.
  • FIG. 3 is a schematic diagram of an application scenario of the method for obtaining information according to this embodiment.
  • the image acquisition device 101 sends the acquired image to be detected containing the XXX image of the object to be detected to the server 105 via the network 104.
  • the server 105 obtains at least one image feature from the image to be inspected, and then imports the image to be inspected and at least one image feature into the defect detection model, and obtains defect information corresponding to the object to be inspected may be: object to be inspected: XXX; defect information: 1, Surface scratches; 2. Uneven color; 3. Surface impurities.
  • the method provided by the foregoing embodiment of the present disclosure first obtains at least one image feature from the image to be detected, and then imports the image to be detected and the at least one image feature into a pre-trained defect detection model to obtain defect information corresponding to the object to be detected. In this way, it is beneficial to improve the accuracy of obtaining defect information of the detected object.
  • FIG. 4 shows a flow 400 of an embodiment of a method for training a defect detection model.
  • the process 400 of the defect detection model training method includes the following steps:
  • Step 401 Acquire multiple sample information groups and sample defect information corresponding to each of the multiple sample information groups.
  • the execution body of the defect detection model training method (for example, the server 105 shown in FIG. 1) can obtain multiple sample information groups and sample defects corresponding to each of the multiple sample information groups. information.
  • the execution body can obtain multiple sample information groups and play them for those skilled in the art.
  • the skilled person can mark sample defect information for each sample information group in the multiple sample information groups based on experience.
  • the above-mentioned sample information group may include a sample image and at least one sample image feature corresponding to the sample image.
  • the above-mentioned sample image features are obtained by the following steps: feature extraction is performed on the sample image through the above-mentioned setting method.
  • the execution subject can extract at least one image feature from the sample image through the above setting method. It should be noted that when the sample image has the corresponding image feature, the sample image feature can be extracted by the corresponding setting method; when the sample image does not have the corresponding image feature, the sample image feature cannot be extracted by the corresponding setting method . Therefore, the characteristics of the sample image are less than or equal to the above-mentioned setting method.
  • Step 402 Input each sample information group of the plurality of sample information groups into the initialization defect detection model in turn to obtain predicted defect information corresponding to each sample information group of the plurality of sample information groups.
  • the execution subject may sequentially input the sample image contained in each sample information group of the multiple sample information groups and at least one sample image feature corresponding to the sample image To initialize different input channels of the defect detection model, thereby obtaining predicted defect information corresponding to each sample information group in the multiple sample information groups.
  • the execution body can input each sample information group from the input side of the initialized defect detection model, sequentially process the parameters of each layer in the initialized defect detection model, and output from the output side of the initialized defect detection model, and output from the output side.
  • the information is the predicted defect information corresponding to the sample information group.
  • the initialization defect detection model may be an untrained deep learning model or an untrained deep learning model, each layer of which is set with initialization parameters, and the initialization parameters can be continuously adjusted during the training process of the initialization defect detection model.
  • Each sample image feature participates in data processing in the corresponding network structure or computing unit to realize the organic fusion of multiple sample image features and sample images.
  • Step 403 Compare the predicted defect information corresponding to each sample information group in the multiple sample information groups with the sample defect information corresponding to the sample information group to obtain the prediction accuracy rate of the initialized defect detection model.
  • the executive body may assign the corresponding prediction defect information to each sample information group in the multiple sample information groups.
  • the predicted defect information is compared with the sample defect information corresponding to the sample information group, so as to obtain the prediction accuracy rate of the initialized defect detection model. Specifically, if the predicted defect information corresponding to a sample information group is the same as or similar to the sample defect information corresponding to the sample information group, the initial defect detection model prediction is correct; if the predicted defect information corresponding to a sample information group is the same as the sample defect information If the sample defect information corresponding to the sample information group is different or not similar, the initial defect detection model predicts the error.
  • the execution body can calculate the ratio of the number of correct predictions to the total number of samples, and use this ratio as the prediction accuracy rate of the initialized defect detection model.
  • Step 404 Determine whether the prediction accuracy is greater than a preset accuracy threshold.
  • the execution entity may compare the prediction accuracy of the initialized defect detection model with a preset accuracy threshold. If it is greater than the preset accuracy threshold, step 405 is executed; if it is not greater than the preset accuracy threshold, step 406 is executed.
  • Step 405 Use the above-mentioned initialized defect detection model as a defect detection model completed by training.
  • the execution body can use the initialized defect detection model as the completed defect detection. Model.
  • Step 406 Adjust the parameters of the above-mentioned initialization defect detection model.
  • the execution subject can adjust the parameters of the initialized defect detection model and return to step 402 until the training can characterize the above-mentioned waiting It is up to the deep learning model that detects the correspondence between an image and at least one image feature.
  • the present disclosure provides an embodiment of a device for obtaining information.
  • the device embodiment corresponds to the method embodiment shown in FIG.
  • the device can be applied to various electronic devices.
  • the apparatus 500 for acquiring information in this embodiment may include: a feature acquiring unit 501 and a defect information acquiring unit 502.
  • the feature acquiring unit 501 is configured to acquire at least one image feature from the image to be inspected, wherein the image to be inspected includes an image of the object to be inspected, and the image feature is used to characterize the surface feature information of the object to be inspected; defect information acquisition
  • the unit 502 is configured to import the image to be detected and the at least one image feature into a pre-trained defect detection model to obtain defect information corresponding to the object to be detected, wherein the defect detection model is based on the sample image, the sample image feature, and the sample defect.
  • the information is obtained through training and is used to characterize the corresponding relationship between the image to be detected and at least one image feature.
  • the feature acquisition unit 501 may include: a reference feature acquisition subunit (not shown in the figure) and an image feature acquisition subunit (not shown in the figure).
  • the reference feature acquisition subunit is configured to acquire the reference features of the object to be detected, and the reference feature includes at least one of the following: color features, structural features, and planar features;
  • the image feature acquisition subunit is configured to pass based on the reference features.
  • the setting mode performs image processing on the above-mentioned image to be detected to obtain corresponding image characteristics, wherein the above-mentioned setting mode includes at least one of the following: color contrast enhancement, filtering, and texture acquisition.
  • the defect information acquiring unit 502 may include: an information input subunit (not shown in the figure) configured to import the image to be detected and the at least one image feature into the image The corresponding input channel of the defect detection model obtains defect information corresponding to the object to be detected.
  • the foregoing apparatus 500 for obtaining information may include a defect detection model training unit (not shown in the figure), configured to train a defect detection model, and the foregoing defect detection model training unit It may include: a sample acquisition subunit (not shown in the figure) and a model training subunit (not shown in the figure).
  • the sample acquisition subunit is configured to acquire a plurality of sample information groups and sample defect information corresponding to each sample information group in the plurality of sample information groups, wherein the sample information group includes sample images and corresponding sample information groups. At least one sample image feature;
  • the model training subunit is configured to take each sample information group of the multiple sample information groups as input, and take the sample defect corresponding to each sample information group of the multiple sample information groups The information is used as output, and the defect detection model is trained.
  • the above-mentioned model training subunit may include: a model training module (not shown in the figure) configured to sequentially arrange each of the above-mentioned multiple sample information groups Input to the initial defect detection model to obtain the predicted defect information corresponding to each sample information group in the multiple sample information groups, and compare the predicted defect information corresponding to each sample information group in the multiple sample information groups with the The sample defect information corresponding to the sample information group is compared to obtain the prediction accuracy of the initialized defect detection model, and it is determined whether the prediction accuracy is greater than the preset accuracy threshold. If it is greater than the preset accuracy threshold, the initialization is performed The defect detection model is used as the defect detection model completed by training.
  • a model training module (not shown in the figure) configured to sequentially arrange each of the above-mentioned multiple sample information groups Input to the initial defect detection model to obtain the predicted defect information corresponding to each sample information group in the multiple sample information groups, and compare the predicted defect information corresponding to each sample information group in the multiple sample information groups with the The sample defect information
  • the above-mentioned model training subunit may include: a parameter adjustment module (not shown in the figure), configured to adjust the above-mentioned initialization defect in response to being not greater than the above-mentioned preset accuracy threshold Detect the parameters of the model and return to the above model training module.
  • a parameter adjustment module (not shown in the figure), configured to adjust the above-mentioned initialization defect in response to being not greater than the above-mentioned preset accuracy threshold Detect the parameters of the model and return to the above model training module.
  • This embodiment also provides an electronic device, including: one or more processors; a memory, on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, The above-mentioned one or more processors are caused to execute the above-mentioned method for obtaining information.
  • This embodiment also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the foregoing method for obtaining information is implemented.
  • FIG. 6 shows a schematic structural diagram of a computer system 600 of an electronic device (for example, the server 105 in FIG. 1) suitable for implementing an embodiment of the present disclosure.
  • the electronic device shown in FIG. 6 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present disclosure.
  • the electronic device 600 may include a processing device (such as a central processing unit, a graphics processor, etc.) 601, which can be loaded into a random access device according to a program stored in a read-only memory (ROM) 602 or from a storage device 608.
  • the program in the memory (RAM) 603 executes various appropriate actions and processing.
  • various programs and data required for the operation of the electronic device 600 are also stored.
  • the processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604.
  • An input/output (I/O) interface 605 is also connected to the bus 604.
  • the following devices can be connected to the I/O interface 605: including input devices 606 such as touch screens, touch pads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; including, for example, liquid crystal displays (LCD), speakers, vibration An output device 607 such as a device; a storage device 608 such as a magnetic tape, a hard disk, etc.; and a communication device 609.
  • the communication device 609 may allow the electronic device 600 to perform wireless or wired communication with other devices to exchange data.
  • FIG. 6 shows an electronic device 600 having various devices, it should be understood that it is not required to implement or have all of the illustrated devices. It may be implemented alternatively or provided with more or fewer devices. Each block shown in FIG. 6 can represent one device, or can represent multiple devices as needed.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program may be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602.
  • the processing device 601 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the above-mentioned computer-readable medium in the embodiment of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above.
  • Computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable removable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as a part of a carrier wave, and a computer-readable program code is carried therein.
  • This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium.
  • the computer-readable signal medium may send, propagate, or transmit the program for use by or in combination with the instruction execution system, apparatus, or device .
  • the program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist alone without being assembled into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs.
  • the electronic device When the above-mentioned one or more programs are executed by the electronic device, the electronic device: An image of the object to be detected, the above image features are used to characterize the surface feature information of the object to be detected; the image to be detected and the at least one image feature are imported into a pre-trained defect detection model to obtain defect information corresponding to the object to be detected, wherein
  • the above-mentioned defect detection model is obtained through training of sample images, sample image features, and sample defect information, and is used to characterize the corresponding relationship between the above-mentioned image to be detected and at least one image feature.
  • the computer program code for performing the operations of the embodiments of the present disclosure may be written in one or more programming languages or a combination thereof.
  • the above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and Including conventional procedural programming languages-such as "C" language or similar programming languages.
  • the program code can be executed entirely on the user's computer, partly on the user's computer, executed as an independent software package, partly on the user's computer and partly executed on a remote computer, or entirely executed on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to pass Internet connection).
  • LAN local area network
  • WAN wide area network
  • each block in the flowchart or block diagram can represent a module, program segment, or part of code, and the module, program segment, or part of code contains one or more for realizing the specified logic function.
  • Executable instructions can also occur in a different order from the order marked in the drawings. For example, two blocks shown one after another can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified function or operation Or it can be realized by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure can be implemented in software or hardware.
  • the described unit may also be provided in the processor, for example, it may be described as: a processor includes a feature acquisition unit and a defect information acquisition unit.
  • the names of these units do not constitute a limitation on the unit itself under certain circumstances.
  • the feature acquisition unit can also be described as "a unit that acquires multiple image features of the object to be detected from the image to be detected.”

Landscapes

  • Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

用于获取信息的方法及装置,涉及云计算技术领域。该方法包括:从待检测图像中获取至少一条图像特征(201),其中,上述待检测图像包含待检测物体图像,上述图像特征用于表征上述待检测物体的表面特征信息;将上述待检测图像和上述至少一条图像特征导入预先训练的缺陷检测模型,得到对应上述待检测物体的缺陷信息(202),其中,上述缺陷检测模型通过样本图像、样本图像特征和样本缺陷信息训练得到,用于表征上述待检测图像和至少一条图像特征的对应关系。该方法有利于提高获取检测待检测物体的缺陷信息的准确性。

Description

用于获取信息的方法及装置
本专利申请要求于2020年03月11日提交的、申请号为202010165825.4、发明名称为“用于获取信息的方法及装置”的中国专利申请的优先权,该申请的全文以引用的方式并入本申请中。
技术领域
本公开的实施例涉及计算机技术领域,具体涉及用于获取信息的方法及装置。
背景技术
随着科技的发展,很多物品可以进行大规模的工业生产。生产物品的过程中,可能存在诸多因素导致物品出现缺陷,进而影响到物品的质量出现问题。
为此,通常采用两种方式对物品的质量进行检测。第一种为通过技术人员对物品的质量进行检测,以查询可能的缺陷;第二种为通过设备对物品的质量进行检测,但设备也是基于技术人员的经验来进行相关的设置,检测缺陷的效果与人工基本相同。且无需休息,可以长时间对海量的物品进行检测。
发明内容
本公开的实施例提出了用于获取信息的方法及装置。
第一方面,本公开的实施例提供了一种用于获取信息的方法,该方法包括:从待检测图像中获取至少一条图像特征,其中,上述待检测图像包含待检测物体图像,上述图像特征用于表征上述待检测物体的表面特征信息;将上述待检测图像和上述至少一条图像特征导入预先训练的缺陷检测模型,得到对应上述待检测物体的缺陷信息,其中,上述缺陷检测模型通过样本图像、样本图像特征和样本缺陷信息训练 得到,用于表征上述待检测图像和至少一条图像特征的对应关系。
在一些实施例中,上述从待检测图像中获取至少一条图像特征,包括:获取上述待检测物体的基准特征,上述基准特征包括以下至少一项:颜色特征、结构特征、平面特征;基于上述基准特征通过设定方式对上述待检测图像进行图像处理,得到对应的图像特征,其中,上述设定方式包括以下至少一项:色彩对比增强、滤波、纹理采集。
在一些实施例中,上述将上述待检测图像和上述至少一条图像特征导入预先训练的缺陷检测模型,得到对应上述待检测物体的缺陷信息,包括:将上述待检测图像和上述至少一条图像特征导入上述缺陷检测模型的对应输入通道,得到对应上述待检测物体的缺陷信息。
在一些实施例中,上述缺陷检测模型通过以下步骤训练得到:获取多个样本信息组和上述多个样本信息组中的每个样本信息组所对应的样本缺陷信息,其中,上述样本信息组包括样本图像和对应样本图像的至少一条样本图像特征;将上述多个样本信息组中的每个样本信息组作为输入,将上述多个样本信息组中的每个样本信息组所对应的上述样本缺陷信息作为输出,训练得到缺陷检测模型。
在一些实施例中,上述将上述多个样本信息组中的每个样本信息组作为输入,将上述多个样本信息组中的每个样本信息组所对应的上述样本缺陷信息作为输出,训练得到缺陷检测模型,包括:执行以下训练步骤:将上述多个样本信息组中的每个样本信息组依次输入至初始化缺陷检测模型,得到上述多个样本信息组中的每个样本信息组所对应的预测缺陷信息,将上述多个样本信息组中的每个样本信息组所对应的预测缺陷信息与该样本信息组所对应的上述样本缺陷信息进行比较,得到上述初始化缺陷检测模型的预测准确率,确定上述预测准确率是否大于预设准确率阈值,若大于上述预设准确率阈值,则将上述初始化缺陷检测模型作为训练完成的缺陷检测模型。
在一些实施例中,上述将上述多个样本信息组中的每个样本信息组作为输入,将上述多个样本信息组中的每个样本信息组所对应的上述样本缺陷信息作为输出,训练得到缺陷检测模型,包括:响应于不大于上述预设准确率阈值,调整上述初始化缺陷检测模型的参数,并 继续执行上述训练步骤。
第二方面,本公开的实施例提供了一种用于获取信息的装置,该装置包括:特征获取单元,被配置成从待检测图像中获取至少一条图像特征,其中,上述待检测图像包含待检测物体图像,上述图像特征用于表征上述待检测物体的表面特征信息;缺陷信息获取单元,被配置成将上述待检测图像和上述至少一条图像特征导入预先训练的缺陷检测模型,得到对应上述待检测物体的缺陷信息,其中,上述缺陷检测模型通过样本图像、样本图像特征和样本缺陷信息训练得到,用于表征上述待检测图像和至少一条图像特征的对应关系。
在一些实施例中,上述特征获取单元包括:基准特征获取子单元,被配置成获取上述待检测物体的基准特征,上述基准特征包括以下至少一项:颜色特征、结构特征、平面特征;图像特征获取子单元,被配置成基于上述基准特征通过设定方式对上述待检测图像进行图像处理,得到对应的图像特征,其中,上述设定方式包括以下至少一项:色彩对比增强、滤波、纹理采集。
在一些实施例中,上述缺陷信息获取单元包括:信息输入子单元,被配置成将上述待检测图像和上述至少一条图像特征导入上述缺陷检测模型的对应输入通道,得到对应上述待检测物体的缺陷信息。
在一些实施例中,上述装置包括缺陷检测模型训练单元,被配置成训练缺陷检测模型,上述缺陷检测模型训练单元包括:样本获取子单元,被配置成获取多个样本信息组和上述多个样本信息组中的每个样本信息组所对应的样本缺陷信息,其中,上述样本信息组包括样本图像和对应样本图像的至少一条样本图像特征;模型训练子单元,被配置成将上述多个样本信息组中的每个样本信息组作为输入,将上述多个样本信息组中的每个样本信息组所对应的上述样本缺陷信息作为输出,训练得到缺陷检测模型。
在一些实施例中,上述模型训练子单元包括:模型训练模块,被配置成将上述多个样本信息组中的每个样本信息组依次输入至初始化缺陷检测模型,得到上述多个样本信息组中的每个样本信息组所对应的预测缺陷信息,将上述多个样本信息组中的每个样本信息组所对应 的预测缺陷信息与该样本信息组所对应的上述样本缺陷信息进行比较,得到上述初始化缺陷检测模型的预测准确率,确定上述预测准确率是否大于预设准确率阈值,若大于上述预设准确率阈值,则将上述初始化缺陷检测模型作为训练完成的缺陷检测模型。
在一些实施例中,上述模型训练子单元包括:参数调整模块,响应于不大于上述预设准确率阈值,被配置成调整上述初始化缺陷检测模型的参数,返回上述模型训练模块。
第三方面,本公开的实施例提供了一种电子设备,包括:一个或多个处理器;存储器,其上存储有一个或多个程序,当上述一个或多个程序被上述一个或多个处理器执行时,使得上述一个或多个处理器执行上述第一方面的用于获取信息的方法。
第四方面,本公开的实施例提供了一种计算机可读介质,其上存储有计算机程序,其特征在于,该程序被处理器执行时实现上述第一方面的用于获取信息的方法。
本公开的实施例提供的用于获取信息的方法及装置,首先从待检测图像中获取至少一条图像特征,然后将待检测图像和至少一条图像特征导入预先训练的缺陷检测模型,得到对应上述待检测物体的缺陷信息。如此,有利于提高获取检测待检测物体的缺陷信息的准确性。
附图说明
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述,本公开的其它特征、目的和优点将会变得更明显:
图1是本公开的一个实施例可以应用于其中的示例性系统架构图;
图2是根据本公开的用于获取信息的方法的一个实施例的流程图;
图3是根据本公开的用于获取信息的方法的一个应用场景的示意图;
图4是根据本公开的缺陷检测模型训练方法的一个实施例的流程图;
图5是根据本公开的用于获取信息的装置的一个实施例的结构示意图;
图6是适于用来实现本公开的实施例的电子设备结构示意图。
具体实施方式
下面结合附图和实施例对本公开作进一步的详细说明。可以理解的是,此处所描述的具体实施例仅仅用于解释相关发明,而非对该发明的限定。另外还需要说明的是,为了便于描述,附图中仅示出了与有关发明相关的部分。
需要说明的是,在不冲突的情况下,本公开中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本公开。
图1示出了可以应用本公开的实施例的用于获取信息的方法或用于获取信息的装置的示例性系统架构100。
如图1所示,系统架构100可以包括图像采集设备101、102、103,网络104和服务器105。网络104用以在图像采集设备101、102、103和服务器105之间提供通信链路的介质。网络104可以包括各种连接类型,例如有线、无线通信链路或者光纤电缆等等。
图像采集设备101、102、103通过网络104与服务器105交互,以接收或发送消息等。图像采集设备101、102、103上可以安装有各种图像处理应用,例如图像采集应用、图像调整应用、图像压缩应用、图像加密应用等。
图像采集设备101、102、103可以是具有显示屏并且支持图像采集的各种电子设备,包括远程摄像头、数字摄像头、监控摄像头等。在此不做具体限定。
服务器105可以是提供各种服务的服务器,例如对图像采集设备101、102、103发来的待检测图像提供支持的缺陷检测服务器。缺陷检测服务器可以对接收到的待检测图像等数据进行分析等处理,并获取到对应待检测图像的缺陷信息。
需要说明的是,本公开的实施例所提供的用于获取信息的方法一 般由服务器105执行,相应地,用于获取信息的装置一般设置于服务器105中。
需要说明的是,服务器可以是硬件,也可以是软件。当服务器为硬件时,可以实现成多个服务器组成的分布式服务器集群,也可以实现成单个服务器。当服务器为软件时,可以实现成多个软件或软件模块(例如用来提供分布式服务),也可以实现成单个软件或软件模块,在此不做具体限定。
应该理解,图1中的图像采集设备、网络和服务器的数目仅仅是示意性的。根据实现需要,可以具有任意数目的图像采集设备、网络和服务器。
继续参考图2,示出了根据本公开的用于获取信息的方法的一个实施例的流程200。该用于获取信息的方法包括以下步骤:
步骤201,从待检测图像中获取至少一条图像特征。
在本实施例中,用于获取信息的方法的执行主体(例如图1所示的服务器105)可以通过有线连接方式或者无线连接方式从图像采集设备101、102、103接收待检测图像。其中,本申请的图像采集设备101、102、103可以设置在物品生产线的多个位置,以获取物品在生产的各个环节的待检测图像。即,本申请的待检测图像包含待检测物体图像。需要指出的是,上述无线连接方式可以包括但不限于3G/4G连接、WiFi连接、蓝牙连接、WiMAX连接、Zigbee连接、UWB(Ultra Wideband)连接、以及其他现在已知或将来开发的无线连接方式。
现有通过设备检测物品缺陷的过程中,由于设备的拍摄角度、无法人工触摸物品等原因,使得设备不易对某些光线不敏感或需要手动触摸等缺陷进行准确检测。
为此,本申请在获取到待检测图像后,并不是直接对待检测图像进行检测,以识别出待检测物体的表面可能存在的缺陷,而是对待检测图像进行相关的处理,以从待检测图像中获取至少一条图像特征。其中,上述图像特征可以用于表征上述待检测物体的表面特征信息。
在本实施例的一些可选的实现方式中,上述从待检测图像中获取至少一条图像特征,可以包括以下步骤:
第一步,获取上述待检测物体的基准特征。
执行主体可以首先从待检测图像中识别出待检测物体,然后通过查询的方式从数据库等数据存储设备获取到对应待检测物体的基准特征。待检测物体可以是笔记本电脑、主板、显示器、硅晶片等不同的物体。因此,不同的待检测物体有其对应的基准特征。其中,上述基准特征可以包括以下至少一项:颜色特征、结构特征、平面特征(即待检测物体的多个面)、材料特征、光感应特征等。基准特征还可以是其他特征,具体视实际的待检测物体而定。
第二步,基于上述基准特征通过设定方式对上述待检测图像进行图像处理,得到对应的图像特征。
图像采集设备101、102、103在采集待检测图像时,可以获取到与待检测物体相关的基准特征的图像。即待检测图像包含了待检测物体相关的基准特征的图像特征。对于不同的基准特征,执行主体可以采用不同的方式对待检测图像进行处理,以从待检测图像中提取对应的图像特征。其中,上述设定方式可以包括以下至少一项:色彩对比增强、滤波、纹理采集。如此,可以针对待检测物体自身的基准特征采取不同的方式提取图像特征,并且可以采集到肉眼不易发现的缺陷(例如可以是待检测物体的纹理特征),有利于提高获取缺陷信息的准确性和有效性。
步骤202,将上述待检测图像和上述至少一条图像特征导入预先训练的缺陷检测模型,得到对应上述待检测物体的缺陷信息。
得到图像特征后,执行主体可以将待检测图像和至少一条图像特征导入预先训练的缺陷检测模型。由上述描述可知,不同的待检测物体具有不同的基准特征和图像特征,因此,此处的缺陷检测模型为能够对待检测图像和对应的图像特征进行处理的模型。缺陷检测模型通过待检测图像和图像特征来检测待检测图像中待检测物体的缺陷信息。如此,提高了获取缺陷信息的准确性。其中,上述缺陷检测模型可以通过样本图像、样本图像特征和样本缺陷信息训练得到,用于表征上述待检测图像和至少一条图像特征的对应关系。
在本实施例的一些可选的实现方式中,上述将上述待检测图像和 上述至少一条图像特征导入预先训练的缺陷检测模型,得到对应上述待检测物体的缺陷信息,可以包括:将上述待检测图像和上述至少一条图像特征导入上述缺陷检测模型的对应输入通道,得到对应上述待检测物体的缺陷信息。
本申请的缺陷检测模型可以有多个输入通道,每个输入通道可以输入不同的输入信息。不同的输入信息被输入至缺陷检测模型后,可以在缺陷检测模型对应的网络结构或计算单元进行处理,实现了多个图像特征和待检测图像的融合,有利于提高获取缺陷信息的准确性和有效性。
继续参见图3,图3是根据本实施例的用于获取信息的方法的应用场景的一个示意图。在图3的应用场景中,图像采集设备101将采集到的、包含待检测物体XXX图像的待检测图像通过网络104发送给服务器105。服务器105从待检测图像中获取至少一条图像特征,然后将待检测图像和至少一条图像特征导入缺陷检测模型,得到对应待检测物体的缺陷信息可以为:待检测物:XXX;缺陷信息:1、表面划痕;2、颜色不均;3、表面杂质。
本公开的上述实施例提供的方法首先从待检测图像中获取至少一条图像特征,然后将待检测图像和至少一条图像特征导入预先训练的缺陷检测模型,得到对应上述待检测物体的缺陷信息。如此,有利于提高获取检测待检测物体的缺陷信息的准确性。
进一步参考图4,其示出了缺陷检测模型训练方法的一个实施例的流程400。该缺陷检测模型训练方法的流程400,包括以下步骤:
步骤401,获取多个样本信息组和上述多个样本信息组中的每个样本信息组所对应的样本缺陷信息。
在本实施例中,缺陷检测模型训练方法的执行主体(例如图1所示的服务器105)可以获取多个样本信息组和上述多个样本信息组中的每个样本信息组所对应的样本缺陷信息。
在本实施例中,执行主体可以获取多个样本信息组,并为本领域技术人员播放,本领域技术人员可以根据经验对多个样本信息组中的每个样本信息组标注样本缺陷信息。其中,上述样本信息组可以包括 样本图像和对应样本图像的至少一条样本图像特征。
在本实施例的一些可选的实现方式中,上述样本图像特征通过以下步骤得到:通过上述设定方式对样本图像进行特征提取。
执行主体可以通过上述的设定方式从样本图像中提取出至少一条图像特征。需要说明的是,样本图像存在对应的图像特征时,通过对应的设定方式能够提取出样本图像特征;样本图像不存在对应的图像特征时,无法通过对应的设定方式能够提取出样本图像特征。因此,样本图像特征的小于等于上述的设定方式。
步骤402,将上述多个样本信息组中的每个样本信息组依次输入至初始化缺陷检测模型,得到上述多个样本信息组中的每个样本信息组所对应的预测缺陷信息。
在本实施例中,基于步骤401所获取的多个样本信息组,执行主体可以将多个样本信息组中的每个样本信息组包含的样本图像和对应样本图像的至少一条样本图像特征依次输入至初始化缺陷检测模型的不同输入通道,从而得到多个样本信息组中的每个样本信息组所对应的预测缺陷信息。这里,执行主体可以将每个样本信息组从初始化缺陷检测模型的输入侧输入,依次经过初始化缺陷检测模型中的各层的参数的处理,并从初始化缺陷检测模型的输出侧输出,输出侧输出的信息即为该样本信息组所对应的预测缺陷信息。其中,初始化缺陷检测模型可以是未经训练的深度学习模型或未训练完成的深度学习模型,其各层设置有初始化参数,初始化参数在初始化缺陷检测模型的训练过程中可以被不断地调整。每一条样本图像特征在对应的网络结构或计算单元参与数据处理,以实现多个样本图像特征和样本图像的有机融合。
步骤403,将上述多个样本信息组中的每个样本信息组所对应的预测缺陷信息与该样本信息组所对应的上述样本缺陷信息进行比较,得到上述初始化缺陷检测模型的预测准确率。
在本实施例中,基于步骤402所得到的多个样本信息组中的每个样本信息组所对应的预测缺陷信息,执行主体可以将多个样本信息组中的每个样本信息组所对应的预测缺陷信息与该样本信息组所对应的 样本缺陷信息进行比较,从而得到初始化缺陷检测模型的预测准确率。具体地,若一个样本信息组所对应的预测缺陷信息与该样本信息组所对应的样本缺陷信息相同或相近,则初始化缺陷检测模型预测正确;若一个样本信息组所对应的预测缺陷信息与该样本信息组所对应的样本缺陷信息不同或不相近,则初始化缺陷检测模型预测错误。这里,执行主体可以计算预测正确的数目与样本总数的比值,并将该比值作为初始化缺陷检测模型的预测准确率。
步骤404,确定上述预测准确率是否大于预设准确率阈值。
在本实施例中,基于步骤403所得到的初始化缺陷检测模型的预测准确率,执行主体可以将初始化缺陷检测模型的预测准确率与预设准确率阈值进行比较。若大于预设准确率阈值,则执行步骤405;若不大于预设准确率阈值,则执行步骤406。
步骤405,将上述初始化缺陷检测模型作为训练完成的缺陷检测模型。
在本实施例中,在初始化缺陷检测模型的预测准确率大于预设准确率阈值的情况下,说明该缺陷检测模型训练完成,此时,执行主体可以将初始化缺陷检测模型作为训练完成的缺陷检测模型。
步骤406,调整上述初始化缺陷检测模型的参数。
在本实施例中,在初始化缺陷检测模型的预测准确率不大于预设准确率阈值的情况下,执行主体可以调整初始化缺陷检测模型的参数,并返回执行步骤402,直至训练出能够表征上述待检测图像和至少一条图像特征的对应关系的深度学习模型为止。
进一步参考图5,作为对上述各图所示方法的实现,本公开提供了一种用于获取信息的装置的一个实施例,该装置实施例与图2所示的方法实施例相对应,该装置具体可以应用于各种电子设备中。
如图5所示,本实施例的用于获取信息的装置500可以包括:特征获取单元501和缺陷信息获取单元502。其中,特征获取单元501被配置成从待检测图像中获取至少一条图像特征,其中,上述待检测图像包含待检测物体图像,上述图像特征用于表征上述待检测物体的表面特征信息;缺陷信息获取单元502被配置成将上述待检测图像和 上述至少一条图像特征导入预先训练的缺陷检测模型,得到对应上述待检测物体的缺陷信息,其中,上述缺陷检测模型通过样本图像、样本图像特征和样本缺陷信息训练得到,用于表征上述待检测图像和至少一条图像特征的对应关系。
在本实施例的一些可选的实现方式中,上述特征获取单元501可以包括:基准特征获取子单元(图中未视出)和图像特征获取子单元(图中未视出)。其中,基准特征获取子单元被配置成获取上述待检测物体的基准特征,上述基准特征包括以下至少一项:颜色特征、结构特征、平面特征;图像特征获取子单元被配置成基于上述基准特征通过设定方式对上述待检测图像进行图像处理,得到对应的图像特征,其中,上述设定方式包括以下至少一项:色彩对比增强、滤波、纹理采集。
在本实施例的一些可选的实现方式中,上述缺陷信息获取单元502可以包括:信息输入子单元(图中未视出),被配置成将上述待检测图像和上述至少一条图像特征导入上述缺陷检测模型的对应输入通道,得到对应上述待检测物体的缺陷信息。
在本实施例的一些可选的实现方式中,上述用于获取信息的装置500可以包括缺陷检测模型训练单元(图中未视出),被配置成训练缺陷检测模型,上述缺陷检测模型训练单元可以包括:样本获取子单元(图中未视出)和模型训练子单元(图中未视出)。其中,样本获取子单元被配置成获取多个样本信息组和上述多个样本信息组中的每个样本信息组所对应的样本缺陷信息,其中,上述样本信息组包括样本图像和对应样本图像的至少一条样本图像特征;模型训练子单元被配置成将上述多个样本信息组中的每个样本信息组作为输入,将上述多个样本信息组中的每个样本信息组所对应的上述样本缺陷信息作为输出,训练得到缺陷检测模型。
在本实施例的一些可选的实现方式中,上述模型训练子单元可以包括:模型训练模块(图中未视出),被配置成将上述多个样本信息组中的每个样本信息组依次输入至初始化缺陷检测模型,得到上述多个样本信息组中的每个样本信息组所对应的预测缺陷信息,将上述多个 样本信息组中的每个样本信息组所对应的预测缺陷信息与该样本信息组所对应的上述样本缺陷信息进行比较,得到上述初始化缺陷检测模型的预测准确率,确定上述预测准确率是否大于预设准确率阈值,若大于上述预设准确率阈值,则将上述初始化缺陷检测模型作为训练完成的缺陷检测模型。
在本实施例的一些可选的实现方式中,上述模型训练子单元可以包括:参数调整模块(图中未视出),响应于不大于上述预设准确率阈值,被配置成调整上述初始化缺陷检测模型的参数,返回上述模型训练模块。
本实施例还提供了一种电子设备,包括:一个或多个处理器;存储器,其上存储有一个或多个程序,当上述一个或多个程序被上述一个或多个处理器执行时,使得上述一个或多个处理器执行上述的用于获取信息的方法。
本实施例还提供了一种计算机可读介质,其上存储有计算机程序,该程序被处理器执行时实现上述的用于获取信息的方法。
下面参考图6,其示出了适于用来实现本公开的实施例的电子设备(例如,图1中的服务器105)的计算机系统600的结构示意图。图6示出的电子设备仅仅是一个示例,不应对本公开的实施例的功能和使用范围带来任何限制。
如图6所示,电子设备600可以包括处理装置(例如中央处理器、图形处理器等)601,其可以根据存储在只读存储器(ROM)602中的程序或者从存储装置608加载到随机访问存储器(RAM)603中的程序而执行各种适当的动作和处理。在RAM 603中,还存储有电子设备600操作所需的各种程序和数据。处理装置601、ROM 602以及RAM 603通过总线604彼此相连。输入/输出(I/O)接口605也连接至总线604。
通常,以下装置可以连接至I/O接口605:包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置606;包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置607;包括例如磁带、硬盘等的存储装置608;以及通信装置609。通信装置 609可以允许电子设备600与其他设备进行无线或有线通信以交换数据。虽然图6示出了具有各种装置的电子设备600,但是应理解的是,并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。图6中示出的每个方框可以代表一个装置,也可以根据需要代表多个装置。
特别地,根据本公开的实施例,上文参考流程图描述的过程可以被实现为计算机软件程序。例如,本公开的实施例包括一种计算机程序产品,其包括承载在计算机可读介质上的计算机程序,该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中,该计算机程序可以通过通信装置609从网络上被下载和安装,或者从存储装置608被安装,或者从ROM 602被安装。在该计算机程序被处理装置601执行时,执行本公开的实施例的方法中限定的上述功能。
需要说明的是,本公开的实施例上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件,或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于:具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开的实施例中,计算机可读存储介质可以是任何包含或存储程序的有形介质,该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开的实施例中,计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号,其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式,包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质,该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当 的介质传输,包括但不限于:电线、光缆、RF(射频)等等,或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的;也可以是单独存在,而未装配入该电子设备中。上述计算机可读介质承载有一个或者多个程序,当上述一个或者多个程序被该电子设备执行时,使得该电子设备:从待检测图像中获取至少一条图像特征,其中,上述待检测图像包含待检测物体图像,上述图像特征用于表征上述待检测物体的表面特征信息;将上述待检测图像和上述至少一条图像特征导入预先训练的缺陷检测模型,得到对应上述待检测物体的缺陷信息,其中,上述缺陷检测模型通过样本图像、样本图像特征和样本缺陷信息训练得到,用于表征上述待检测图像和至少一条图像特征的对应关系。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的实施例的操作的计算机程序代码,上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++,还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中,远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机,或者,可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图,图示了按照本公开的各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上,流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分,该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意,在有些作为替换的实现中,方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如,两个接连地表示的方框实际上可以基本并行地执行,它们有时也可以按相反的顺序执行,这依所涉及的功能而定。也要注意的是, 框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合,可以用执行规定的功能或操作的专用的基于硬件的系统来实现,或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开的实施例中所涉及到的单元可以通过软件的方式实现,也可以通过硬件的方式来实现。所描述的单元也可以设置在处理器中,例如,可以描述为:一种处理器包括特征获取单元和缺陷信息获取单元。其中,这些单元的名称在某种情况下并不构成对该单元本身的限定,例如,特征获取单元还可以被描述为“从待检测图像中获取待检测物体的多个图像特征的单元”。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解,本公开中所涉及的发明范围,并不限于上述技术特征的特定组合而成的技术方案,同时也应涵盖在不脱离上述发明构思的情况下,由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。

Claims (14)

  1. 一种用于获取信息的方法,包括:
    从待检测图像中获取至少一条图像特征,其中,所述待检测图像包含待检测物体图像,所述图像特征用于表征所述待检测物体的表面特征信息;以及
    将所述待检测图像和所述至少一条图像特征导入预先训练的缺陷检测模型,得到对应所述待检测物体的缺陷信息,其中,所述缺陷检测模型通过样本图像、样本图像特征和样本缺陷信息训练得到,用于表征所述待检测图像和至少一条图像特征的对应关系。
  2. 根据权利要求1所述的方法,其中,所述从待检测图像中获取至少一条图像特征,包括:
    获取所述待检测物体的基准特征,所述基准特征包括以下至少一项:颜色特征、结构特征、平面特征;以及
    基于所述基准特征通过设定方式对所述待检测图像进行图像处理,得到对应的图像特征,其中,所述设定方式包括以下至少一项:色彩对比增强、滤波、纹理采集。
  3. 根据权利要求1所述的方法,其中,所述将所述待检测图像和所述至少一条图像特征导入预先训练的缺陷检测模型,得到对应所述待检测物体的缺陷信息,包括:
    将所述待检测图像和所述至少一条图像特征导入所述缺陷检测模型的对应输入通道,得到对应所述待检测物体的缺陷信息。
  4. 根据权利要求1所述的方法,其中,所述缺陷检测模型通过以下步骤训练得到:
    获取多个样本信息组和所述多个样本信息组中的每个样本信息组所对应的样本缺陷信息,其中,所述样本信息组包括样本图像和对应样本图像的至少一条样本图像特征;以及
    将所述多个样本信息组中的每个样本信息组作为输入,将所述多个样本信息组中的每个样本信息组所对应的所述样本缺陷信息作为输出,训练得到缺陷检测模型。
  5. 根据权利要求4所述的方法,其中,所述将所述多个样本信息组中的每个样本信息组作为输入,将所述多个样本信息组中的每个样本信息组所对应的所述样本缺陷信息作为输出,训练得到缺陷检测模型,包括:
    执行以下训练步骤:将所述多个样本信息组中的每个样本信息组依次输入至初始化缺陷检测模型,得到所述多个样本信息组中的每个样本信息组所对应的预测缺陷信息,将所述多个样本信息组中的每个样本信息组所对应的预测缺陷信息与该样本信息组所对应的所述样本缺陷信息进行比较,得到所述初始化缺陷检测模型的预测准确率,确定所述预测准确率是否大于预设准确率阈值,若大于所述预设准确率阈值,则将所述初始化缺陷检测模型作为训练完成的缺陷检测模型。
  6. 根据权利要求5所述的方法,其中,所述将所述多个样本信息组中的每个样本信息组作为输入,将所述多个样本信息组中的每个样本信息组所对应的所述样本缺陷信息作为输出,训练得到缺陷检测模型,包括:
    响应于不大于所述预设准确率阈值,调整所述初始化缺陷检测模型的参数,并继续执行所述训练步骤。
  7. 一种用于获取信息的装置,包括:
    特征获取单元,被配置成从待检测图像中获取至少一条图像特征,其中,所述待检测图像包含待检测物体图像,所述图像特征用于表征所述待检测物体的表面特征信息;以及
    缺陷信息获取单元,被配置成将所述待检测图像和所述至少一条图像特征导入预先训练的缺陷检测模型,得到对应所述待检测物体的缺陷信息,其中,所述缺陷检测模型通过样本图像、样本图像特征和 样本缺陷信息训练得到,用于表征所述待检测图像和至少一条图像特征的对应关系。
  8. 根据权利要求7所述的装置,其中,所述特征获取单元包括:
    基准特征获取子单元,被配置成获取所述待检测物体的基准特征,所述基准特征包括以下至少一项:颜色特征、结构特征、平面特征;以及
    图像特征获取子单元,被配置成基于所述基准特征通过设定方式对所述待检测图像进行图像处理,得到对应的图像特征,其中,所述设定方式包括以下至少一项:色彩对比增强、滤波、纹理采集。
  9. 根据权利要求7所述的装置,其中,所述缺陷信息获取单元包括:
    信息输入子单元,被配置成将所述待检测图像和所述至少一条图像特征导入所述缺陷检测模型的对应输入通道,得到对应所述待检测物体的缺陷信息。
  10. 根据权利要求7所述的装置,其中,所述装置包括缺陷检测模型训练单元,被配置成训练缺陷检测模型,所述缺陷检测模型训练单元包括:
    样本获取子单元,被配置成获取多个样本信息组和所述多个样本信息组中的每个样本信息组所对应的样本缺陷信息,其中,所述样本信息组包括样本图像和对应样本图像的至少一条样本图像特征;以及
    模型训练子单元,被配置成将所述多个样本信息组中的每个样本信息组作为输入,将所述多个样本信息组中的每个样本信息组所对应的所述样本缺陷信息作为输出,训练得到缺陷检测模型。
  11. 根据权利要求10所述的装置,其中,所述模型训练子单元包括:
    模型训练模块,被配置成将所述多个样本信息组中的每个样本信 息组依次输入至初始化缺陷检测模型,得到所述多个样本信息组中的每个样本信息组所对应的预测缺陷信息,将所述多个样本信息组中的每个样本信息组所对应的预测缺陷信息与该样本信息组所对应的所述样本缺陷信息进行比较,得到所述初始化缺陷检测模型的预测准确率,确定所述预测准确率是否大于预设准确率阈值,若大于所述预设准确率阈值,则将所述初始化缺陷检测模型作为训练完成的缺陷检测模型。
  12. 根据权利要求11所述的装置,其中,所述模型训练子单元包括:
    参数调整模块,响应于不大于所述预设准确率阈值,被配置成调整所述初始化缺陷检测模型的参数,返回所述模型训练模块。
  13. 一种电子设备,包括:
    一个或多个处理器;
    存储器,其上存储有一个或多个程序,
    当所述一个或多个程序被所述一个或多个处理器执行时,使得所述一个或多个处理器执行权利要求1至6中任一所述的方法。
  14. 一种计算机可读介质,其上存储有计算机程序,其特征在于,该程序被处理器执行时实现如权利要求1至6中任一所述的方法。
PCT/CN2020/116527 2020-03-11 2020-09-21 用于获取信息的方法及装置 WO2021179565A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
KR1020217024357A KR20210102458A (ko) 2020-03-11 2020-09-21 정보를 취득하는 방법 및 장치
JP2021541617A JP2022526473A (ja) 2020-03-11 2020-09-21 情報を取得するための方法および装置、電子機器、記憶媒体並びにコンピュータプログラム
EP20913051.7A EP3907697A4 (en) 2020-03-11 2020-09-21 INFORMATION ACQUISITION METHOD AND APPARATUS
US17/443,013 US20220270228A1 (en) 2020-03-11 2021-07-19 Method and apparatus for obtaining information

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010165825.4A CN111402220B (zh) 2020-03-11 2020-03-11 用于获取信息的方法及装置
CN202010165825.4 2020-03-11

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/443,013 Continuation US20220270228A1 (en) 2020-03-11 2021-07-19 Method and apparatus for obtaining information

Publications (1)

Publication Number Publication Date
WO2021179565A1 true WO2021179565A1 (zh) 2021-09-16

Family

ID=71428739

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/116527 WO2021179565A1 (zh) 2020-03-11 2020-09-21 用于获取信息的方法及装置

Country Status (3)

Country Link
EP (1) EP3907697A4 (zh)
CN (1) CN111402220B (zh)
WO (1) WO2021179565A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111402220B (zh) * 2020-03-11 2023-06-09 北京百度网讯科技有限公司 用于获取信息的方法及装置
CN113781460A (zh) * 2021-09-16 2021-12-10 北京远舢智能科技有限公司 烟包外观缺陷在线检测方法、装置、电子设备和存储介质
CN115131327B (zh) * 2022-07-14 2024-04-30 电子科技大学 一种颜色特征融合的显示屏彩色线缺陷检测方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018000731A1 (zh) * 2016-06-28 2018-01-04 华南理工大学 一种曲面表面缺陷自动检测方法及其装置
CN110009614A (zh) * 2019-03-29 2019-07-12 北京百度网讯科技有限公司 用于输出信息的方法和装置
CN110148130A (zh) * 2019-05-27 2019-08-20 北京百度网讯科技有限公司 用于检测零件缺陷的方法和装置
CN110517259A (zh) * 2019-08-30 2019-11-29 北京百度网讯科技有限公司 一种产品表面状态的检测方法、装置、设备及介质
CN111402220A (zh) * 2020-03-11 2020-07-10 北京百度网讯科技有限公司 用于获取信息的方法及装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018208791A1 (en) * 2017-05-08 2018-11-15 Aquifi, Inc. Systems and methods for inspection and defect detection using 3-d scanning
CN109829483B (zh) * 2019-01-07 2021-05-18 鲁班嫡系机器人(深圳)有限公司 缺陷识别模型训练方法、装置、计算机设备和存储介质
CN109949286A (zh) * 2019-03-12 2019-06-28 北京百度网讯科技有限公司 用于输出信息的方法和装置
CN109978870A (zh) * 2019-03-29 2019-07-05 北京百度网讯科技有限公司 用于输出信息的方法和装置
CN109961433A (zh) * 2019-03-29 2019-07-02 北京百度网讯科技有限公司 产品缺陷检测方法、装置及计算机设备
CN110276754B (zh) * 2019-06-21 2021-08-20 厦门大学 一种表面缺陷检测方法、终端设备及存储介质
CN110378900B (zh) * 2019-08-01 2020-08-07 北京迈格威科技有限公司 产品缺陷的检测方法、装置及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018000731A1 (zh) * 2016-06-28 2018-01-04 华南理工大学 一种曲面表面缺陷自动检测方法及其装置
CN110009614A (zh) * 2019-03-29 2019-07-12 北京百度网讯科技有限公司 用于输出信息的方法和装置
CN110148130A (zh) * 2019-05-27 2019-08-20 北京百度网讯科技有限公司 用于检测零件缺陷的方法和装置
CN110517259A (zh) * 2019-08-30 2019-11-29 北京百度网讯科技有限公司 一种产品表面状态的检测方法、装置、设备及介质
CN111402220A (zh) * 2020-03-11 2020-07-10 北京百度网讯科技有限公司 用于获取信息的方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3907697A4

Also Published As

Publication number Publication date
CN111402220B (zh) 2023-06-09
CN111402220A (zh) 2020-07-10
EP3907697A4 (en) 2022-11-30
EP3907697A1 (en) 2021-11-10

Similar Documents

Publication Publication Date Title
WO2021179565A1 (zh) 用于获取信息的方法及装置
WO2020155907A1 (zh) 用于生成漫画风格转换模型的方法和装置
US11276158B2 (en) Method and apparatus for inspecting corrosion defect of ladle
WO2020006963A1 (zh) 生成图像检测模型的方法和装置
WO2020000879A1 (zh) 图像识别方法和装置
WO2020233200A1 (zh) 用于训练模型的方法和装置、用于预测信息的方法和装置
WO2019237657A1 (zh) 用于生成模型的方法和装置
CN109447156B (zh) 用于生成模型的方法和装置
JP2020030811A (ja) 応答時間を確定するための方法及び装置
WO2020029608A1 (zh) 用于检测电极片毛刺的方法和装置
CN108830837A (zh) 一种用于检测钢包溶蚀缺陷的方法和装置
WO2022252881A1 (zh) 图像处理方法、装置、可读介质和电子设备
WO2020093724A1 (zh) 生成信息的方法和装置
US20220270228A1 (en) Method and apparatus for obtaining information
US20220309405A1 (en) Combined-learning-based internet of things data service method and apparatus, device and medium
CN109902446A (zh) 用于生成信息预测模型的方法和装置
CN107704388A (zh) 用于确定应用的启动时间的方法和装置
CN110059624A (zh) 用于检测活体的方法和装置
CN114419400B (zh) 图像识别模型的训练方法、识别方法、装置、介质和设备
CN111062914B (zh) 用于采集脸部图像的方法、装置、电子设备和计算机可读介质
CN111862081B (zh) 图像评分方法、分数预测网络的训练方法、装置
CN116662193A (zh) 页面测试方法和装置
CN112315463B (zh) 一种婴幼儿听力测试方法、装置及电子设备
CN110084298B (zh) 用于检测图像相似度的方法及装置
CN113255819A (zh) 用于识别信息的方法和装置

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2021541617

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20217024357

Country of ref document: KR

Kind code of ref document: A

Ref document number: 2020913051

Country of ref document: EP

Effective date: 20210719

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20913051

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE