WO2021079728A1 - 欠陥位置判定システム、外観検査方法およびプログラム - Google Patents

欠陥位置判定システム、外観検査方法およびプログラム Download PDF

Info

Publication number
WO2021079728A1
WO2021079728A1 PCT/JP2020/037747 JP2020037747W WO2021079728A1 WO 2021079728 A1 WO2021079728 A1 WO 2021079728A1 JP 2020037747 W JP2020037747 W JP 2020037747W WO 2021079728 A1 WO2021079728 A1 WO 2021079728A1
Authority
WO
WIPO (PCT)
Prior art keywords
inspected
defect
image
information
type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/037747
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
優香 大島
卓郎 鹿嶋
勇介 小板橋
松田　淳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to US17/770,109 priority Critical patent/US12175653B2/en
Priority to JP2021554232A priority patent/JP7248144B2/ja
Publication of WO2021079728A1 publication Critical patent/WO2021079728A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/0008—Industrial image inspection checking presence/absence
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G01N2021/8861—Determining coordinates of flaws
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G01N2021/888—Marking defects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G01N2021/889—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques providing a bare video image, i.e. without visual measurement aids
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker

Definitions

  • the present invention relates to a defect position determination system for determining the position of a defect occurring in a product, a terminal device, a defect position determination method, an appearance inspection method, and an appearance inspection program.
  • Painted products such as automobile bodies may have various defects in the manufacturing process and distribution process of the products.
  • defects include unevenness in painting due to dust biting on the painting line, color unevenness during painting, and scratches during painting work and transportation. Inspection of such defects is generally performed at the time of manufacture or prior to sale by the dealer.
  • Patent Document 1 describes a surface inspection device that inspects defects on the surface of an automobile.
  • the surface inspection apparatus described in Patent Document 1 irradiates the surface of the object to be inspected with light, forms a light receiving image based on the reflected light from the surface to be inspected, and is on the surface to be inspected based on the received image. Detects defects present in.
  • an image of the position where the defect has occurred is generally taken manually at the store, and the manufacturer (manufacturer) who acquired the image determines the type of defect from the image. And size.
  • the situation in which the defect is photographed is not determined, so that the image of the defect photographed differs depending on the position and angle at which the defect is photographed. Therefore, it is difficult to accurately determine the position of the defect from such an image.
  • a light receiving image is formed on the production line and defects are detected. That is, by arranging this surface inspection device at a specific position on the manufacturing line, it is possible to acquire the position of the defect, the content of the defect, the size of the defect, and the like. However, it is difficult to introduce a surface inspection device as described in Patent Document 1 into individual retailers.
  • a defect position determination system a terminal device, a defect position determination method, an appearance inspection method, and an appearance inspection program that can accurately collect information on the position of the defect while reducing the man-hours required for determining the position of the defect.
  • the purpose is to provide.
  • the defect position determination system uses a first imaging means for generating a first image of an image to be inspected and a model for determining the object to be inspected from the image to obtain an object to be inspected from the first image.
  • a guide display means for determining and displaying an illustration showing the object to be inspected as a guide and an object to be inspected having a marker attached in the vicinity of the defect which can be recognized regardless of the appearance color of the object to be inspected are superimposed on the guide.
  • a second imaging means for generating the second image captured by the second image, a defect position determining means for determining the position of a defect included in the object to be inspected from the positional relationship between the illustration and the marker included in the second image, and the defect position determining means. It is characterized by being provided with an information collecting means for collecting defect information in which the type of the object to be inspected and the position of the defect are associated with each other.
  • Another defect position determination system includes a terminal device and a server device, and the terminal device obtains a first imaging means for generating a first image of an image to be inspected and an object to be inspected from the image.
  • a guide display means that determines the object to be inspected from the first image using the determination model and displays an illustration showing the object to be inspected as a guide, and a marker that can be recognized regardless of the color of the appearance of the object to be inspected.
  • the terminal device determines an object to be inspected from the first image by using a first imaging means for generating a first image of the object to be inspected and a model for determining the object to be inspected from the image.
  • a guide display means for displaying an illustration showing the object to be inspected as a guide, and an object to be inspected having a marker attached near the defect, which can be recognized regardless of the appearance color of the object to be inspected, are superimposed on the guide and imaged.
  • the defect position determination method generates a first image of an image to be inspected, determines the object to be inspected from the first image using a model for determining the object to be inspected from the image, and determines the object to be inspected.
  • An illustration showing the inspection object is displayed as a guide, and a second image is generated by superimposing the inspection object with a marker attached near the defect, which can be recognized regardless of the appearance color of the inspection object, on the guide.
  • the position of the defect contained in the object to be inspected is determined from the positional relationship between the illustration included in the second image and the marker, and the defect information associated with the type of the object to be inspected and the position of the defect is collected. It is characterized by.
  • the visual inspection method generates a first image of an image to be inspected, uses a model for determining the object to be inspected from the image, determines the object to be inspected from the first image, and inspects the object.
  • An illustration showing the object is displayed as a guide, and a second image is generated by superimposing the object to be inspected with a marker attached near the defect, which can be recognized regardless of the appearance color of the object to be inspected, on the guide.
  • a server device that determines the position of a defect contained in an object to be inspected from the positional relationship between the illustration included in the second image and the marker, and collects defect information associated with the type of the object to be inspected and the position of the defect. It is characterized in that the position of the determined defect is transmitted to the object.
  • the visual inspection program uses a computer to generate a first image of an object to be inspected, a first imaging process, and a model for determining the object to be inspected from the image, and the object to be inspected from the first image.
  • Guide display processing that displays an illustration showing the object to be inspected as a guide, and superimposes the object to be inspected with a marker attached near the defect that can be recognized regardless of the color of the appearance of the object to be inspected.
  • the present invention it is possible to accurately collect information on the position of a defect while reducing the man-hours required for determining the position of the defect.
  • FIG. 1 is a block diagram showing a configuration example of the first embodiment of the defect position determination system according to the present invention.
  • the defect position determination system 10 of the present embodiment includes a terminal device 110 and a server device 120.
  • the terminal device 110 and the server device 120 are connected to each other via a communication network.
  • the terminal device 110 transmits the storage unit 111, the first imaging unit 112, the guide display unit 113, the display unit 114, the second imaging unit 115, the identification information input unit 116, the defect type acquisition unit 117, and the transmission unit 110. Includes part 118.
  • the storage unit 111 stores various information necessary for the terminal device 110 to perform processing. Specifically, the storage unit 111 stores a model in which the guide display unit 113, which will be described later, determines the object to be inspected from the target image. For example, when the object to be inspected is an automobile, the above model corresponds to a model for determining an automobile.
  • the storage unit 111 may store a model for determining the type and orientation of the object to be inspected from the target image. In this case, for example, if the object to be inspected is an automobile, the model corresponds to a model for determining the vehicle type and the orientation of the automobile.
  • the storage unit 111 may store the identification information that can uniquely identify the object to be inspected and the attribute of the object to be inspected in association with each other.
  • the identification information is a vehicle identification number (VIN: Vehicle Identification Number) or the like
  • the attribute information is a production date, a production factory, a country of production, or the like. This information may be stored in the storage unit 121 of the server device 120, which will be described later.
  • the first imaging unit 112 generates a first image of an image of the object to be inspected. Specifically, the first imaging unit 112 generates a first image in which the entire object to be inspected including a defect on the surface is imaged according to the operation of the user. For example, when the object to be inspected is an automobile, the first imaging unit 112 generates a first image of the automobile according to the positioning performed by the user so that the entire automobile is imaged.
  • the first imaging unit 112 may generate the first image in response to the user's explicit imaging instruction, or the timing at which the object to be inspected is detected by the sensor that automatically detects the object to be inspected.
  • the first image may be generated according to the above.
  • the guide display unit 113 determines the object to be inspected from the generated first image by using the model stored in the storage unit 111.
  • the model is a model for determining an automobile learned by using an external image of a vehicle body, and the learning method and the mode of the model are not particularly limited.
  • the guide display unit 113 uses this model to generate a first image. Determine the type and orientation of the object to be inspected.
  • this model is a model for determining the vehicle type and the orientation of the vehicle body learned by using the external image of the vehicle body, and the learning method and the mode of the model are not particularly limited.
  • the configuration may be such that the user directly selects the vehicle type before the first image is generated by the first imaging unit 112. In this case, the guide display unit 113 determines the orientation of the vehicle body using the model.
  • the guide display unit 113 generates an illustration showing the object to be inspected and displays it as a guide on the display unit 114 described later. Further, the guide display unit 113 may generate an illustration showing the type and orientation of the object to be inspected and display it as a guide on the display unit 114 described later.
  • the method by which the guide display unit 113 generates an illustration is arbitrary. For example, a three-dimensional skeleton image is stored in advance in the storage unit 111 according to the type of the object to be inspected, and the guide display unit 113 rotates the skeleton image according to the type and orientation of the determined object to be inspected. May be displayed on the display unit 114.
  • the display unit 114 is a device that displays the object to be inspected (imaging target) via a lens or the like and displays the guide generated by the guide display unit 113.
  • the display unit 114 is realized by, for example, a display device.
  • the second imaging unit 115 generates a second image captured by superimposing the object to be inspected on the guide of the display unit 114.
  • a marker is manually attached near the defect of the object to be inspected, and the second imaging unit 115 superimposes the object to be inspected to which the marker is attached on the guide of the display unit 114 by the user's operation.
  • a second image is generated by taking an image.
  • the marker attached to the object to be inspected in the present embodiment is a marker that can be recognized regardless of the color of the appearance of the object to be inspected.
  • the attached marker is a marker that can be recognized regardless of the body color (painted color). Since an image including the marker and the entire object to be inspected is imaged by the second imaging unit 115, the marker preferably has a size that can be recognized when the object to be inspected is imaged as a pulling image, for example, several centimeters in height and width. It is formed by a square or a circle with a diameter of several centimeters.
  • FIG. 2 is an explanatory diagram showing an example of a marker.
  • the marker shown in FIG. 2 is an example of a marker that can be recognized regardless of the color of the appearance of the object to be inspected.
  • the marker 41 is a circular marker using two colors, and each semicircle is represented by a different color.
  • the marker 42 is a circular marker using four colors, and the circle is divided into four equal parts, each of which is represented by a different color.
  • the marker 43 is a marker in which a square region inscribed in the circle and a region other than the square in the circle are represented by different colors.
  • a slightly smaller square is arranged inside the square, and the inner square area and the outer square area excluding the inner square area are represented by different colors. It is a marker that has been made.
  • the marker shown in FIG. 2 is an example, and the aspect of the marker is not limited to the content illustrated in FIG. Specifically, it is preferable that the marker is formed so that two or more kinds of colors are used and a part or all of a square or a circle can be specified in each of at least two colors.
  • the markers illustrated in FIG. 2 are displayed in white and black or white and black shaded areas, but the white or black and the shaded areas are white and white, respectively. It may be represented by a color other than black.
  • FIG. 3 is an explanatory diagram showing an example of processing until the second image is generated.
  • an automobile is assumed as the object to be inspected.
  • the marker 41 is attached by a user or the like to the vicinity of a defect of the automobile 31 which is an object to be inspected (step S101).
  • the automobile 31 is displayed on the display unit 114 by an operation of the user or the like, and the first imaging unit 112 of the terminal device 110 captures an image of the automobile 31 including the marker 41 to generate the first image 32 (. Step S102).
  • the guide display unit 113 determines the orientation of the automobile 31 from the first image 32 using the model, and displays the illustration 33 of the automobile 31 as a guide on the display unit 114 (step S103).
  • the second image pickup unit 115 superimposes the guide of the display unit 114 and the automobile 31 by the operation of the user or the like (step S104), and generates the second image 34 by taking an image (step S105).
  • the image of the automobile 31 in which the marker 41 is attached to the left front mirror is generated as the second image 34.
  • the identification information input unit 116 accepts the input of the identification information of the object to be inspected.
  • the identification information input unit 116 may directly accept the input of the identification information from the user or the like, or may extract the identification information from the first image or the second image captured.
  • the attribute of the object to be inspected associated with the identification information is stored in the storage unit 111, the attribute of the object to be inspected can also be replenished.
  • the defect type acquisition unit 117 acquires the type of defect included in the object to be inspected.
  • the defect type acquisition unit 117 can accurately collect information on the position of the defect by acquiring the defect type.
  • the defect type acquisition unit 117 may receive input of the defect type from the user or the like and acquire the input as the defect type. In addition, labels of different types may be prepared according to the type of defect. In this case, the defect type acquisition unit 117 may extract the label type from the captured second image and determine the defect type based on the extracted label type.
  • the transmission unit 118 transmits the generated second image to the server device 120. Further, when the identification information and the type of the defect can be acquired, the transmission unit 118 may transmit the acquired identification information, the attribute based on the identification information, and the type of the defect to the server device 120.
  • the server device 120 includes a storage unit 121, an input unit 122, a defect position determination unit 123, a defect site identification unit 124, an information collection unit 125, an analysis unit 126, and an output unit 127.
  • the storage unit 121 stores various information necessary for the server device 120 to perform processing and the collected information. Further, the storage unit 121 may store a part master in which information indicating a position in the object to be inspected is associated with a part or a part of the object to be inspected for each type of the object to be inspected.
  • the position in the object to be inspected is represented, for example, in the range of three-dimensional coordinates.
  • the part master holds the left front mirror of the automobile and the range of coordinates indicating the left front mirror in association with each other for each type of automobile.
  • the input unit 122 receives the input of the second image from the terminal device 110. Further, the input unit 122 may accept input of identification information, attributes based on the identification information, and types of defects.
  • the defect position determination unit 123 determines the position of the defect included in the object to be inspected from the positional relationship between the illustration included in the second image and the marker. Specifically, in the present embodiment, since the defect generated in the appearance of the object to be inspected is imaged, the defect position determination unit 123 is the surface of the object to be inspected on the illustration corresponding to the position where the marker is imaged. The position of is determined. Since the method of specifying the position in the three-dimensional image is widely known, detailed description thereof will be omitted here.
  • the defect site identification unit 124 identifies the site or part in which the defect has occurred from the position of the determined defect based on the site master. This makes it possible to collect information on the specific part or part where the defect is occurring.
  • the information collecting unit 125 collects defect information associated with the type of the object to be inspected and the position of the defect, and stores it in the storage unit 121. In addition, when a defective portion or part is specified, the information collecting unit 125 may collect defect information associated with the type of the object to be inspected and the defective portion. Further, when the type of the object to be inspected and the attributes of the object to be inspected are acquired from the terminal device 110, the information collecting unit 125 may collect defect information including the information further.
  • the storage unit 121 stores the identification information of the object to be inspected in association with the attribute of the object to be inspected, and when the input unit 122 accepts the input of the identification information of the object to be inspected, the information is collected.
  • the unit 125 may acquire the attribute corresponding to the identification information from the storage unit 121 and include it in the defect information.
  • the analysis unit 126 analyzes the tendency of defects based on the collected defect information.
  • the content of the tendency analysis performed by the analysis unit 126 is arbitrary.
  • the analysis unit 126 may total the number of occurrences (specifically, the number of applications) for each type of the object to be inspected in which the defect has occurred and the part of the defect.
  • the output unit 127 outputs the analysis result. Further, in the present embodiment, when the collected defect information satisfies a predetermined condition, the output unit 127 outputs an alert based on the defect information. For example, as a result of totaling for each defect of the same part, when the number of applications for a specific type of attribute is a certain number or more, the output unit 127 outputs information associating the defective part with the attribute as an alert. You may.
  • the terminal device 110 is realized by, for example, a tablet terminal or the like.
  • the first imaging unit 112, the guide display unit 113, the second imaging unit 115, the identification information input unit 116, the defect type acquisition unit 117, and the transmission unit 118 are programs (visual inspection program).
  • a computer processor for example, CPU (Central Processing Unit), GPU (Graphics Processing Unit)).
  • the program is stored in the storage unit 111, the processor reads the program, and according to the program, the first imaging unit 112, the guide display unit 113, the second imaging unit 115, the identification information input unit 116, and the defect type acquisition unit It may operate as 117 and a transmitter 118.
  • the function of the terminal device 110 may be provided in the SaaS (Software as a Service) format.
  • the first imaging unit 112, the guide display unit 113, the second imaging unit 115, the identification information input unit 116, the defect type acquisition unit 117, and the transmission unit 118 are each realized by dedicated hardware. It may have been done. Further, a part or all of each component of each device may be realized by a general-purpose or dedicated circuit (circuitry), a processor, or a combination thereof. These may be composed of a single chip or may be composed of a plurality of chips connected via a bus. A part or all of each component of each device may be realized by a combination of the above-mentioned circuit or the like and a program.
  • each component of the terminal device 110 when a part or all of each component of the terminal device 110 is realized by a plurality of information processing devices and circuits, the plurality of information processing devices and circuits may be centrally arranged or distributed. It may be arranged.
  • the information processing device, the circuit, and the like may be realized as a form in which each is connected via a communication network, such as a client-server system and a cloud computing system.
  • the server device 120 is realized by, for example, a server device connected to a storage.
  • the input unit 122, the defect position determination unit 123, the defect site identification unit 124, the information collection unit 125, the analysis unit 126, and the output unit 127 operate according to a program (defect information collection program). It is realized by the processor of the computer.
  • the storage unit 111 and the storage unit 121 are realized by, for example, a magnetic disk or the like.
  • FIG. 4 is a flowchart showing an operation example of the defect position determination system 10 of the present embodiment.
  • the first imaging unit 112 of the terminal device 110 generates a first image of the object to be inspected (step S11).
  • the guide display unit 113 determines an object to be inspected from the first image using a model (step S12), and displays an illustration showing the object to be inspected as a guide (step S13).
  • the second imaging unit 115 generates a second image captured by superimposing an object to be inspected with a marker attached in the vicinity of the defect on the guide (step S14). Then, the transmission unit 118 transmits the generated second image to the server device 120 (step S15).
  • the transmission unit 118 may transmit the identification information received by the identification information input unit 116 or the defect type acquired by the defect type acquisition unit 117 to the server device 120.
  • the input unit 122 of the server device 120 receives the input of the second image (step S16).
  • the defect position determination unit 123 determines the position of the defect included in the object to be inspected from the positional relationship between the illustration included in the second image and the marker (step S17).
  • the defect site identification unit 124 may identify the site or component in which the defect has occurred from the position of the determined defect based on the site master.
  • the information collecting unit 125 collects defect information in which the type of the object to be inspected and the position of the defect are associated with each other (step S18).
  • the analysis unit 126 performs a defect tendency analysis based on the collected defect information, and the output unit 127 outputs the analysis result.
  • the first imaging unit 112 of the terminal device 110 generates the first image of the object to be inspected.
  • the guide display unit 113 determines an object to be inspected from the first image using a model, and displays an illustration showing the object to be inspected as a guide.
  • the second imaging unit 115 generates a second image captured by superimposing an object to be inspected with a marker attached in the vicinity of the defect on the guide, and the defect position determination unit 123 of the server device 120 displays the second image.
  • the position of the defect contained in the object to be inspected is determined from the positional relationship between the included illustration and the marker.
  • the information collecting unit 125 collects defect information in which the type of the object to be inspected and the position of the defect are associated with each other. Therefore, it is possible to accurately collect the information on the position of the defect while reducing the man-hours required for determining the position of the defect.
  • Embodiment 2 Next, a second embodiment of the defect position determination system of the present invention will be described.
  • a method has been described in which the terminal device 110 generates a second image and transmits it to the server device 120, and the server device 120 determines the position of the defect.
  • the server device 120 determines the position of the defect.
  • a configuration for determining the position of a defect on the device side that generated the second image will be described.
  • FIG. 5 is a block diagram showing a configuration example of a second embodiment of the defect position determination system according to the present invention.
  • the defect position determination system 20 of the present embodiment includes a terminal device 210 and a server device 220.
  • the terminal device 210 and the server device 220 are connected to each other via a communication network.
  • the terminal device 210 transmits the storage unit 211, the first imaging unit 112, the guide display unit 113, the display unit 114, the second imaging unit 115, the identification information input unit 116, the defect type acquisition unit 117, and the transmission.
  • a unit 218, a defect position determination unit 223, and a defect site identification unit 224 are included. That is, the terminal device 210 of the present embodiment includes the storage unit 211 and the transmission unit 218 instead of the storage unit 111 and the transmission unit 118 as compared with the terminal device 110 of the first embodiment, and includes the defect position determination unit 223. It differs from the first embodiment in that it further includes a defect site identification portion 224.
  • the server device 220 includes a storage unit 221, an input unit 222, an information collection unit 125, an analysis unit 126, and an output unit 127. That is, the server device 220 of the present embodiment includes the storage unit 221 and the input unit 222 instead of the storage unit 121 and the input unit 122 as compared with the server device 120 of the first embodiment, and includes the defect position determination unit 123. It is different from the first embodiment in that it does not include the defect site identifying portion 124.
  • the contents of the defect position determination unit 223 and the defect site identification unit 224 are the same as the contents of the defect position determination unit 123 and the defect site identification unit 124 of the first embodiment. That is, it can be said that the defect position determination system 20 of the present embodiment has a configuration in which the functions of the defect position determination unit 123 and the defect site identification unit 124 of the defect position determination system 10 in the first embodiment are transferred to the terminal device 210. .. Therefore, the terminal device 210 of the present embodiment can be referred to as a visual inspection device.
  • the storage unit 211 may store the site master in addition to the information stored in the storage unit 111 of the first embodiment. That is, the storage unit 221 does not have to store the site master as compared with the storage unit 121 of the first embodiment. Further, the transmission unit 218 transmits the position of the defect included in the object to be inspected to the server device 220. The input unit 222 receives the input of the position of the defect contained in the object to be inspected.
  • the terminal device 210 is also realized by, for example, a tablet terminal or the like. Specifically, the first imaging unit 112, the guide display unit 113, the second imaging unit 115, the identification information input unit 116, the defect type acquisition unit 117, the transmission unit 218, and the defect position determination unit 223.
  • the defect site identification unit 224 is realized by a computer processor that operates according to a program (visual inspection program).
  • the server device 220 is also realized by, for example, a server device connected to a storage.
  • the input unit 222, the information collection unit 125, the analysis unit 126, and the output unit 127 are realized by a computer processor that operates according to a program (defect information collection program).
  • the storage unit 211 and the storage unit 221 are realized by, for example, a magnetic disk or the like.
  • FIG. 6 is a flowchart showing an operation example of the defect position determination system 20 of the present embodiment.
  • the process until the second image is generated is the same as the process from step S11 to step S14 illustrated in FIG.
  • the defect position determination unit 223 of the terminal device 210 determines the position of the defect included in the object to be inspected from the positional relationship between the illustration included in the second image and the marker (step S21). Then, the transmission unit 218 transmits information indicating the position of the defect to the server device 220 (step S21).
  • the input unit 222 of the server device 220 accepts the input of information indicating the position of the defect (step S23). After that, the process of collecting defect information is the same as the process of step S18 illustrated in FIG.
  • the defect position determination system 20 of the present embodiment can also accurately collect the information on the position of the defect while reducing the man-hours required for determining the position of the defect.
  • the first embodiment and the second embodiment exemplify the case where the defect position determination unit and the defect site identification unit are included in the same device.
  • the defect position determination system may have a configuration in which the defect position determination unit is included in the terminal device and the defect site identification unit is included in the server device.
  • FIG. 7 is a block diagram showing an outline of the defect position determination system according to the present invention.
  • the defect position determination system 70 (for example, defect position determination systems 10 and 20) of the present invention generates a first image of an object to be inspected (for example, an automobile). Using the unit 112) and a model for determining the object to be inspected from the image, the object to be inspected is determined from the first image, and an illustration showing the object to be inspected is displayed as a guide (for example, on the display unit 114).
  • a guide display unit 72 for example, a guide display unit 113 and an object to be inspected to which a marker (for example, a marker illustrated in FIG.
  • a defect position determination unit 74 (for example, defect position determination units 123, 223) that determines the position of a contained defect and an information collection unit 75 (for example, an information collection unit 75) that collects defect information in which the type of the object to be inspected and the position of the defect are associated with each other. , Information gathering unit 125).
  • the defect position determination system 70 determines the position of the defect based on the part master in which the information indicating the position in the object to be inspected is associated with the part of the object to be inspected for each type of the object to be inspected.
  • a defect site identification unit (for example, a defect site identification unit 124, 224) may be provided to identify a site where a defect has occurred. Then, the information collecting unit 75 may collect defect information in which the type of the object to be inspected and the portion where the defect has occurred are associated with each other.
  • the guide display unit 72 determines the type and orientation of the object to be inspected from the first image by using a model for determining the type and orientation of the object to be inspected from the image, and the type of the object to be inspected. And an illustration showing the orientation may be displayed as a guide.
  • the defect position determination system 70 inputs an attribute storage unit (for example, storage unit 111) that stores the attributes of the object to be inspected in association with the identification information of the object to be inspected, and the identification information that identifies the object to be inspected. It may be provided with an identification information input unit (for example, an identification information input unit 116) that receives the above. Then, the information collecting unit 75 may collect defect information associated with the type of the object to be inspected, the position of the defect, and the attributes of the object to be inspected specified by the identification information.
  • an attribute storage unit for example, storage unit 111
  • an identification information input unit for example, an identification information input unit 116 that receives the above.
  • the information collecting unit 75 may collect defect information associated with the type of the object to be inspected, the position of the defect, and the attributes of the object to be inspected specified by the identification information.
  • the defect position determination system 70 includes an analysis unit (for example, analysis unit 126) that analyzes the tendency of defects based on the collected defect information, and an output unit (for example, output unit 127) that outputs the analysis result. May be provided. Then, if the collected defect information satisfies a predetermined condition, the output unit may output an alert based on the defect information.
  • analysis unit for example, analysis unit 126
  • output unit for example, output unit 1207
  • FIG. 8 is a block diagram showing another outline of the defect position determination system according to the present invention.
  • Another defect position determination system according to the present invention includes a terminal device 80 (for example, a terminal device 110) and a server device 90 (for example, a server device 120).
  • the terminal device 80 uses a first imaging unit 81 (for example, the first imaging unit 112) that generates a first image of an object to be inspected (for example, an automobile) and a model for determining the object to be inspected from the image.
  • the guide display unit 82 (for example, the guide display unit 113) that determines the object to be inspected from the first image and displays an illustration showing the object to be inspected as a guide (for example, on the display unit 114) and the inspected object.
  • a second image is generated by superimposing an object to be inspected on which a marker (for example, a marker illustrated in FIG. 2) that can be recognized regardless of the appearance color of the object is attached in the vicinity of a defect on a guide.
  • the server device 90 determines the position of the defect included in the object to be inspected based on the positional relationship between the input unit 91 (for example, the input unit 122) that receives the input of the second image and the illustration and the marker included in the second image.
  • a defect position determination unit 92 for example, a defect position determination unit 123) and an information collection unit 93 (for example, an information collection unit 125) that collects defect information in which the type of the object to be inspected and the position of the defect are associated with each other. including.
  • FIG. 9 is a block diagram showing an outline of the terminal device according to the present invention.
  • the terminal device 60 (for example, the terminal device 210) according to the present invention includes a first imaging unit 61 (for example, a first imaging unit 112) that generates a first image of an object to be inspected (for example, an automobile) and an image.
  • a guide display unit 62 (for example, on the display unit 114) that determines the object to be inspected from the first image using a model for determining the object to be inspected and displays an illustration showing the object to be inspected as a guide (for example, on the display unit 114).
  • the guide display unit 113) and the object to be inspected to which a marker for example, the marker illustrated in FIG.
  • the defect position determination unit 64 for example, defect position determination unit 223 for determination and the server device (for example, server device 220) that collects defect information associated with the type of the object to be inspected and the position of the defect. It is provided with a transmission unit 65 (for example, transmission unit 218) that transmits the position of the defect.
  • the object to be inspected is determined from the first image by using the first imaging unit that generates the first image of the object to be inspected and the model for determining the object to be inspected from the image.
  • a guide display unit that displays an illustration showing an object to be inspected as a guide and an object to be inspected having a marker attached in the vicinity of a defect that can be recognized regardless of the appearance color of the object to be inspected are superimposed on the guide and imaged.
  • a second imaging unit that generates a second image, a defect position determination unit that determines the position of a defect included in the object to be inspected from the positional relationship between the illustration and the marker included in the second image, and a defect position determination unit.
  • a defect position determination system including an information collecting unit that collects defect information in which the type of the object to be inspected and the position of the defect are associated with each other.
  • Appendix 2 A defect has occurred from the position of the determined defect based on the part master in which the information indicating the position in the object to be inspected and the part of the object to be inspected are associated with each type of the object to be inspected.
  • the defect position determination system according to Appendix 1 which includes a defect site identification unit for specifying a site, and the information collection unit collects defect information associated with the type of the object to be inspected and the site where the defect has occurred.
  • the guide display unit determines the type and orientation of the object to be inspected from the first image using a model for determining the type and orientation of the object to be inspected from the image, and the type and orientation of the object to be inspected.
  • the defect position determination system according to Appendix 1 or Appendix 2, which displays an illustration representing the above as a guide.
  • Appendix 4 Information is collected by providing an attribute storage unit that stores the attributes of the object to be inspected in association with the identification information of the object to be inspected and an identification information input unit that accepts input of identification information that identifies the object to be inspected.
  • the unit is described in any one of Appendix 1 to Appendix 3 for collecting defect information associated with the type of the object to be inspected, the position of the defect, and the attributes of the object to be inspected specified by the identification information. Defect location determination system.
  • Appendix 5 An analysis unit that analyzes the tendency of defects based on the collected defect information and an output unit that outputs the analysis result are provided, and the output unit satisfies the conditions predetermined by the collected defect information.
  • the defect position determination system according to any one of Appendix 1 to Appendix 4, which outputs an alert based on the defect information when satisfied.
  • a defect type acquisition unit for acquiring the type of defect contained in the object to be inspected is provided, and the information collection unit collects defect information associated with the type of the object to be inspected, the position of the defect, and the type of the defect.
  • the defect position determination system according to any one of Supplementary note 1 to Supplementary note 5.
  • a terminal device and a server device are provided, and the terminal device uses a first imaging unit that generates a first image of an image to be inspected and a model for determining an object to be inspected from the image.
  • a guide display unit that determines the object to be inspected from the first image and displays an illustration showing the object to be inspected as a guide, and a marker that can be recognized regardless of the color of the appearance of the object to be inspected are attached near the defect.
  • the server includes a second imaging unit that generates a second image captured by superimposing the object to be inspected on the guide, and a transmitting unit that transmits the generated second image to the server device.
  • the device determines the position of the defect included in the object to be inspected from the positional relationship between the input unit that receives the input of the second image and the illustration and the marker included in the second image.
  • a defect position determination system including a determination unit and an information collection unit that collects defect information in which the type of the object to be inspected and the position of the defect are associated with each other.
  • a terminal device and a server device are provided, and the terminal device uses a first imaging unit that generates a first image of an image to be inspected and a model for determining an object to be inspected from the image.
  • a guide display unit that determines the object to be inspected from the first image and displays an illustration showing the object to be inspected as a guide, and a marker that can be recognized regardless of the color of the appearance of the object to be inspected are attached near the defect.
  • the object to be inspected is based on the positional relationship between the second imaging unit that generates a second image obtained by superimposing the object to be inspected on the guide and the illustration and the marker included in the second image.
  • the server device includes a defect position determination unit for determining the position of a defect included in the above and a transmission unit for transmitting the determined defect position to the server device, and the server device includes the type of the object to be inspected and the position of the defect.
  • a defect position determination system including an information collecting unit that collects defect information associated with.
  • the object to be inspected is determined from the first image by using the first imaging unit that generates the first image of the object to be inspected and the model for determining the object to be inspected from the image.
  • a guide display unit that displays an illustration showing an object to be inspected as a guide and an object to be inspected having a marker attached in the vicinity of a defect that can be recognized regardless of the appearance color of the object to be inspected are superimposed on the guide and imaged.
  • a second imaging unit that generates a second image, a defect position determination unit that determines the position of a defect included in the object to be inspected from the positional relationship between the illustration and the marker included in the second image, and a defect position determination unit.
  • a terminal device including a transmission unit that transmits the determined defect position to a server device that collects defect information associated with the type of the object to be inspected and the defect position.
  • a defect has occurred from the position of the determined defect based on the part master in which the information indicating the position in the object to be inspected and the part of the object to be inspected are associated with each type of the object to be inspected.
  • the defect position determination method according to Appendix 10 which identifies a portion and collects defect information associated with the type of the object to be inspected and the portion where the defect has occurred.
  • Appendix 12 Using a model that generates a first image of an image to be inspected and determines the object to be inspected from the image, the object to be inspected is determined from the first image, and the object to be inspected is represented.
  • the illustration is displayed as a guide, and a second image is generated by superimposing the object to be inspected with a marker attached in the vicinity of the defect, which can be recognized regardless of the appearance color of the object to be inspected, on the guide. From the positional relationship between the illustration and the marker included in the second image, the position of the defect contained in the object to be inspected is determined.
  • a visual inspection method comprising transmitting the determined defect position to a server device that collects defect information associated with the type of the object to be inspected and the defect position.
  • a defect has occurred from the position of the determined defect based on the part master in which the information indicating the position in the object to be inspected and the part of the object to be inspected are associated with each type of the object to be inspected.
  • a computer is used for a first imaging process for generating a first image of an image to be inspected, and a model for determining the object to be inspected from the image is used to determine the object to be inspected from the first image.
  • Guide display processing that displays an illustration showing the object to be inspected as a guide, and an image of the object to be inspected with a marker attached near the defect that can be recognized regardless of the appearance color of the object to be inspected, superimposed on the guide.
  • a visual inspection program for causing a server device that collects defect information associated with the type of the object to be inspected and the position of the defect to execute a transmission process for transmitting the position of the determined defect.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Signal Processing (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
PCT/JP2020/037747 2019-10-25 2020-10-05 欠陥位置判定システム、外観検査方法およびプログラム Ceased WO2021079728A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/770,109 US12175653B2 (en) 2019-10-25 2020-10-05 Defect position determination system, appearance inspection method and program
JP2021554232A JP7248144B2 (ja) 2019-10-25 2020-10-05 欠陥位置判定システム、外観検査方法およびプログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-194144 2019-10-25
JP2019194144 2019-10-25

Publications (1)

Publication Number Publication Date
WO2021079728A1 true WO2021079728A1 (ja) 2021-04-29

Family

ID=75619958

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/037747 Ceased WO2021079728A1 (ja) 2019-10-25 2020-10-05 欠陥位置判定システム、外観検査方法およびプログラム

Country Status (3)

Country Link
US (1) US12175653B2 (https=)
JP (1) JP7248144B2 (https=)
WO (1) WO2021079728A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021079728A1 (ja) * 2019-10-25 2021-04-29 日本電気株式会社 欠陥位置判定システム、外観検査方法およびプログラム

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04308979A (ja) * 1991-04-06 1992-10-30 Mazda Motor Corp カラー色の識別方法
JPH1178807A (ja) * 1997-08-29 1999-03-23 Koto Sangyo Kk 車体損傷部の測定用シート
JP2011035509A (ja) * 2009-07-30 2011-02-17 Fujifilm Corp 複眼撮影装置及びその制御方法
JP2017009528A (ja) * 2015-06-25 2017-01-12 ダイハツ工業株式会社 不具合の許否判定方法
JP2017055182A (ja) * 2015-09-07 2017-03-16 オムロン株式会社 撮影装置、撮影方法及び撮影プログラム
US20180165541A1 (en) * 2016-12-12 2018-06-14 Jack Cooper Logistics, LLC System, method, and apparatus for detection of damages on surfaces

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3503130B2 (ja) 1997-08-25 2004-03-02 日産自動車株式会社 表面検査装置
JP2003075361A (ja) 2001-09-06 2003-03-12 Hitachi Ltd 非破壊検査方法
US20110218825A1 (en) * 2010-03-03 2011-09-08 International Business Machines Corporation Three-dimensional interactive vehicle damage claim interface
JP6326996B2 (ja) * 2014-06-13 2018-05-23 富士通株式会社 端末装置、情報処理システム、及び表示制御プログラム
US10755222B2 (en) * 2014-06-13 2020-08-25 Hitachi Systems, Ltd. Work management apparatus, work defect prevention program, and work defect prevention method
JP6924413B2 (ja) * 2017-12-25 2021-08-25 オムロン株式会社 データ生成装置、データ生成方法及びデータ生成プログラム
US20190278994A1 (en) * 2018-03-08 2019-09-12 Capital One Services, Llc Photograph driven vehicle identification engine
TWI805725B (zh) * 2018-06-07 2023-06-21 日商索尼半導體解決方案公司 資訊處理裝置、資訊處理方法及資訊處理系統
US10452915B1 (en) * 2018-08-08 2019-10-22 Capital One Services, Llc Systems and methods for depicting vehicle information in augmented reality
US11010888B2 (en) * 2018-10-29 2021-05-18 International Business Machines Corporation Precision defect detection based on image difference with respect to templates
US10410182B1 (en) * 2019-04-17 2019-09-10 Capital One Services, Llc Visualizing vehicle condition using extended reality
WO2021079728A1 (ja) * 2019-10-25 2021-04-29 日本電気株式会社 欠陥位置判定システム、外観検査方法およびプログラム
US10818042B1 (en) * 2020-01-14 2020-10-27 Capital One Services, Llc Vehicle information photo overlay
JPWO2022059710A1 (https=) * 2020-09-18 2022-03-24
KR20220130398A (ko) * 2021-03-18 2022-09-27 현대자동차주식회사 차량 검사 장치 및 차량의 검사 방법

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04308979A (ja) * 1991-04-06 1992-10-30 Mazda Motor Corp カラー色の識別方法
JPH1178807A (ja) * 1997-08-29 1999-03-23 Koto Sangyo Kk 車体損傷部の測定用シート
JP2011035509A (ja) * 2009-07-30 2011-02-17 Fujifilm Corp 複眼撮影装置及びその制御方法
JP2017009528A (ja) * 2015-06-25 2017-01-12 ダイハツ工業株式会社 不具合の許否判定方法
JP2017055182A (ja) * 2015-09-07 2017-03-16 オムロン株式会社 撮影装置、撮影方法及び撮影プログラム
US20180165541A1 (en) * 2016-12-12 2018-06-14 Jack Cooper Logistics, LLC System, method, and apparatus for detection of damages on surfaces

Also Published As

Publication number Publication date
JPWO2021079728A1 (https=) 2021-04-29
JP7248144B2 (ja) 2023-03-29
US20220392057A1 (en) 2022-12-08
US12175653B2 (en) 2024-12-24

Similar Documents

Publication Publication Date Title
US20180350056A1 (en) Augmented reality application for manufacturing
JP7621586B2 (ja) 生産ラインスマート監視システム及び監視方法
TW201840991A (zh) 電路板點膠檢測裝置及檢測方法
CN114419038A (zh) 轮毂表面缺陷的识别方法及装置、存储介质、电子设备
US20220335585A1 (en) Set up of a visual inspection process
US12524866B2 (en) Optimizing a set-up stage in an automatic visual inspection process
WO2019131742A1 (ja) 検査処理装置、検査処理方法、およびプログラム
US12141959B2 (en) Streamlining an automatic visual inspection process
WO2020079694A1 (en) Optimizing defect detection in an automatic visual inspection process
US20080205754A1 (en) Color identifying apparatus and color identifying method
WO2021079727A1 (ja) 外観検査装置、外観検査方法および外観検査プログラム
CN118941499A (zh) 一种线路板缺陷的视觉检测方法、电子设备及存储介质
CN114219758A (zh) 缺陷检测方法、系统、电子设备和计算机可读存储介质
JP7248144B2 (ja) 欠陥位置判定システム、外観検査方法およびプログラム
US20230096532A1 (en) Machine learning system, learning data collection method and storage medium
CN117173125A (zh) 一种基于全景图的缺陷点位显示方法、装置及存储介质
CN108957384A (zh) 一种周转箱及电能计量设备方向判别方法与装置
JP2011191170A (ja) 画像処理装置
WO2025100154A1 (ja) 歯車の歯当たり検査装置及び方法
CN113591594B (zh) 称重处理方法、装置及电子设备
JP2009115526A (ja) 被検査物の真円度測定方法
JP6015832B2 (ja) ラベル台帳検査装置及びラベル台帳検査方法
JPWO2021079728A5 (https=)
CN115797672A (zh) 巡检图像对齐方法、装置、计算机可读存储介质及机器人
CN114066799A (zh) 检测系统和检测方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20879862

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021554232

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20879862

Country of ref document: EP

Kind code of ref document: A1