WO2023055375A1

WO2023055375A1 - Image comparison to determine device abnormalities

Info

Publication number: WO2023055375A1
Application number: PCT/US2021/052862
Authority: WO
Inventors: Qian Lin; Deangeli NEVES; Tharsis VIANA; Hugo ALVES; Augusto VALENTE; Danny Dagan; Alvaro MACEDO
Original assignee: Hewlett-Packard Development Company, L.P.
Priority date: 2021-09-30
Filing date: 2021-09-30
Publication date: 2023-04-06

Abstract

In some examples, the disclosure describes a computing device that includes an imaging device to capture an image of a device comprising a plurality of components, and a processor to: access a reference image associated with the device, mask a boundary of the captured image of the device, align the captured image with the reference image based on a location of the plurality of components of the device, and determine an abnormality of the device.

Description

IMAGE COMPARISON TO DETERMINE DEVICE ABNORMALITIES Background

[0001] Computing devices are utilized to perform particular functions. Computing devices utilize a plurality of components to perform the particular functions. In some examples, the plurality of components are coupled to a printed circuit board (PCB) and/or coupled to other components connected to the PCB. The connections and interconnections of the plurality of components allows the computing devices to perform functions.

Brief Description of the Drawings

[0002] Figure 1 illustrates an example of a computing device for determining device abnormalities.

[0003] Figure 2 illustrates an example of a memory resource for determining device abnormalities.

[0004] Figure 3 illustrates an example of a system for determining device abnormalities.

[0005] Figure 4 illustrates an example of a method for determining device abnormalities.

[0006] Figure 5 illustrates an example of a method for determining device abnormalities.

[0007] Figure 6 illustrates an example of an image for determining device abnormalities.

[0008] Figure 7 illustrates an example of an image for determining device abnormalities.

Detailed Description

[0009] A user may utilize a computing device for various purposes, such as for business and/or recreational use. As used herein, the term computing device refers to an electronic device having a processor and a memory resource. Examples of computing devices can include, for instance, a laptop computer, a notebook computer, a desktop computer, and/or a mobile device (e.g., a smart phone, tablet, personal digital assistant, smart glasses, a wrist-worn device, etc.), among other types of computing devices. Computing devices can be utilized to perform a plurality of computing functions. Computing devices utilize a plurality of components (e.g., hardware computing components, etc.) to perform the functions. In some examples, the plurality of components are arranged and/or connected in a way to allow the computing device to function.

[0010] In some examples, a device (e.g., computing device, etc.) includes a plurality of components coupled to a substrate (e.g., printed circuit board (PCB), printed circuit assembly (PCA), etc.) to operate together and/or perform particular functions. In some examples, the device may not function according to specifications if the plurality of components are not connected to a particular specification or orientation. For example, a particular specification of a device includes connecting a first component to a second component utilizing a cable. In this example, the device may not function properly if the first component is not connected to the second component utilizing the cable. In this way, determining when the device is manufactured to the particular specification ensures that the device will work properly or to a specification of the device.

[0011] In some examples, a device can be assembled by connecting the plurality of components to the substrate. In these examples, the assembled device is inspected to ensure that the plurality of components are connected to the substrate and/or to other components as defined by the particular specification of the device. In some examples, human error may occur when a human user is inspecting the device to ensure that the device was assembled correctly. This can lead to devices being incorrectly assembled and/or devices being provided to an end user that does not operate to the particular specifications of the device.

[0012] The present disclosure relates to image inspection of physical devices that include a plurality of components. The image inspection ensures that the plurality of components are present on the physical device and that the plurality of components are installed or assembled in a way that provides particular functions of the physical device. In some examples, an image is captured of a physical device (e.g., computing device, PCB, PCA, etc.) that includes a plurality of components (e.g., electrical components, computing components, devices, etc.) installed on the physical device. The image is aligned with a reference image of the particular physical device. The image is then utilized to identify the plurality of components associated with the physical device and a plurality of tests are performed to determine whether the plurality of components are in an accepted state (e.g., “normar’, installed to specification, etc.) or are in a failed state (e.g., “abnormal”, installed incorrectly, installed inconsistent with a specification, etc.). In this way, the physical devices that are assembled are more accurately inspected compared to human inspection of the physical devices.

[0013] Figure 1 illustrates an example of a computing device 102 for determining device abnormalities. In some examples, the computing device 102 includes a processor 104 and a memory resource 106 to store instructions that are executed by the processor 104. In some examples, the computing device 102 includes a processor 104 and a memory resource 106 storing instructions 108, 110, 112, 114, that can be executed by the processor 104 to perform particular functions. In some examples, the computing device 102 is communicatively coupled to an imaging device 116 through a communication path 120. In some examples, the communication path 120 allows the computing device 102 to send and receive signals (e.g., communication signals, electrical signals, etc.) with the imaging device 116.

[0014] In some examples, the imaging device 116 is capable of capturing an image of a device 118. In some examples, the device 118 includes a plurality of components that are captured by the imaging device 116. In these examples, the imaging device 116 captures an image of the device 118 and sends the image to the device 102 through the communication path 120. In some examples, the imaging device 116 is a camera that captures still images (e.g., digital images, etc.) of the device 118.

[0015] The computing device 102 includes instructions 108 stored by the memory resource 106 that is executed by the processor 104 to access a reference image associated with the device 118. In some examples, the reference image of the device 118 is an image of a similar device or an image of the same type of device with the same components positioned at a same location on the device. For example, the device 118 can be a physical computing device or part of a physical computing device. In this example, the device 118 includes a plurality of computing components (e.g., processor 105, memory resource 106, wires, cables, capacitors, resistors, etc.). The plurality of components are aligned or coupled to the device 118 at particular locations to allow the device 118 to function to a manufacturer specification. For example, the reference image is an image of the device 118 without abnormalities.

[0016] The image captured by the imaging device 116 includes the plurality of components that are compared to the reference image that has previously confirmed to have a corresponding plurality of components that are correctly aligned or coupled to the computing device. For example, the computing device 102 compares the plurality of components within the captured image (e.g. , input image, etc.) to a corresponding plurality of components within the reference image.

[0017] In some examples, the reference image is stored in a database or other type of memory resource that is accessible by the computing device 102. In some examples, the database includes a plurality of reference images for a plurality of different types of devices. For example, the reference image for the device 118 may be stored in a database with a plurality of additional reference images for other devices. In some examples, the reference image for the device 118 is accessible to the computing device 102 utilizing a reference number associated with the device 118. For example, the device 118 may include a model number that identifies the type and/or plurality of components associated with the device 118. In this way, the computing device 102 accesses a reference image for the device 118 based on the identification number associated with the device 118.

[0018] As described herein, the imaging device 116 captures an image of the device 118 and provides the captured image (e.g., input image, etc.) to the computing device 120. In some examples, the computing device 102 includes instructions to instruct the imaging device 116 to capture an image of the device 118. In some examples, the captured image may include a background or portion of the captured image that is not associated with the device 118. In some examples, the imaging device 116 is not able to capture an image of the device 118 at the same angle or orientation as the reference image. For example, the imaging device 116 may be positioned at an inspection area where the device 118 is provided upon assembly. In some examples, the device 118 is provided to the imaging device 116 by a mechanized device such as a conveyer belt. In these examples, the imaging device 116 and/or the device 118 may not be aligned to capture an image with the precise orientation of the reference image. [0019] The computing device 102 includes instructions 110 stored by the memory resource 106 that is executed by the processor 104 to mask a boundary of the captured image of the device 118. In some examples, the boundaries of the device 118 within the image are identified by the computing device 102 such that portion of the captured image that are not part of the device 118 are masked or removed from the captured image of the device 118. In some examples, a mask includes a filter to prevent a particular portion of an image from being viewed or utilized by the computing device 102 for further actions. In this way, the boundaries of the captured image can be removed to prevent the boundaries from being utilized to determine abnormalities with the device 118. In some examples, the computing device 102 includes instructions to mask portions of the captured image that surround the device 118 comprising the plurality of components. In this way, only the device 118 is presented or utilized for determining abnormalities or for alignment with the reference image.

[0020] The computing device 102 includes instructions 112 stored by the memory resource 106 that is executed by the processor 104 to align the captured image with the reference image based on a location of the plurality of components of the device. In some examples, the computing device 102 identifies the location of the plurality components of the device 118 within the captured image. In some examples, a database that includes the reference image includes component data that describes a location of the plurality of components on the device 118. In other examples, the plurality of components can be identified utilizing the reference image to create a list of components to identify within the captured image of the device 118. [0021] In some examples, the plurality of components associated with the device 118 are identified in the captured image. In some examples, a portion of the plurality of components are identified by the computing device 102 to align the captured image with the reference image. For example, the plurality of components include screws or coupling devices that connect components to a substrate of the device 118. In this example, the screws of the captured image and the corresponding screws of the reference image are utilized to align the captured image with the reference image, in some examples, a first boundary box is generated to surround a screw within the captured image and a second boundary box is generated to surround a corresponding screw within the reference image. In these examples, the first boundary box and the second boundary box are aligned to align components of the captured image with the components of the reference image. [0022] The computing device 102 includes instructions 114 stored by the memory resource 106 that is executed by the processor 104 to determine an abnormality of the device 118 based on a comparison between the reference image and the captured image and a test model applied to the captured image. In some examples, the comparison between the captured image and the reference image can be utilized to generate a test model to perform a plurality of tests that is utilized to determine if an anomaly exists between the captured image and the reference image. In some examples, the plurality of tests are each directed to a particular component or set of components from the plurality of components. For example, a first test is utilized to count the quantity of a particular component of the plurality of components. In some examples, the counting test is to count the plurality of components installed on a device 118 (e.g., PCB, PCA, etc.) to a pre-identified quantity of components installed on a corresponding device. In another example, a second test is utilized to determine if a cable is interacting with a cable hook in a particular way. In these examples, the plurality of tests includes an interaction test to determine when a first component of the plurality of components is interacting with a second component of the plurality of components. For example, the interaction test can include a test of whether a hook on the device 118 is interacting in a particular way with a cable.

[0023] A plurality of additional tests can be added and/or removed from the plurality of tests. In this way, a first test may not depend on a second test. For example, the first test is independent from the second test. In this example, the plurality of tests are independent tests where a first test of the plurality of tests is unaffected by a second test of the plurality of tests. In a specific example, the first test is a counting test to determine the quantity of components on the device 118 and a second test is a component test. In this example, the first test can pass or fail independently from the second test. That is, the counting of all of the components can fail while the component test for a specific component can pass. In this way, the plurality of tests provide greater details of the inspection since an indication of an abnormality is specific to a particular test without affecting an outcome of a different test. [0024] In some examples, the computing device 102 includes instructions to determine heuristic information associated with the plurality of components. As used herein, the heuristic information includes a set of rules for identifying a particular component and/or identifying the predicted location for a particular component. For example, the computing device 102 determines heuristic information for a cable of the device 118. In some examples, the heuristic information for the cable of the device 118 includes identifying instances of the cable within the captured image (e.g., input image, etc.) and the reference image. In some examples, the heuristic information for the cable includes finding the correspondences among the points in segmentation masks for the input and reference image for the cables. In some examples, a distance-based (e.g., pixel distance based, etc.) method to find the correspondences is utilized to find the correspondences among the points of the segmentation masks applied to the cables.

[0025] In some examples, the heuristic information includes calculating the orientation for each point pair from the captured image and the reference image. In some examples, a Sobel gradient is employed to estimate the orientation of each point. As used herein a Sobel gradient includes a model for generating an image that focuses on the edges or boundaries. If the distance between two points, given a correspondence, is greater than Tdistance and the difference between the orientations, (angledift), is in the range of angleiower bound < anglediff < angleupper bound, the pair is considered defective. That is, the computing device 102 determines when the angle of the cable is between an upper bound and a lower bound. If the angle difference is not within the lower bound angle and upper bound angle, the device 118 is determined to fail. If the angle difference is between the lower bound angle and upper bound angle, the device 118 is determined to pass. In some examples, the test parameters, Tdistance, angleiower bound, and angleupper bound, can be set to define how big should be a deviation between a pair of points to be considered a defect.

[0026] As described herein, the computing device 102 can include a processor 104 communicatively coupled to a memory resource 106 through a communication path. As used herein, the processor 104 can include, but is not limited to: a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a metal-programmable cell array (MPCA), a semiconductor-based microprocessor, or other combination of circuitry and/or logic to orchestrate execution of instructions 108, 110, 112, 114. In other examples, the computing device can include instructions 108, 110, 112, 114, stored on a machine-readable medium (e.g., memory resource 106, non-transitory computer-readable medium, etc,) and executable by a processor 104, In a specific example, the computing device utilizes a non-transitory computer-readable medium storing instructions 108, 110, 112, 114, that, when executed, cause the processor 104 to perform corresponding functions.

[0027] Figure 2 illustrates an example of a memory resource 206 for determining device abnormalities. In some examples, the memory resource 206 can be a part of a computing device or controller that can be communicatively coupled to a computing system. For example, the memory resource 206 can be part of a device 102 as referenced in Figure 1. In some examples, the memory resource 206 can be communicatively coupled to a processor 204 that can execute instructions 222, 224, 226, 228, 230 stored on the memory resource 206, For example, the memory resource 206 can be communicatively coupled to the processor 204 through a communication path 220. In some examples, a communication path 220 can include a wired or wireless connection that can allow communication between devices and/or components within a single device.

[0028] The memory resource 206 may be electronic, magnetic, optical, or other physical storage device that stores executable instructions. Thus, a non- transitory machine-readable medium (MRM) (e.g., a memory resource 206) may be, for example, a non-transitory MRM comprising Random-Access Memory (RAM), read-only memory (ROM), an Electrically-Erasable Programmable ROM (EEPROM), a storage drive, an optical disc, and the like. The non-transitory machine-readable medium (e.g., a memory resource 206) may be disposed within a controller and/or computing device. In this example, the executable instructions 222, 224, 226, 228, 230, can be “installed” on the device. Additionally, and/or alternatively, the non- transitory machine-readable medium (e.g., a memory resource) can be a portable, external, or remote storage medium, for example, that allows a computing system to download the instructions 222, 224, 226, 228, 230, from the portable/external/remote storage medium. In this situation, the executable instructions may be part of an “installation package”.

[0029] In some examples, the memory resource 206 can include instructions 222 to instruct an imaging device to capture an image of a circuit board including a plurality of components. As described herein, the imaging device includes a camera that captures an image of a physical device. In some examples, the physical device is a circuit board. As used herein, a circuit board includes a substrate that includes an electrical circuit. In some examples, the circuit board is a printed circuit board (PCB) or printed circuit assembly (PGA) that can be utilized with a computing device or computing system.

[0030] As described herein, a circuit board includes a plurality of components that can be attached or coupled to a substrate of the circuit board. In some examples, the plurality of components can be electrical components that are coupled to other components through cables or other types of connections. As described herein, a misplaced component, a component that is coupled to the substrate at an incorrect location, and/or coupled to an incorrect component can result in the circuit board malfunctioning during use. In some examples, memory resource 206 includes instructions to train a model for a particular component of the plurality of components utilizing image samples associated with an accepted state of the particular component and image samples associated with a failed state of the particular component. As described further herein, the memory resource 206 includes instructions to compare boundary boxes from the model to the first set of boundary boxes and the second set of boundary boxes to determine a state of the plurality of components.

[0031] In some examples, the memory resource 206 can include instructions 224 to generate a first set of boundary boxes for the plurality of components based on an alignment model. As used herein, a boundary box is utilized to virtually surround an object, such as a representation of a component, to analyze the component or a position of the component. In some examples, the first set of boundary boxes can each surround a corresponding component of the plurality of components. For example, the plurality of components include a first screw and a second screw. In this example, the first set of boundary boxes includes a first boundary box at a predicted location of the first screw and a second boundary box at a predicted location of the second screw.

[0032] In some examples, the predicted location is based on a location of the corresponding component within the reference image. In this way, the location of the first screw within the reference image is utilized to generate the first boundary box at a corresponding location of the captured image and the location of the second screw within the reference image is utilized to generate a second boundary box at a corresponding iocation of the captured image. Although examples are provided related to screws of the device, this can be utilized for the other components of the plurality of components

[0033] In some examples, the predicted location of the component is calculated utilizing Equation 1. Equation 1 includes Breference that can include four points that form the corners of the boundary box of a component within the reference image. Equation 1 includes T that defines a transformation map that maps a plurality of points of the reference image to a corresponding plurality of points of the captured image. In addition, Equation 1 includes Binput that includes four points that form the corners of the boundary box of the component within the captured image (e.g., input image captured by the imaging device.

Binput “ T( Breference)

Equation 1

[0034] In some examples, each boundary box for each of the plurality of components that include a boundary box can have a corresponding cropped image utilizing the boundary box as the boundaries of the cropped image. In this way, the portion within the boundary box is utilized for a plurality of testing models as described further herein. In some examples, the boundary boxes generated based on the transition data (e.g., transformation map, transition map, coordinate data, etc.) can include errors or be slightly off of center of a component. In this way, a second set of boundary boxes can be generated for the plurality of components.

[0035] In some examples, the memory resource 206 can include instructions 226 to generate a second set of boundary boxes for the plurality of components based on a detection model. As used herein, a detection model is a method of identifying a particular object within an image based on the image profiles. For example, the second set of boundary boxes can correspond to the same components for the first set of boundary boxes. Thus, in some examples, the first set of boundary boxes can correspond to screws and the second set of boundary boxes are generated to by a detection model to identify screws within the captured image or input image.

[0036] In some examples, the detection model can be utilized separately from the alignment model. In other examples, the alignment model can be utilized to generate the first set of boundary boxes and the detection model can be utilized to detect the object within an area of the first set of boundary boxes. In this way, the detection model can focus on particular areas within the captured image instead of being performed on the entire captured image.

[0037] In some examples, the memory resource 206 can include instructions 228 to compare the first set of boundary boxes to the second set of boundary boxes. In some examples, the first set of boundary boxes and the second set of boundary boxes will cover a different portion of the captured image. For example, the first set of boundary boxes may not be centered on the particular component to be identified while the second set of boundary boxes may be more closely centered on the particular component compared to the first set of boundary boxes. In some examples, the first set of boundary boxes may not correspond to a component if the component was mistakenly not installed. In these examples, the detection model may not identify a component in that location and the second set of boundary boxes may not include a boundary box in that location. In this way, a number of missing components can be identified by a test model as described further herein.

[0038] In some examples, the memory resource 206 can include instructions 230 to identify the plurality of components when the first set of boundary boxes intersect the second set of boundary boxes for corresponding components of the plurality of components. In some examples, it can be determined that the first set of boundary boxes and the second set of boundary boxes are focusing on the same corresponding components when the first set of boundary boxes overlap the second set of boundary boxes. In these cases where there is overlap, the second set of boundary boxes can be utilized for the model tests.

[0039] However, if the first set of boundary boxes and the second set of boundary boxes do not overlap, the first set of boundary boxes can be utilized to generate the cropped portions for the model tests. For example, the memory resource 206 includes instructions to utilize the first set of boundary boxes for a test associated with a particular component when the first set of boundary boxes associated with the particular component does not intersect the second set of boundary boxes associated with the particular component.

[0040] As described herein, if there is no overlap between a particular boundary box from the first set of boundary boxes and a corresponding boundary box from the second set of boundary boxes, a component corresponding to that location may be missing from the circuit board. [0041] Figure 3 illustrates an example of a system 332 for determining device abnormalities. In some examples, the system 332 includes a device 302 that includes a processor 304 communicatively coupled to a memory resource 306. In some examples, the device 302 can include a computing device that includes a processor 304 and a memory resource 306 storing instructions 334, 336, 338, 340, that are executed by the processor 304 to perform particular functions.

[0042] In some examples, the system 332 includes an imaging device 316 to capture images of a printed circuit board (RGB) 318. In some examples, the RGB 318 includes a physical device that includes a particular specification for connections and locations of a plurality of components. For example, the PCB 318 includes a plurality of connections, cables, and/or computing components positioned on the PCB 318. In this way, the device 302 inspects the PCB 318 based on a plurality of tests performed on a captured image received from the imaging device 316. In some examples, the device 302 instructs the imaging device 316 to capture an image of the PCB 318. In these examples, the imaging device sends the captured image to the device 302 through a communication path 320-1.

[0043] The system 332 includes a database 342 that stores information related to a plurality of devices including the PCB 318. In some examples, the database 342 includes reference images for a plurality of devices including the PCB 318. As described herein, a reference image is an image of a device that is assembled to a particular specification and includes particular properties. In this way, the reference image can be utilized as a key or a guide to ensure that a plurality of components of the PCB 318 are installed or aligned according to the particular specification. In some examples, the device 102 extracts a reference image or previously captured image of a corresponding device to the PCB 318 to be utilized as a reference image through communication path 320-2.

[0044] The device 302 includes instructions 334 stored by the memory resource 306 that can be executed by the processor 304 to align the captured image of the PCB 318 with the previously captured images of the corresponding PCB. As described herein, the database 342 includes reference images for a plurality of devices including the PCB 318. The device 302 utilizes the communication path 320- 2 to extract a previously captured image a corresponding PCB to utilize as a reference image for the PCB 318. [0045] The device 302 includes instructions 336 stored by the memory resource 306 that can be executed by the processor 304 to generate a board model for the PCB 318 based on a comparison between the aligned captured image of the PCB 318 and the previously captured images of the corresponding PCB. in some examples, the board model can be an image overlay of the PCB 318 within the captured image over the corresponding PCB in the reference image. In this way, the board model can be a mapping of pixels between the captured image and the reference image when the two images are aligned based on the components. In some examples, the board model includes aligning pixels of the captured image of the PCB 318 with the previously captured image of the corresponding PCB. In this way, an equation can be utilized to identify locations of the plurality of components on the PCB 318 within the captured image.

[0046] The device 302 includes instructions 338 stored by the memory resource 306 that can be executed by the processor 304 to generate a component model for the plurality of components based on component detection. In these examples, the component detection is based on an indicated location of the plurality of components of the previously captured images. As described herein, the previously captured image or images of the corresponding PCB can be utilized to identify the location of the plurality of components on the PCB 318 within the captured image. In some examples, a component detection model can be utilized to identify components within the captured image. In these examples, boundary boxes can be generated for the plurality of components and cropped images of the plurality of components can be generated based on the coordinates of the boundary boxes. In this way, a component model can be generated utilizing the plurality of cropped images of the plurality of components.

[0047] The device 302 includes instructions 340 stored by the memory resource 306 that can be executed by the processor 304 to perform a plurality of tests utilizing the board model and the component model to determine when the PCB is in an accepted state or a failed state. As described herein, the plurality of tests can be performed independently such that each of the plurality of tests can be utilized to generate an accepted state or a failed state. As used herein, an accepted state refers to a test determining that the component is within the particular specification of the component. As used herein, a failed state refers to a test determining that the component is not within the particular specification or is outside the particular specification of the component.

[0048] In some examples, the plurality of tests utilize the cropped images of the plurality of components to determine the state of the plurality of components and/or a state of the PCB 318. In some examples, the plurality of tests include, but are not limited to: a cable on hook test, a counting test, a misplaced test, a screw test, a component test, a cable routing test, among other tests that are generated to determine a state of the plurality of components and/or the PCB 318. As used herein, cable on hook test utilizes the cropped images of hooks positioned on the PCB 318 to determine if a particular hook is capturing a corresponding cable. As used herein, a hook refers to a cable management device to secure a portion of the cable while allowing a portion of the cable to be unsecure.

[0049] As used herein, a counting test includes determining a quantity of each of the plurality of components on the PCB 318 within the captured image. In some examples, the quantity of the plurality of components on the PCB 318 is compared to the quantity of components within the reference image. In some examples, the counting test includes determining a quantity of each type of component from the plurality of components. For example, the counting test can include determining the quantity of screws, determining the quantity of cables, and/or determining the quantity of hooks positioned on the PCB 318 within the captured image. The quantity of the plurality of components can be utilized to compare to the determined quantity of the components within the reference image or within data associated within the database 342.

[0050] As used herein, a misplaced test includes determining that a component is located in a different location than what is identified by the reference image. In some examples, the misplaced test includes identifying that a particular cable is positioned at a particular location based on the plurality of cropped images. As used herein, the screw test includes identifying the presence of a plurality of screws connected to the PCB 318 within the captured image. In some examples, the component test includes a similar identification process for other components of the plurality of components.

[0051] As used herein, the cable misrouting test includes utilizing a cable mask to determine whether a cable is following a particular path from a first point to a second point. For example, a mask can be positioned on the captured image based on a position of the cabie in the reference image to determine if the cable in the captured image is following the same or similar path. Additional tests can be added or fewer tests can be utilized without affecting the outcomes of the other plurality of tests. In this way, the plurality of test are independently performed or executed and independently generate an accepted state or failed state as a result.

[0052] Figure 4 illustrates an example of a method 450 for determining device abnormalities. In some examples, the method 450 is executed by a computing device or system. For example, the method 450 is executable by the system 332 as referenced in Figure 3. In some examples, the method 450 is utilized to identify abnormalities or inconsistencies between a printed circuit board (PCB) and a reference image of a corresponding circuit board that includes a plurality of components that have been determined to be accepted. In this way, the PCB is able to be inspected through a plurality of tests that can be developed specifically for the particular PCB.

[0053] In some examples, the method 450 includes capturing an input image 452. In some examples, capturing an input image includes utilizing an imaging device to capture a digital image of the PCB to be utilized for comparison with a reference image. In some examples, the input image may not have the same or similar orientation as the input image. For example, the imaging device may not be positioned at the same angle or position as an imaging device that previously captured the reference image. In some examples, the input image is captured when a PCB is to be inspected to determine the state of the PCB.

[0054] In some examples, the method 450 includes utilizing model board detection 454 to determine a type of PCB that is captured in the input image 452. In some examples, a reference image is selected based on the determined type or model of PCB. In this way, the selected reference image is utilized to compare with the input image 452 to determine the state of the input image 452.

[0055] In some examples, the method 450 includes performing an image alignment 454. In some examples, the image alignment 454 includes aligning the captured image or input image 452 with the reference image. As described herein, a mask can be utilized to remove the boundary of the PCB such that only the PCB and the plurality of components are viewable within the input image 452. In some examples, the input image 452 is segmented or split into a plurality of segments and segments that do not include a portion of the PCB are masked or removed from the input image 452.

[0056] In some examples, the method 450 includes utilizing the input image 452 model component detection 456. As described herein, model component detection 456 includes a model to detect the plurality of components of the PCB. In some examples, the plurality of components can be identified utilizing boundary boxes and the boundary boxes are utilized to generate cropped images of the plurality of components such that each of the cropped images include a corresponding component of the plurality of components. For example, the method 450 includes storing bounding boxes and segmentation masks 458. The bounding boxes and segmentation masks included the cropped images of each of the plurality of components of the PCB. In some examples, the cropped images and/or a portion of the cropped images are utilized to perform anomaly detection tests 460.

[0057] The anomaly detection tests 460 include the model tests described herein to identify if components are positioned in a particular location, that the quantity of components correspond to the reference image, and/or other tests that identify differences between the reference image and the input image 452. As described herein, the anomaly detection tests 460 can be individual test to allow for one test to not be affected by a different test. In this way, detailed anomaly detection is performed by the method 450.

[0058] Based on the anomaly detection tests 460, the method 450 determines whether the PCB within the input image 452 normal 464 or abnormal 462. In some examples, normal 464 includes a PCB within the input image 452 that has passed inspection and complies with the specification of the PCB. In other examples, abnormal 462 includes a PCB within the input image 452 that has failed inspection and does not comply with the specification. In some examples, the abnormal 462 state includes generating a report that includes cropped images of the input image 452 and corresponding cropped images from the reference image to identify reasons why the input image 452 received an indication of abnormal 462.

[0059] Figure 5 illustrates an example of a method 570 for determining device abnormalities. In some examples, the method 570 is executable by a computing device and/or system to determine abnormalities of a device such as a PCB. In some examples, the method 570 includes determining a component location 572 for a plurality of components connected to a device. As described herein, the component location 572 determination is utilized to determine a location within an input image or captured image of a component positioned on the device. In some examples, the component location for a plurality of components of the device are compared to a corresponding component location within a reference image corresponding to the device within the input image.

[0060] In some examples, the method 570 includes performing an alignment 574 between the input image and a reference image to align the components of the device within the input image and the device within the reference image. In this way, the location and presence of the plurality of components are identified within the input image to ensure the plurality of components are installed correctly according to a particular specification.

[0061] In some examples, the method 570 includes performing a crop generation 576 for the plurality of components. In some examples, a bounding box is utilized to surround the plurality of components such that the bounding box is utilized to crop out each of the plurality of components from the input image. In this way, a cropped component image from the input image is compared to a corresponding cropped component image from the reference image. In some examples, the plurality of cropped images of the plurality of components are utilized in an anomaly model 578. The anomaly model 578 includes a plurality of tests that are executed utilizing the plurality of cropped component images from the input image and/or the plurality of cropped images from the reference image. In this way, the anomaly model 578 is utilized to determine if the device within the input image is abnormal 580 or normal 582.

[0062] Figure 6 illustrates an example of an image 690 for determining device abnormalities. In some examples, the image 690 illustrates the bounding boxes 694, 696 surrounding a component 692. In some examples, the component 692 is a screw, but could be a plurality of other types of components that are coupled to a device such as a PCB.

[0063] In some examples, the image 690 is an input image. As described herein, the image 690 is a captured image of portion of a PCB. The image 690 illustrates a screw that is connected to a portion of the PCB, but different components of the PCB are also identified in a similar way. In some examples, the first boundary box 694 is a boundary box based on a transitional location that is determined utilizing the location data of the component within the reference image to identify a predicted location of the component 692 within the image 690. As described herein, the predicted location can be calculated utilizing Equation 1. [0064] In some examples, the second boundary box 696 is generated utilizing a component detection model performed on the image 690. In these examples, the component detection model is utilized to search for the component 692 within the image 690 and/or search for the component 692 within a proximate area of the first bounding box 694. In some examples, the component detection model that generates the second bounding box 696 is relatively more accurate than the predicted location of the transitional location predicted location. That is, the second bounding box 694 may be more accurate than the first bounding box 696.

[0065] In some examples, the first bounding box 694 are compared to the second bounding box 696 to determine when the first bounding box 694 intersects or overlaps the second bounding box 696. When the first bounding box 694 overlaps a portion of the second bounding box 696 as illustrated within the image 690, the second bounding box 696 is utilized to crop an image of the component 692 utilizing the second bounding box 696. In some examples, the first boundary box 694 is selected to utilize for generating a crop of the image of the component 692 when the first boundary box 694 does not overlap the second boundary box 696. In some examples, the second boundary box 696 may not overlap or intersect with the first boundary box 694 when the component detection model does not identify the existence of the component 692. In these examples, the component 692 may have not been installed or may not have been installed correctly according to a specification for the device.

[0066] Figure 7 illustrates an example of an image 701 for determining device abnormalities. In some examples, the image 701 is utilized to determine whether the plurality of components are correctly positioned on a device. In some examples, determining whether a component is positioned at a correct location can be based on a distance 707 between a first component identified by a first boundary box 703 of a component within the reference image and a second component identified by a second boundary box 705.

[0067] As described herein, the first boundary box 703 and the second boundary box 705 are selected based on a transitional model and/or a component detection model. In some examples, the distance 707 illustrates a difference in a location between the same component within the input image and a reference image. In this way, the component is determined to be normal if the distance 707 is below a threshold distance and the component is determined to be abnormal if the distance 707 is above a threshold distance. In this way, the boundary boxes 703, 705 are utilized to determine when a component is positioned at an acceptable position on the device.

[0068] In the foregoing detailed description of the disclosure, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration how examples of the disclosure may be practiced. These examples are described in sufficient detail to enable those of ordinary skill in the art to practice the examples of this disclosure, and it is to be understood that other examples may be utilized and that process, electrical, and/or structural changes may be made without departing from the scope of the disclosure. Further, as used herein, “a” refers to one such thing or more than one such thing.

[0069] The figures herein follow a numbering convention in which the first digit corresponds to the drawing figure number and the remaining digits identify an element or component in the drawing. For example, reference numeral 102 may refer to element 102 in Figure 1 and an analogous element may be identified by reference numeral 302 in Figure 3. Elements shown in the various figures herein can be added, exchanged, and/or eliminated to provide additional examples of the disclosure. In addition, the proportion and the relative scale of the elements provided in the figures are intended to illustrate the examples of the disclosure, and should not be taken in a limiting sense.

[0070] It can be understood that when an element is referred to as being "on," "connected to", “coupled to”, or "coupled with" another element, it can be directly on, connected, or coupled with the other element or intervening elements may be present. In contrast, when an object is “directly coupled to” or “directly coupled with” another element it is understood that are no intervening elements (adhesives, screws, other elements) etc.

[0071] The above specification, examples, and data provide a description of the system and methods of the disclosure. Since many examples can be made without departing from the spirit and scope of the system and method of the disclosure, this specification merely sets forth some of the many possible example configurations and implementations.

Claims

What is claimed is:

1. A computing device, comprising: an imaging device to capture an image of a device comprising a plurality of components; a processor to: access a reference image associated with the device; mask a boundary of the captured image of the device; align the captured image with the reference image based on a location of the plurality of components of the device; and determine an abnormality of the device based on: a comparison between the reference image and the captured image; and a test model applied to the captured image.

2. The computing device of claim 1, wherein the processor is to compare the plurality of components within the captured image to a corresponding plurality of components within the reference image.

3. The computing device of claim 1 , wherein the processor is to mask portions of the captured image that surround the device comprising the plurality of components.

4. The computing device of claim 1, wherein the reference image is an image of the device without abnormalities.

5. The computing device of claim 1, wherein the processor is to determine heuristic information associated with the plurality of components.

6. The computing device of claim 1, wherein the processor is to calculate an angle of a cable of the plurality of components based on a cable segmentation mask.

7. The computing device of claim 6, wherein the processor is to determine when the angle of the cable is between an upper bound and a lower bound.

8. A non-transitory memory resource storing machine-readable instructions stored thereon that, when executed, cause a processor of a computing device to: instruct an imaging device to capture an image of a circuit board including a plurality of components; generate a first set of boundary boxes for the plurality of components based on an alignment model; generate a second set of boundary boxes for the plurality of components based on a detection model; compare the first set of boundary boxes to the second set of boundary boxes; and identify the plurality of components when the first set of boundary boxes intersect the second set of boundary boxes for corresponding components of the plurality of components.

9. The memory resource of claim 8, wherein the processor is to utilize the first set of boundary boxes for a test associated with a particular component when the first set of boundary boxes associated with the particular component does not intersect the second set of boundary boxes associated with the particular component.

10. The memory resource of claim 8, wherein the processor is to train a model for a particular component of the plurality of components utilizing image samples associated with an accepted state of the particular component and image samples associated with a failed state of the particular component.

11. The memory resource of claim 10, wherein the processor is to compare boundary boxes from the model to the first set of boundary boxes and the second set of boundary boxes to determine a state of the plurality of components.

12. A system, comprising: an imaging device to capture an image of a printed circuit board (PCB) that includes a plurality of components that have been installed on the PCB: and a database storing previously captured images of a corresponding PCB, wherein the previously captured images include the corresponding PCB in an accepted state and a failed state; and a processor to: align the captured image of the PCB with the previously captured images of the corresponding PCB; generate a board model for the PCB based on a comparison between the aligned captured image of the PCB and the previously captured images of the corresponding PCB; generate a component model for the plurality of components based on component detection, wherein the component detection is based on an indicated location of the plurality of components of the previously captured images; and perform a plurality of tests utilizing the board model and the component model to determine when the PCB is in an accepted state or a failed state.

13. The system of claim 12, wherein the plurality of tests includes a counting test to count the plurality of components installed on the PCB to a pre-identified quantity of components installed on the corresponding PCB.

14. The system of claim 12, wherein the plurality of tests includes an interaction test to determine when a first component of the plurality of components is interacting with a second component of the plurality of components.

15. The system of claim 12, wherein the plurality of tests are independent tests where a first test of the plurality of tests is unaffected by a second test of the plurality of tests.