CN116703926B - Defect detection method, defect detection device, computer equipment and computer readable storage medium - Google Patents

Defect detection method, defect detection device, computer equipment and computer readable storage medium Download PDF

Info

Publication number
CN116703926B
CN116703926B CN202310988253.3A CN202310988253A CN116703926B CN 116703926 B CN116703926 B CN 116703926B CN 202310988253 A CN202310988253 A CN 202310988253A CN 116703926 B CN116703926 B CN 116703926B
Authority
CN
China
Prior art keywords
boundary line
window
path point
current position
acquisition window
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310988253.3A
Other languages
Chinese (zh)
Other versions
CN116703926A (en
Inventor
刘思丹
周超
吕江波
沈小勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Simou Intelligent Technology Co ltd
Shenzhen Smartmore Technology Co Ltd
Original Assignee
Suzhou Simou Intelligent Technology Co ltd
Shenzhen Smartmore Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Simou Intelligent Technology Co ltd, Shenzhen Smartmore Technology Co Ltd filed Critical Suzhou Simou Intelligent Technology Co ltd
Priority to CN202310988253.3A priority Critical patent/CN116703926B/en
Publication of CN116703926A publication Critical patent/CN116703926A/en
Application granted granted Critical
Publication of CN116703926B publication Critical patent/CN116703926B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/60Analysis of geometric attributes
    • G06T7/62Analysis of geometric attributes of area, perimeter, diameter or volume
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30121CRT, LCD or plasma display

Abstract

The application relates to a defect detection method, a defect detection device, computer equipment and a computer readable storage medium. The method comprises the following steps: acquiring a regional frame set; generating a first boundary line, a second boundary line, a third boundary line and a fourth boundary line; determining the current position of the path point acquisition window in the target area based on the first boundary line and the second boundary line, and determining the path point based on the current position of the path point acquisition window; updating the current position of the path point acquisition window towards the direction close to the third boundary line based on the frames of the areas included in the current position of the first boundary line and the path point acquisition window, and updating the position of the first boundary line towards the direction close to the fourth boundary line under the condition that the path point acquisition window is intersected with the third boundary line until the path point acquisition window is intersected with both the third boundary line and the fourth boundary line; a defect detection path is generated based on at least a portion of the path points. The defect detection efficiency can be improved by adopting the method.

Description

Defect detection method, defect detection device, computer equipment and computer readable storage medium
Technical Field
The present application relates to the field of computer vision, and in particular, to a defect detection method, a defect detection device, a defect detection computer device, and a defect detection computer readable storage medium.
Background
With the development of computer vision technology, defect detection technology has emerged, for example, defect detection can be performed on a plurality of elements distributed on an electronic panel, which may be a circuit board or an LED (Light Emitting Diode ) backlight board. Since the size of the inspection field of view for defect inspection is smaller than the size of the electronic panel, it is necessary to plan the defect inspection path in advance before performing defect inspection on the electronic panel.
In the conventional art, a defect detection path is generally planned based on a distance between adjacent elements on an electronic panel.
However, since the components on the electronic panel are generally randomly distributed, the types and the sizes of the components are not consistent, and a defect detection path is planned based on the distance between adjacent components on the electronic panel, which takes a long time, and the defect detection is easy to have the condition of missing detection or repeated detection, so that the defect detection efficiency is low.
Disclosure of Invention
The application provides a defect detection method, a defect detection device, computer equipment and a computer readable storage medium, which can improve the defect detection efficiency.
In a first aspect, the present application provides a defect detection method, including:
acquiring the region borders of each element region in the image of the electronic panel to obtain a region border set;
generating a first boundary line and a fourth boundary line in a first direction, and generating a second boundary line and a third boundary line in a second direction; each region frame in the region frame set is positioned in a target region surrounded by the first boundary line, the second boundary line, the third boundary line and the fourth boundary line;
determining the current position of a path point acquisition window in the target area based on a first boundary line and the second boundary line, and updating the current position of the path point acquisition window based on each area frame included by the path point acquisition window in the current position;
after updating, determining a path point based on the current position of the path point acquisition window;
updating the current position of the waypoint acquisition window towards the direction close to the third boundary line based on the first boundary line and each region frame included in the waypoint acquisition window at the current position, and updating the position of the first boundary line towards the direction close to the fourth boundary line in the case that the waypoint acquisition window intersects with the third boundary line, returning to the step of determining the current position of the waypoint acquisition window in the target region based on the first boundary line and the second boundary line until the waypoint acquisition window intersects with both the third boundary line and the fourth boundary line;
Generating a defect detection path based on at least a portion of the path points; when the electronic panel is detected, the defect detection is carried out according to the defect detection path.
In a second aspect, the present application also provides a defect detection apparatus, including:
the region frame acquisition module is used for acquiring the region frames of the element regions in the image of the electronic panel to obtain a region frame set;
a boundary line generation module for generating a first boundary line and a fourth boundary line in a first direction and generating a second boundary line and a third boundary line in a second direction; each region frame in the region frame set is positioned in a target region surrounded by the first boundary line, the second boundary line, the third boundary line and the fourth boundary line;
the position determining module is used for determining the current position of the path point acquisition window in the target area based on the first boundary line and the second boundary line, and updating the current position of the path point acquisition window based on each area frame included by the path point acquisition window in the current position;
the path point determining module is used for determining a path point based on the current position of the path point acquisition window after updating;
A position updating module, configured to update a current position of the waypoint acquisition window toward a direction approaching the third boundary line based on each region border included in the current position of the first boundary line and the waypoint acquisition window, and to update a position of the first boundary line toward a direction approaching the fourth boundary line in a case where the waypoint acquisition window intersects the third boundary line, and to return to the step of determining the current position of the waypoint acquisition window in the target region based on the first boundary line and the second boundary line until the waypoint acquisition window intersects both the third boundary line and the fourth boundary line;
a path generation module for generating a defect detection path based on at least part of the path points; when the electronic panel is detected, the defect detection is carried out according to the defect detection path.
In a third aspect, the present application also provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the steps in the above defect detection method when executing the computer program.
In a fourth aspect, the present application also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the above defect detection method.
In a fifth aspect, the present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the above-described defect detection method.
According to the defect detection method, the device, the computer equipment, the computer readable storage medium and the computer program product, the regional border sets of the component regions in the image of the electronic panel are obtained through obtaining the regional borders of the component regions, the first border line and the fourth border line are generated in the first direction, the second border line and the third border line are generated in the second direction, the current position of the route point acquisition window in the target region is determined based on the first border line and the second border line, the current position of the route point acquisition window is continuously updated towards the direction close to the third border line until the route point acquisition window is intersected with the third border line, the position of the first border line is updated towards the direction close to the fourth border line until the route point acquisition window is intersected with the third border line and the fourth border line, so that new route points are continuously obtained, a defect detection path can be quickly generated based on at least part of route points under the condition that the components on the electronic panel are in various types, multiple sizes and are randomly distributed, and in the process of detecting the defects of the electronic panel by using the generated defect detection path, the repeated detection or the condition of the detected components is less, so that the defect detection efficiency is improved.
Drawings
FIG. 1 is an application environment diagram of a defect detection method according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of a defect detection method according to an embodiment of the present application;
fig. 3 is a schematic diagram of a region frame in an image of an electronic panel according to an embodiment of the present application;
fig. 4A is a schematic diagram of boundary lines of a target area according to an embodiment of the present application;
fig. 4B is a schematic diagram of an initial position and an updated position of a waypoint acquisition window in a target area according to an embodiment of the present application;
FIG. 4C is a schematic diagram illustrating a method for updating a current position of a waypoint collection window to the right according to an embodiment of the present application;
FIG. 4D is a schematic diagram showing an updated first boundary line updating the current position of the waypoint collection window according to an embodiment of the present application;
FIG. 4E is a schematic diagram illustrating a step of screening waypoints according to an embodiment of the present application;
FIG. 5 is a flowchart illustrating another defect detection method according to an embodiment of the present application;
FIG. 6 is a block diagram illustrating a defect detecting apparatus according to an embodiment of the present application;
FIG. 7 is an internal block diagram of a computer device in accordance with an embodiment of the present application;
FIG. 8 is an internal block diagram of another computer device in accordance with an embodiment of the present application;
fig. 9 is an internal structural diagram of a computer-readable storage medium in an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The defect detection method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. The application environment includes a computer device 102 and a server 104, wherein the computer device 102 communicates with the server 104 over a communication network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server.
Specifically, the computer device 102 obtains an image of the electronic panel, for example, from the server 104, and from other devices. Then the computer device 102 obtains the region borders of each element region in the image of the electronic panel to obtain a region border set; generating a first boundary line and a fourth boundary line in a first direction, and generating a second boundary line and a third boundary line in a second direction; each region border in the region border set is positioned in a target region surrounded by the first border line, the second border line, the third border line and the fourth border line; the computer device 102 determines a current position of the path point acquisition window in the target area based on the first boundary line and the second boundary line, and updates the current position of the path point acquisition window based on each area frame included in the path point acquisition window under the current position; after updating, determining a path point based on the current position of the path point acquisition window, updating the current position of the path point acquisition window towards the direction close to the third boundary line based on the first boundary line and each region frame included in the path point acquisition window in the current position, and updating the position of the first boundary line towards the direction close to the fourth boundary line when the path point acquisition window intersects with the third boundary line, and returning to the step of determining the current position of the path point acquisition window in the target region based on the first boundary line and the second boundary line until the path point acquisition window intersects with both the third boundary line and the fourth boundary line. Computer device 102 generates a defect detection path based on at least a portion of the path points; when the electronic panel is detected, the defect detection is carried out according to the defect detection path. The computer device 102 may save the defect detection path and may also send the defect detection path to the server 104. The server 104 receives and saves the defect detection path.
Wherein the computer device 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The portable wearable device may be a smart watch, smart bracelet, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.
In some embodiments, as shown in fig. 2, a defect detection method is provided, and the method is applied to the computer device 102 in fig. 1 for illustration, and includes the following steps:
step 202, obtaining the region borders of each element region in the image of the electronic panel to obtain a region border set.
The electronic panel is a carrier for electrically connecting electronic components, and includes a plurality of elements thereon, for example, the electronic panel may be a printed circuit board or an LED backlight board, and the LED backlight board may be a Mini-LED backlight board. The image of the electronic panel may be a design drawing of the electronic panel, or may be an image obtained by photographing the electronic panel. The image of the electronic panel includes an element area, where the element area is an image area corresponding to an element in the electronic panel, and one element area may correspond to one or more elements. The region frame is a frame obtained by labeling the element region, and may be rectangular, for example, an circumscribed rectangular frame of the element region, or may be other shapes, such as a circle, a polygon, or the like. The region frame set is composed of region frames of all the element regions on the image of the electronic panel.
Specifically, the computer device may acquire an image of the electronic panel, and then determine a region frame of each element region in the image of the electronic panel, to obtain a region frame set. As shown in fig. 3, the region frame of a part of the element region in the image of the electronic panel is shown, the circumscribed rectangular frame of one element region is a region frame, the central positions of the three path point collection windows are respectively corresponding to a digital number 112, 113 and 114, the digital numbers are used for representing the arrangement sequence of the path points corresponding to the path point collection windows, the arrangement sequence can be determined according to the time sequence of collection, for example, the number 112 represents the path point corresponding to the 112 th collection.
Step 204 of generating a first boundary line and a fourth boundary line in a first direction and generating a second boundary line and a third boundary line in a second direction; each region border in the region border set is located in a target region surrounded by the first border line, the second border line, the third border line and the fourth border line.
The first direction and the second direction are perpendicular to each other, the first boundary line and the fourth boundary line are generated in the first direction, the second boundary line and the third boundary line are generated in the second direction, and all the region frames in the region frame set are included in the target region surrounded by the first boundary line, the second boundary line, the third boundary line and the fourth boundary line, for example, the first direction may be a horizontal direction, the second direction may be a vertical direction, the first boundary line and the fourth boundary line are two non-coincident horizontal straight lines, and the second boundary line and the third boundary line are two non-coincident vertical straight lines.
Specifically, the computer device may determine a minimum bounding rectangle including each region border in the set of region borders, and then determine a first direction and a second direction based on the minimum bounding rectangle, for example, may determine a direction of a straight line where a rectangular lower border of the minimum bounding rectangle is located as the first direction, determine a direction of a straight line where a rectangular left border of the minimum bounding rectangle is located as the second direction, generate a first boundary line and a fourth boundary line parallel to each other in the first direction, generate a second boundary line and a third boundary line parallel to each other in the second direction, and make a target region surrounded by the first boundary line, the second boundary line, the third boundary line, and the fourth boundary line may completely include the minimum bounding rectangle. As shown in fig. 4A, a schematic diagram in which each region frame is rectangular is shown, and a target region enclosed by a first boundary line, a second boundary line, a third boundary line, and a fourth boundary line may be a minimum external moment, where the first boundary line may also be referred to as a global lower boundary, the fourth boundary line may also be referred to as a global upper boundary, the second boundary line may also be referred to as a global left boundary, and the third boundary line may also be referred to as a global right boundary.
And 206, determining the current position of the path point acquisition window in the target area based on the first boundary line and the second boundary line, and updating the current position of the path point acquisition window based on each area frame included in the path point acquisition window in the current position.
The size and shape of the path point collection window are consistent with those of a detection area for detecting the defects, namely the area under a shooting view field for shooting the electronic panel in the defect detection process, is generally rectangular, and the size of the path point collection window is far smaller than that of a target area. The current position of the path point acquisition window in the target area is continuously updated, so that the path point acquisition window is continuously moved in the target area, and the moving direction is consistent with the moving direction of image acquisition in the defect detection process, for example, the path point acquisition window can be moved from left to right line by line or from top to bottom line by line. The current position may be represented by the center point of the path point acquisition window, and may also be represented by the position of the window boundary line of the path point acquisition window.
The path point acquisition window is provided with a plurality of window boundary lines, a first window boundary line in the first direction, a fourth window boundary line in the first direction and the first boundary line are parallel to each other, and the distance between the first window boundary line in the first direction and the first boundary line is smaller than the distance between the fourth window boundary line in the first direction and the first boundary line; the second window boundary line in the second direction, the third window boundary line in the second direction, and the second boundary line are parallel to each other, and a distance between the second window boundary line in the second direction and the second boundary line is smaller than a distance between the third window boundary line in the second direction and the second boundary line, for example, in a case where the first boundary line is a global lower boundary and the second boundary line is a global left boundary, the first window boundary line in the first direction, the fourth window boundary line in the first direction, the second window boundary line in the second direction, and the third window boundary line in the second direction may be respectively the lower boundary, the upper boundary, the left boundary, and the right boundary of the point acquisition window.
Specifically, the computer device may determine an initial position of the path point collection window in the target area according to the position of the first boundary line and the position of the second boundary line, take the initial position as a current position, then determine each area frame included in the path point collection window in the current position, and update the current position of the path point collection window according to the respective boundary line of each area frame. The update is to make the number of region borders included in the path point acquisition window as much as possible. It is understood that, each region frame included in the current position of the path point acquisition window herein refers to a region frame in which all regions are located in the path point acquisition window, and a region frame in which part of regions are located in the path point acquisition window does not belong to each region frame included in the current position of the path point acquisition window.
For example, as shown in fig. 4B, the shape of the waypoint collection window is rectangular, the shape of the target area is rectangular, the first boundary line is a global lower boundary, the moving direction of the waypoint collection window is from left to right, the computer device may determine an initial position of the waypoint collection window in the target area based on the positions of the first boundary line and the second boundary line, the first window boundary line in the first direction of the waypoint collection window at the initial position coincides with the first boundary line, and the second window boundary line in the second direction of the waypoint collection window coincides with the second boundary line. The computer equipment takes the initial position as the current position, and updates the current position of the path point acquisition window according to the area frames included by the path point acquisition window in the current position, so that a first window boundary line in the first direction of the updated path point acquisition window coincides with the lower boundary line of at least one area frame in the area frames, and a second window boundary line in the second direction of the updated path point acquisition window coincides with the left boundary line of at least one area frame in the area frames.
And step 208, after updating, determining the path point based on the current position of the path point acquisition window.
The path point is determined based on the current position of the path point acquisition window, and the center position of the path point acquisition window, which can be the current position, can be represented by a coordinate point and is used for generating a defect detection path.
Specifically, after updating the current position of the waypoint collection window, the computer device may determine the center position of the waypoint collection window at the current position, take the center position as the waypoint, and record the waypoint, for example, may record coordinates of the waypoint in the target area.
In some embodiments, after determining the path point each time, the computer device may further mark each area border included in the path point collection window at the current position, and mark the area borders to distinguish the area borders traversed by the path point collection window from the area borders not traversed, so as to prevent the condition of missing detection elements during defect detection, where the element areas corresponding to the marked area borders are also referred to as planned elements, and the element areas corresponding to the unmarked area borders are also referred to as unplanned elements. For example, as shown in fig. 4C, each region frame in the initial state may be identified by 0, and each region frame included in the path point acquisition window in the current position may be marked as 1.
Step 210, updating the current position of the route point collection window towards the direction close to the third boundary line based on the respective region borders included in the current position of the first boundary line and the route point collection window, and updating the position of the first boundary line towards the direction close to the fourth boundary line in the case that the route point collection window intersects with the third boundary line, returning to the step of determining the current position of the route point collection window in the target region based on the first boundary line and the second boundary line until the route point collection window intersects with both the third boundary line and the fourth boundary line.
Specifically, the computer device redetermines each region border included in the path point collection window under the current position, in order to determine the next path point, the current position of the path point collection window may be updated towards a direction close to the third border line by using the first border line and each region border included in the path point collection window under the current position, and judges whether the path point collection window under the current position intersects with the third border line, and in the case that the path point collection window does not intersect with the third border line, in order to make the number of region borders included in the path point collection window under the current position greater, the computer device may return to perform the step of updating the current position of the path point collection window based on each region border included in the path point collection window under the current position; in the case where the route point collection window intersects the third boundary line, it is determined whether the route point collection window intersects the fourth boundary line, and in the case where the route point collection window does not intersect the fourth boundary line, the position of the first boundary line is updated toward a direction close to the fourth boundary line, and then the step of determining the current position of the route point collection window in the target area based on the first boundary line and the second boundary line is returned until the route point collection window intersects both the third boundary line and the fourth boundary line.
In some embodiments, to update the current position of the waypoint collection window toward a direction approaching the third boundary line, the computer device may determine, from each of the region borders included in the waypoint collection window at the current position, a region border point having a smallest distance from the third window boundary line in the second direction of the waypoint collection window, update the current position of the waypoint collection window toward a direction approaching the third boundary line according to the position of the first boundary line and the position of the region border point, and after updating, the first window boundary line in the first direction of the waypoint collection window coincides with the first boundary line, and the third window boundary line in the second direction of the waypoint collection window is a preset distance from the region border point. The region border point is determined from each region border included in the path point acquisition window in the current position, for example, as shown in fig. 4C, where the region border points are all rectangular, and may be any point on a right border line of the region border closest to the third border line in each region border included in the path point acquisition window in the current position.
In some embodiments, after each determination of the waypoint, the computer device may further mark each region border included in the waypoint collection window at the current location, and after the marking, confirm that the waypoint collection window intersects the third border line and does not intersect the fourth border line, the computer device may determine a third region border having the smallest distance from the first border line from among the unlabeled region borders in the region border set, and then update the location of the first border line toward a direction approaching the fourth border line based on the third region border line.
Step 212, generating a defect detection path based on at least part of the path points; when the electronic panel is detected, the defect detection is carried out according to the defect detection path.
The defect detection path is used for detecting defects of the electronic panel and is a series of coordinates consisting of coordinates of path points. The defect detection path can be generated based on partial path points or based on all path points, and the defect detection path is related to shooting modes for shooting the electronic panel in the defect detection process, wherein the shooting modes comprise a first shooting mode and a second shooting mode, the first shooting mode is also called stop-and-go shooting, and the mode of stopping shooting before shooting is performed when the image acquisition equipment moves to one shooting point in the moving shooting process; the second shooting mode is also called as fly shooting, and refers to a mode that the image acquisition device does not stop in the moving shooting process, and shoots when moving to a shooting point, when shooting is performed by adopting the second shooting mode, because shooting is performed in the moving process, if the moving displacement is large, the quality of the shot image is poor, and in order to improve the quality of the image, the displacement of the adjacent shooting points in the preset direction can be required to be not exceeding a distance threshold.
Specifically, the generating mode of the defect detection path is related to the shooting mode of defect detection, and in the case that the shooting mode is the first shooting mode, the computer device may generate the defect detection path by using all path points, for example, the path points may be ordered according to the time sequence of obtaining the path points to obtain a path point sequence, and the path point sequence is used as the defect detection path; in the case where the photographing mode is the second photographing mode, the computer apparatus may generate the defect detection path using the partial path point.
In some embodiments, since the position of the first boundary line is updated in the process of updating the current position of the path point acquisition window, the computer device may consider the path points determined by using the first boundary line in the same position as the path points belonging to the same row, so that all the determined path points are divided according to rows, and when the path points are ordered, the path points with odd row numbers are ordered according to the time sequence of obtaining the path points; and aiming at the path points with even line numbers, arranging the path points in a reverse order according to the time sequence of the obtained path points, and then combining the path point sequences of each line together according to the order of the line numbers from small to large so as to obtain a defect detection path arranged in a serpentine path.
In some embodiments, after the computer device generates the defect detection path, the defect detection path may be utilized to detect defects of the components on the electronic panel. The computer device may control the image acquisition device to perform image acquisition on the electronic panel at the position of each path point according to the arrangement sequence of the path points in the defect detection path, to obtain a target image set, and then perform defect detection on the target image set, for example, each target image in the target image set may be input into a trained defect detection network to perform defect detection, to obtain a defect detection result. The target image is an image obtained by acquiring the image of the electronic panel according to the defect detection path.
In the defect detection method, the region border set is obtained by obtaining the region borders of the element regions in the image of the electronic panel, the first border line and the fourth border line are generated in the first direction, the second border line and the third border line are generated in the second direction, the current position of the path point acquisition window in the target region is determined based on the first border line and the second border line, the current position of the path point acquisition window is continuously updated towards the direction close to the third border line until the path point acquisition window intersects with the third border line, the position of the first border line is updated towards the direction close to the fourth border line until the path point acquisition window intersects with the third border line and the fourth border line, so that new path points are continuously obtained, a defect detection path can be quickly generated based on at least part of the path points under the condition that the elements on the electronic panel are in various types, multiple sizes and random distribution, and in the process of performing defect detection on the electronic panel by using the generated defect detection path, the repeated detection or missing detection element is less, and is improved efficiency is improved.
In some embodiments, step 206 further comprises:
determining an initial position of a path point acquisition window in a target area based on a first boundary line and a second boundary line, and determining the initial position of the path point acquisition window as a current position of the path point acquisition window;
wherein a first window boundary line in a first direction of the path point acquisition window coincides with the first boundary line, and a second window boundary line in a second direction of the path point acquisition window coincides with the second boundary line.
Specifically, since the moving direction of the waypoint collection window is a direction moving toward the third boundary line, the computer device may determine the initial position of the waypoint collection window in the target area from the position of the first boundary line and the position of the second boundary line, regarding the initial position as the current position, the first window boundary line in the first direction of the waypoint collection window at the current position coincides with the first boundary line, and the second window boundary line in the second direction coincides with the second boundary line. For example, as shown in FIG. 4B, an initial position of the waypoint acquisition window in the target region is illustrated, the waypoint acquisition window in the initial position being shown as x min Is the left boundary, y max The size is width×height for the lower boundary.
In this embodiment, since the first window boundary line in the first direction of the path point acquisition window coincides with the first boundary line, and the second window boundary line in the second direction of the path point acquisition window coincides with the second boundary line, the initial position of the path acquisition window is determined, and the initial position is provided for continuously updating the current position of the path point acquisition region according to the first boundary line.
In some embodiments, updating the current position of the waypoint acquisition window based on each region border that the waypoint acquisition window includes under the current position comprises:
determining a first area frame with the minimum distance from a first window boundary line in a first direction of the path point acquisition window from all area frames included in the current position of the path point acquisition window;
determining a second area frame with the minimum distance from a second window boundary line in a second direction of the path point acquisition window from all area frames included in the path point acquisition window;
and updating the current position of the path point acquisition window based on the first area frame and the second area frame.
The first area frame and the second area frame are area frames included by the path point acquisition window in the current position, the distance between the first area frame and a first window boundary line of the path point acquisition window in the first direction is minimum, and the distance between the second area frame and a second window boundary line of the path point acquisition window in the second direction is minimum.
Specifically, after determining the first area frame and the second area frame from the area frames included in the path point acquisition window, the computer device updates the current position of the path point acquisition window based on the first area frame and the second area frame, after updating, the first window boundary line in the first direction of the path point acquisition window coincides with at least one point on the first area frame, and the second window boundary line in the second direction of the path point acquisition window coincides with at least one point on the second area frame.
In some embodiments, each area border is rectangular in shape, and the distance between the area border and the window boundary line of the path point collection window may be the distance between a certain border line of the area border and the window boundary line of the path point collection window. For each region frame included in the path point acquisition window, the computer device may determine a distance between a first frame line in a first direction of the region frame and a first window boundary line in the first direction of the path point acquisition window, and determine the region frame with the smallest distance as the first region frame; and determining the distance between a second frame line in the second direction of the region frame and a second window boundary line in the second direction of the path point acquisition window, and determining the region frame with the smallest distance as the second region frame. For example, as shown in fig. 4B, the first direction is a horizontal direction, the second direction is a vertical direction, the first window boundary line in the first direction is a lower boundary of the waypoint collection window, the second window boundary line in the second direction is a left boundary of the waypoint collection window, the first border line in the first direction is a lower boundary of the regional border, the second border line in the second direction is a left boundary of the regional border, after updating, the lower boundary of the waypoint collection window coincides with the lower boundary of the first regional border, and the left boundary of the waypoint collection window coincides with the left boundary of the second regional border.
In some embodiments, there is at least one region border that is non-rectangular in shape, such as a circle, other polygon, etc., and the distance between the region border and the window boundary line of the waypoint collection window may be the distance between the target point on the region border and the window boundary line of the waypoint collection window, where the target point refers to the point with the smallest distance between the points on the region border and the window boundary line of the waypoint collection window. The computer equipment can find a first target point with the minimum distance from a first window boundary line in the first direction of the path point acquisition window from all the area frames included in the path point acquisition window, and determine the area frame where the first target point is located as a first area frame; and finding a second target point with the smallest distance from the boundary line of the second window in the second direction of the path point acquisition window from all the region borders included in the path point acquisition window, and determining the region border where the second target point is located as the second region border. After updating, a first window boundary line in a first direction of the path point acquisition window coincides with the first target point, and a second window boundary line in a second direction of the path point acquisition window coincides with the second target point.
In this embodiment, a first area frame having a minimum distance from a first window boundary line in a first direction of a path point acquisition window and a second area frame having a minimum distance from a second window boundary line in a second direction of the path point acquisition window are determined from all area frames included in the path point acquisition window in a current position, and based on the first area frame and the second area frame, the current position of the path point acquisition window is updated, so that more area frames can be included in the path point acquisition window as much as possible, the number of image acquisition times in a defect detection process is reduced, and the defect detection efficiency is improved.
In some embodiments, after the step of updating the current position of the waypoint acquisition window based on the first region frame and the second region frame, the defect detection method further comprises:
determining each region frame included in the path point acquisition window at the current position, and generating a surrounding frame surrounding each region frame;
the current position of the waypoint collection window is updated based on the center position of the bounding box.
The bounding box may be a minimum circumscribed rectangular box of each area frame included by the path point acquisition window in the current position.
Specifically, after updating the current position of the path point acquisition window based on the first area frame and the second area frame, the computer device may determine each area frame included in the path point acquisition window at the current position, then generate a minimum circumscribed rectangular frame of each area frame to obtain a surrounding frame, use the central position of the surrounding frame as the next position of the path point acquisition window, and update the current position of the path point acquisition window.
In this embodiment, the current position of the path point acquisition window is updated based on the central position of the bounding box, so that each area frame included in the path point acquisition window can be centered, the fault tolerance of movement of the image acquisition device in the defect detection process is improved, the problem that the element is located at the edge of the image and cannot be detected is reduced, and therefore the defect detection efficiency is improved.
In some embodiments, updating the current position of the waypoint collection window toward a direction proximate to the third boundary based on the respective region borders included under the current position of the first boundary line and the waypoint collection window comprises:
under the condition that the path point acquisition window is not intersected with the third boundary line, determining a region frame point with the minimum distance from the third window boundary line in the second direction of the path point acquisition window from each region frame included in the path point acquisition window at the current position;
Updating the current position of the path point acquisition window based on the first boundary line and the regional frame point; after updating, a first window boundary line in a first direction of the path point acquisition window coincides with the first boundary line, and a third window boundary line in a second direction of the path point acquisition window is separated from the regional frame point by a preset distance.
The area frame points are determined from all area frames included in the path point acquisition window in the current position, and the distance between the area frame points and a third window boundary line in the second direction of the path point acquisition window is the smallest. The preset distance is preset, and may be, for example, a maximum value in the width of each region frame in the region frame set.
Specifically, before updating the current position of the path point collection window, the computer device first determines whether the path point collection window intersects with a third boundary line, and determines, from each region border included in the path point collection window at the current position, a point having a smallest distance from the third window boundary line in the second direction of the path point collection window, to obtain a region border point, in the case that the path point collection window does not intersect with the third boundary line. Then, the computer device may update the current position of the path point collection window by using the position of the first boundary line and the position of the region frame point, and after updating, the first window boundary line in the first direction of the path point collection window coincides with the first boundary line, and the second window boundary line in the second direction of the path point collection window is spaced from the region frame point by a preset distance. For example, as shown in fig. 4C, the region border point is located on the right boundary of the rightmost region border in the region borders included in the current position of the path point acquisition window, and a schematic diagram of the position of the path point acquisition window before and after updating is shown, the preset distance is Wm, and after updating, the left boundary line of the path point acquisition window is parallel to and distant from the vertical straight line where the region border point is located by Wm.
In this embodiment, when the path point collection window and the third boundary line are not intersected, the current position of the path point collection window is updated based on the direction of the first boundary line and the area frame point towards the direction close to the third boundary line, after updating, the first window boundary line in the first direction of the path point collection window coincides with the first boundary line, and the second window boundary line in the second direction of the path point collection window is spaced from the area frame point by a preset distance, and the preset distance can be the maximum value in the width of each area frame in the area frame set, so that the problem of missing detection elements in the defect detection process can be reduced, and the defect detection efficiency is improved.
In some embodiments, determining the waypoint based on the current location of the waypoint acquisition window comprises:
determining a path point based on the current position of the path point acquisition window, and marking each area frame included in the path point acquisition window at the current position;
the defect detection method further comprises the following steps:
under the condition that the path point acquisition window is intersected with a third boundary line, determining a third region frame with the smallest distance from the first boundary line from unlabeled region frames in the region frame set;
Updating the position of the reference line based on the third region frame;
after the updating, the step of determining the current position of the waypoint collection window in the target area based on the first boundary line and the second boundary line is returned.
The third region frame is determined from unlabeled region frames in the region frame set, and when the third region frame is rectangular, the distance between the third region frame and the first boundary line may be the distance from a point on a lower boundary line of the third region frame to the first boundary line; when the third region frame has another shape, the third region frame has a distance from the closest point to the first boundary line.
Specifically, after determining the waypoint, the computer device marks each region border included in the waypoint collection window at the current position, and when determining that the waypoint collection window at the current position intersects with the third border line, that is, when the waypoint collection window has reached the border of the target region, the computer device may determine, from among the unlabeled region borders in the region border set, the region border with the smallest distance to the first border line, obtain the third region border, and then update the position of the first border line by using the position of the third region border line, for example, when the third region border line is rectangular, the position of the first border line may be updated according to the lower border line of the third region border line, the updated first border line is parallel to the first border line before updating, and the distance between the updated first border line and the fourth border line is smaller than the distance between the first border line before updating and the fourth border line. After the updating, the computer device returns to the step of determining the current position of the waypoint collection window in the target area based on the first boundary line and the second boundary line. For example, as shown in fig. 4D, a schematic diagram is shown in which the current position of the route point collection window is updated with the updated first boundary line and the second boundary line, and the current global lower boundary is the updated first boundary line.
In this embodiment, when the path point collection window intersects with the third boundary line, the position of the first boundary line is updated towards the direction close to the fourth boundary line, so that the updated first boundary line is used to determine the current position of the path point collection window in the next row, automatic line feed of path planning is achieved, and the efficiency of path planning is improved.
In some embodiments, determining the waypoint based on the current location of the waypoint acquisition window comprises:
determining a path point based on the current position of the path point acquisition window, recording the path point, and marking the frames of each region included in the path point acquisition window at the current position;
after the step of updating the current position of the waypoint collection window toward a direction approaching the third boundary line based on the respective region borders included in the current position of the first boundary line and the waypoint collection window, the defect detection method further includes:
based on the first boundary line and each region frame included in the path point acquisition window in the current position, updating the current position of the path point acquisition window towards the direction close to the third boundary line, and calculating the distance between the first window boundary lines in the first direction of the path point acquisition window before and after updating;
Under the condition that the distance between the first window boundary lines in the first direction of the before-update and after-update path point acquisition windows is larger than a distance threshold, canceling marks of all area frames included in the before-update path point acquisition windows, and deleting the recorded path points determined according to the current position before update;
generating a defect detection path based on at least a portion of the path points, comprising:
and generating a defect detection path based on each reserved path point.
The distance threshold is set according to parameters of the image acquisition device, and when the distance between two adjacent shooting points in the preset direction is larger than the distance threshold, the image acquisition device cannot accurately move from one shooting point to the next shooting point, so that the quality of a shot image is lower.
Specifically, if the second shooting mode is adopted to perform image acquisition in the defect detection process, after updating the current position of the path point acquisition window towards the direction close to the third boundary line based on each region frame included in the current position of the first boundary line and the path point acquisition window each time, the computer equipment can calculate the distance between the first window boundary lines in the first direction of the path point acquisition window before and after updating to obtain a vertical distance, and cancel the mark of each region frame included in the path point acquisition window before updating and delete the recorded path point determined according to the current position before updating if the vertical distance is greater than the distance threshold, as shown in fig. 4E, a schematic diagram with the vertical distance greater than the distance threshold is displayed, and then the step of updating the current position of the path point acquisition window based on each region frame included in the path point acquisition window in the current position is returned; and under the condition that the vertical distance is smaller than or equal to the distance threshold value, directly returning to the step of updating the current position of the path point acquisition window based on each region frame included by the path point acquisition window under the current position until the path point acquisition window is intersected with the third boundary line and the fourth boundary line. The computer device may then generate a defect detection path based on the retained individual waypoints.
In this embodiment, under the condition that the distance between the boundary lines of the first window in the first direction of the route point collection window before and after updating is greater than the distance threshold, the marks of the borders of each area included in the route point collection window before updating are cancelled, the recorded route points determined according to the current position before updating are deleted, and a defect detection route is generated based on each reserved route point, so that the vertical distance between any two adjacent route points in the defect detection route is smaller than the distance threshold, and the hardware requirement of image collection by adopting the second shooting mode in the defect detection process can be met.
In some embodiments, as shown in fig. 5, a defect detection method is provided, and the method is applied to the computer device 102 for illustration, and includes the following steps:
step 502, obtaining a region frame set of each element region in an image of the electronic panel.
Step 504, generating a first boundary line and a fourth boundary line in a first direction, and generating a second boundary line and a third boundary line in a second direction; each region border in the region border set is located in a target region surrounded by the first border line, the second border line, the third border line and the fourth border line.
The shape of the target area is rectangular, and the shape of the area border is rectangular. Step 506, determining the current position of the waypoint collection window in the target area based on the first boundary line and the second boundary line.
The shape of the path point acquisition window is rectangular.
Step 508, determining a first area frame with the minimum distance from the lower boundary line of the path point acquisition window and a second area frame with the minimum distance from the left boundary line of the path point acquisition window from all area frames included in the path point acquisition window at the current position, and updating the current position of the path point acquisition window based on the first area frame and the second area frame.
After updating, the lower boundary line of the path point acquisition window is overlapped with the lower boundary of the first area frame, and the left boundary line of the path point acquisition window is overlapped with the left boundary of the second area frame.
Step 510, determining each region frame included in the path point acquisition window in the current position, generating a bounding box bounding each region frame, and updating the current position of the path point acquisition window based on the central position of the bounding box.
And step 512, after updating, determining a path point based on the current position of the path point acquisition window, and marking each area frame included in the path point acquisition window in the current position.
Step 514, determining a region border point with the minimum distance from the right border line of the path point acquisition window from the region borders included in the current position of the path point acquisition window, and updating the current position of the path point acquisition window based on the first border line and the third region border.
After updating, the lower boundary line of the path point acquisition window coincides with the first boundary line, and the right boundary line of the path point acquisition window is separated from the regional frame point by a preset distance.
Step 516, judging whether the path point acquisition window intersects with the third boundary line, if not, returning to step 508; if yes, go to step 518.
Step 518, judging whether the path point acquisition window intersects with the fourth boundary line, if not, executing step 520; if yes, go to step 522.
Step 520, determining a third region frame with the smallest distance from the first boundary line from the unlabeled region frames in the region frame set, updating the position of the lower boundary of the third region frame to the position of the first boundary line, and returning to step 506.
Step 522, generating a defect detection path based on each path point; when the electronic panel is detected, the defect detection is carried out according to the defect detection path.
In this embodiment, after determining the current position of the path point acquisition window in the target area based on the left boundary line of the reference line and the target area, the path point acquisition window can include a larger number of area frames by updating the current position of the path point acquisition window based on the first area frame and the second area frame and updating the current position of the path point acquisition window based on the center position of the surrounding frame twice, so that the number of image acquisition times in the defect detection process is reduced, the coverage rate of elements of the defect detection path is improved by updating the current position of the path point acquisition window based on the reference line and the third area frame, and the problem of missing elements in the defect detection process is reduced, thereby improving the defect detection efficiency.
It should be understood that, although the steps in the flowcharts related to the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a defect detection device for realizing the defect detection method. The implementation of the solution provided by the device is similar to that described in the above method, so the specific limitation of one or more embodiments of the defect detection device provided below may be referred to above for limitation of the defect detection method, and will not be repeated here.
In some embodiments, as shown in fig. 6, there is provided a defect detection apparatus including:
the region frame acquisition module 602 is configured to acquire a region frame of each element region in an image of the electronic panel to obtain a region frame set;
a boundary line generation module 604 for generating a first boundary line and a fourth boundary line in a first direction and generating a second boundary line and a third boundary line in a second direction; each region border in the region border set is positioned in a target region surrounded by the first border line, the second border line, the third border line and the fourth border line;
a first location updating module 606, configured to determine a current location of the path point collection window in the target area based on the first boundary line and the second boundary line, and update the current location of the path point collection window based on each area frame included in the path point collection window in the current location;
A path point determining module 608, configured to determine a path point based on the current position of the path point collection window after updating;
a second location updating module 610, configured to update the current location of the waypoint collection window towards a direction close to the third boundary based on each region border included in the current location of the first boundary and the waypoint collection window, and update the location of the first boundary towards a direction close to the fourth boundary if the waypoint collection window intersects the third boundary, and return to the step of determining the current location of the waypoint collection window in the target region based on the first boundary and the second boundary until the waypoint collection window intersects both the third boundary and the fourth boundary;
a path generation module 612 for generating a defect detection path based on at least a portion of the path points; when the electronic panel is detected, the defect detection is carried out according to the defect detection path.
In some embodiments, the first location update module 606 is specifically configured to, in determining a current location of the waypoint acquisition window in the target area based on the first boundary line and the second boundary line:
determining an initial position of the path point acquisition window in the target area based on the first boundary line and the second boundary line, and determining the initial position of the path point acquisition window as a current position of the path point acquisition window;
Wherein a first window boundary line in a first direction of the path point acquisition window coincides with the first boundary line, and a second window boundary line in a second direction of the path point acquisition window coincides with the second boundary line.
In some embodiments, in updating the current position of the waypoint acquisition window based on each region border included in the waypoint acquisition window under the current position, the first position updating module 606 is specifically configured to:
determining a first area frame with the minimum distance from a first window boundary line in a first direction of the path point acquisition window from all area frames included in the current position of the path point acquisition window;
determining a second area frame with the minimum distance from a second window boundary line in a second direction of the path point acquisition window from all area frames included in the path point acquisition window;
and updating the current position of the path point acquisition window based on the first area frame and the second area frame.
In some embodiments, after the step of updating the current position of the waypoint acquisition window based on the first region border and the second region border, the first position updating module 606 is specifically configured to:
determining each region frame included in the path point acquisition window at the current position, and generating a surrounding frame surrounding each region frame;
The current position of the waypoint collection window is updated based on the center position of the bounding box.
In some embodiments, based on the respective region borders included under the current position of the first boundary line and the waypoint collection window, the second location updating module 610 is specifically configured to:
under the condition that the path point acquisition window is not intersected with the third boundary line, determining a region frame point with the minimum distance from the third window boundary line in the second direction of the path point acquisition window from each region frame included in the path point acquisition window at the current position;
updating the current position of the path point acquisition window towards the direction close to the third boundary line based on the first boundary line and the regional boundary line; after updating, a first window boundary line in a first direction of the path point acquisition window coincides with the first boundary line, and a second window boundary line in a second direction of the path point acquisition window is separated from the region frame point by a preset distance.
In some embodiments, the waypoint determination module 608 is specifically configured to, in determining the waypoint based on the current location of the waypoint acquisition window:
Determining a path point based on the current position of the path point acquisition window, and marking each area frame included in the path point acquisition window at the current position;
the defect detection device further comprises a boundary line updating module, wherein the boundary line updating module is specifically used for:
under the condition that the path point acquisition window is intersected with a third boundary line, determining a third region frame with the smallest distance from the first boundary line from unlabeled region frames in the region frame set;
updating the position of the first boundary line based on the third region frame;
after the updating, the step of determining the current position of the waypoint collection window in the target area based on the first boundary line and the second boundary line is returned.
In some embodiments, the waypoint validation module 606 is specifically configured to, in determining a waypoint based on the current location of the waypoint acquisition window:
determining a path point based on the current position of the path point acquisition window, recording the path point, and marking the frames of each region included in the path point acquisition window at the current position;
after the step of updating the current position of the waypoint collection window toward a direction approaching the third boundary based on the respective region borders included in the current position of the first boundary and the waypoint collection window, the position updating module 608 is specifically configured to:
Based on the first boundary line and each region frame included in the path point acquisition window in the current position, updating the current position of the path point acquisition window towards the direction close to the third boundary line, and calculating the distance between the first window boundary lines in the first direction of the path point acquisition window before and after updating;
under the condition that the distance between the first window boundary lines in the first direction of the before-update and after-update path point acquisition windows is larger than a distance threshold, canceling marks of all area frames included in the before-update path point acquisition windows, and deleting the recorded path points determined according to the current position before update;
in terms of generating a defect detection path based on at least some of the path points, path generation module 612 is specifically configured to:
and generating a defect detection path based on each reserved path point.
The respective modules in the above defect detection apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In some embodiments, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing relevant data related to the defect detection method. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement the steps in the defect detection method described above.
In some embodiments, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 8. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement the steps in the defect detection method described above. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by persons skilled in the art that the structures shown in fig. 7 and 8 are block diagrams of only portions of structures associated with the present inventive arrangements and are not limiting of the computer device to which the present inventive arrangements are applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In some embodiments, a computer device is provided, the computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps in the defect detection method described above when the computer program is executed.
In some embodiments, a computer readable storage medium 900 is provided, on which a computer program 902 is stored, where the computer program 902, when executed by a processor, implements the steps in the image data processing method described above, and the internal structure diagram may be as shown in fig. 9.
In some embodiments, a computer program product is provided, the computer program comprising a computer program which, when executed by a processor, implements the steps of the above-described defect detection method.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as Static Random access memory (Static Random access memory AccessMemory, SRAM) or dynamic Random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims (10)

1. A defect detection method, comprising:
acquiring the region borders of each element region in the image of the electronic panel to obtain a region border set;
generating a first boundary line and a fourth boundary line in a first direction, and generating a second boundary line and a third boundary line in a second direction; each region frame in the region frame set is positioned in a target region surrounded by the first boundary line, the second boundary line, the third boundary line and the fourth boundary line;
Determining the current position of a path point acquisition window in the target area based on a first boundary line and the second boundary line, and updating the current position of the path point acquisition window based on each area frame included by the path point acquisition window in the current position;
after updating, determining a path point based on the current position of the path point acquisition window;
updating the current position of the waypoint acquisition window towards the direction close to the third boundary line based on the first boundary line and each region frame included in the waypoint acquisition window at the current position, and updating the position of the first boundary line towards the direction close to the fourth boundary line in the case that the waypoint acquisition window intersects with the third boundary line, returning to the step of determining the current position of the waypoint acquisition window in the target region based on the first boundary line and the second boundary line until the waypoint acquisition window intersects with both the third boundary line and the fourth boundary line;
generating a defect detection path based on at least a portion of the waypoints; and when the electronic panel is subjected to defect detection, performing defect detection according to the defect detection path.
2. The method of claim 1, wherein the determining the current position of a waypoint collection window in the target area based on the first boundary line and the second boundary line comprises:
determining an initial position of a path point acquisition window in the target area based on the first boundary line and the second boundary line, and determining the initial position of the path point acquisition window as a current position of the path point acquisition window;
wherein a first window boundary line in a first direction of the waypoint collection window coincides with a first boundary line, and a second window boundary line in a second direction of the waypoint collection window coincides with the second boundary line.
3. The method of claim 2, wherein updating the current position of the waypoint collection window based on the region borders included in the waypoint collection window at the current position comprises:
determining a first area frame with the minimum distance from a first window boundary line in a first direction of the path point acquisition window from all area frames included in the current position of the path point acquisition window;
determining a second area frame with the minimum distance from a second window boundary line in a second direction of the path point acquisition window from all area frames included in the path point acquisition window;
And updating the current position of the path point acquisition window based on the first area frame and the second area frame.
4. The method of claim 3, wherein after the step of updating the current position of the waypoint collection window based on the first region border and the second region border, the method further comprises:
determining each region frame included in the path point acquisition window at the current position, and generating a surrounding frame surrounding each region frame;
and updating the current position of the path point acquisition window based on the central position of the bounding box.
5. The method of claim 1, wherein updating the current position of the waypoint collection window toward a direction proximate to the third boundary based on the first boundary and the respective region borders included in the waypoint collection window at the current position comprises:
determining a region frame point with the minimum distance from a third window boundary line in a second direction of the path point acquisition window from each region frame included in the path point acquisition window in the current position under the condition that the path point acquisition window is not intersected with the third boundary line;
Updating the current position of the path point acquisition window based on the first boundary line and the region frame point towards the direction approaching the third boundary line; after updating, a first window boundary line in a first direction of the path point acquisition window coincides with the first boundary line, and a third window boundary line in a second direction of the path point acquisition window is separated from the region frame point by a preset distance.
6. The method of claim 5, wherein the determining a waypoint based on the current location of the waypoint acquisition window comprises:
determining a path point based on the current position of the path point acquisition window, and marking each area frame included in the path point acquisition window at the current position;
the updating the position of the first boundary line toward the direction approaching the fourth boundary line in the case where the waypoint collection window intersects the third boundary line includes:
determining a third area frame with the smallest distance from the first boundary line from unlabeled area frames in the area frame set under the condition that the path point acquisition window is intersected with the third boundary line;
updating the position of the first boundary line in a direction approaching to the fourth boundary line based on the third region frame;
After the updating, the step of determining the current position of the route point collection window in the target area based on the first boundary line and the second boundary line is returned.
7. The method of claim 1, wherein the determining a waypoint based on the current location of the waypoint acquisition window comprises:
determining a path point based on the current position of the path point acquisition window, recording the path point, and marking each area frame included in the path point acquisition window at the current position;
after the step of updating the current position of the waypoint collection window toward a direction approaching the third boundary based on the first boundary line and each region border included in the waypoint collection window at the current position, the method further comprises:
updating the current position of the path point acquisition window towards the direction close to the third boundary line based on the first boundary line and each region frame included by the path point acquisition window in the current position, and calculating the distance between the first window boundary lines in the first direction of the path point acquisition window before and after updating;
under the condition that the distance between the first window boundary lines in the first direction of the path point acquisition window before and after updating is larger than a distance threshold value, canceling marks of all area frames included in the path point acquisition window before updating, and deleting the recorded path points determined according to the current position before updating;
The generating a defect detection path based on at least a portion of the path points includes:
and generating a defect detection path based on each reserved path point.
8. A defect detection apparatus, comprising:
the region frame acquisition module is used for acquiring the region frames of the element regions in the image of the electronic panel to obtain a region frame set;
a boundary line generation module for generating a first boundary line and a fourth boundary line in a first direction and generating a second boundary line and a third boundary line in a second direction; each region frame in the region frame set is positioned in a target region surrounded by the first boundary line, the second boundary line, the third boundary line and the fourth boundary line;
the first position updating module is used for determining the current position of the path point acquisition window in the target area based on a first boundary line and the second boundary line, and updating the current position of the path point acquisition window based on each area frame included by the path point acquisition window in the current position;
the path point determining module is used for determining a path point based on the current position of the path point acquisition window after updating;
a second position updating module, configured to update a current position of the waypoint collection window toward a direction approaching the third boundary line based on each region border included in the current position of the first boundary line and the waypoint collection window, and update a position of the first boundary line toward a direction approaching the fourth boundary line in a case where the waypoint collection window intersects the third boundary line, and return to the step of determining the current position of the waypoint collection window in the target region based on the first boundary line and the second boundary line until the waypoint collection window intersects both the third boundary line and the fourth boundary line;
A path generation module for generating a defect detection path based on at least part of the path points; and when the electronic panel is subjected to defect detection, performing defect detection according to the defect detection path.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
CN202310988253.3A 2023-08-08 2023-08-08 Defect detection method, defect detection device, computer equipment and computer readable storage medium Active CN116703926B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310988253.3A CN116703926B (en) 2023-08-08 2023-08-08 Defect detection method, defect detection device, computer equipment and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310988253.3A CN116703926B (en) 2023-08-08 2023-08-08 Defect detection method, defect detection device, computer equipment and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN116703926A CN116703926A (en) 2023-09-05
CN116703926B true CN116703926B (en) 2023-11-03

Family

ID=87839713

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310988253.3A Active CN116703926B (en) 2023-08-08 2023-08-08 Defect detection method, defect detection device, computer equipment and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN116703926B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102156136A (en) * 2011-03-14 2011-08-17 浙江展邦电子科技有限公司 Method for detecting PCB negative film
CN111784685A (en) * 2020-07-17 2020-10-16 国网湖南省电力有限公司 Power transmission line defect image identification method based on cloud edge cooperative detection
CN114429445A (en) * 2021-11-09 2022-05-03 南京师范大学 PCB defect detection and identification method based on MAIRNet

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5597096B2 (en) * 2010-10-18 2014-10-01 キヤノン株式会社 Image processing apparatus, image processing method, and program

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102156136A (en) * 2011-03-14 2011-08-17 浙江展邦电子科技有限公司 Method for detecting PCB negative film
CN111784685A (en) * 2020-07-17 2020-10-16 国网湖南省电力有限公司 Power transmission line defect image identification method based on cloud edge cooperative detection
CN114429445A (en) * 2021-11-09 2022-05-03 南京师范大学 PCB defect detection and identification method based on MAIRNet

Also Published As

Publication number Publication date
CN116703926A (en) 2023-09-05

Similar Documents

Publication Publication Date Title
US20200388080A1 (en) Displaying content in an augmented reality system
EP2745237B1 (en) Dynamic selection of surfaces in real world for projection of information thereon
US20190122435A1 (en) Generating time-delayed augmented reality content
CN110096929A (en) Target detection neural network based
CN110648363A (en) Camera posture determining method and device, storage medium and electronic equipment
CN109977952A (en) Candidate target detection method based on local maximum
CN116703926B (en) Defect detection method, defect detection device, computer equipment and computer readable storage medium
CN113608805B (en) Mask prediction method, image processing method, display method and device
CN115880362B (en) Code region positioning method, device, computer equipment and computer readable storage medium
CN115965646B (en) Region dividing method, device, computer equipment and computer readable storage medium
CN116309824A (en) Ground water track identification method, device, computer equipment and storage medium
JP2009110398A (en) Analysis model generation system
CN113505139B (en) Remote sensing image automatic updating and history backtracking method and device based on single service
CN114663449A (en) Road defect area calculation method and device, computer equipment and storage medium
CN113140292A (en) Image abnormal area browsing method and device, mobile terminal equipment and storage medium
CN116645374B (en) Point defect detection method, point defect detection device, computer equipment and storage medium
CN112767472B (en) Method for positioning lamp beads in display screen image, computing equipment and storage medium
CN116442226A (en) Pose correctness judging method, pose correctness judging device, robot and storage medium
CN116698007A (en) Fusion map construction method, device, robot and storage medium
CN115797615A (en) Method, device and equipment for determining detection area of target object and storage medium
US10140410B1 (en) Representing a routing strip in an integrated circuit design using a digit pattern
CN116612474A (en) Object detection method, device, computer equipment and computer readable storage medium
CN115357647A (en) Report display method and device, computer equipment and storage medium
CN115578254A (en) Screenshot generation method and device, computer equipment and storage medium
CN113838165A (en) Automatic pattern spot numbering method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant