CN114913121A - Screen defect detection system and method, electronic device and readable storage medium - Google Patents

Screen defect detection system and method, electronic device and readable storage medium Download PDF

Info

Publication number
CN114913121A
CN114913121A CN202210344598.0A CN202210344598A CN114913121A CN 114913121 A CN114913121 A CN 114913121A CN 202210344598 A CN202210344598 A CN 202210344598A CN 114913121 A CN114913121 A CN 114913121A
Authority
CN
China
Prior art keywords
image
screen
image acquisition
acquisition device
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210344598.0A
Other languages
Chinese (zh)
Inventor
王建勋
刘伟
杨达坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lenovo Beijing Ltd
Original Assignee
Lenovo Beijing Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lenovo Beijing Ltd filed Critical Lenovo Beijing Ltd
Priority to CN202210344598.0A priority Critical patent/CN114913121A/en
Publication of CN114913121A publication Critical patent/CN114913121A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30121—CRT, LCD or plasma display

Abstract

The application discloses a screen defect detection system, a method, an electronic device and a readable storage medium, wherein the system comprises: setting a first image acquisition device for acquiring a first image of a screen at a target shooting angle; the second image acquisition devices are used for acquiring a plurality of second images of the screen at other different shooting angles; the screen defect detection device is used for acquiring a first image and a plurality of second images; taking the first image as a reference, and registering each second image with the first image; fusing the registered second images with the first image to obtain a multi-channel image; determining screen defects according to the multi-channel image; the first image acquisition device and the second image acquisition device are different types of image acquisition devices, and the focusing range of the second image acquisition device is larger than that of the first image acquisition device. By implementing the method and the device, all defects of the screen can be detected, and the accuracy of detecting the defects of the screen is ensured.

Description

Screen defect detection system and method, electronic device and readable storage medium
Technical Field
The present disclosure relates to the field of display technologies, and in particular, to a system and a method for detecting a screen defect, an electronic device and a readable storage medium.
Background
When the screen leaves the factory, appearance detection is required. At present, defects in the appearance of a screen are generally detected by an Automated Optical Inspection (AOI) method. And when the screen AOI is used, an image is shot on the front side of the screen, and then the image is analyzed to determine the defect of the appearance of the screen. However, some defects of the screen appearance cannot be photographed under the conventional front photographing, thereby causing omission of some defects.
Disclosure of Invention
In view of the above, embodiments of the present application provide a screen defect detecting system, a method, an electronic device and a readable storage medium.
According to a first aspect of the present application, an embodiment of the present application provides a screen defect detecting system, including: the first image acquisition device is used for acquiring a first image of the screen at a target shooting angle; the second image acquisition devices are used for acquiring a plurality of second images of the screen at other different shooting angles; the screen defect detection device is used for acquiring a first image and a plurality of second images; taking the first image as a reference, and registering each second image with the first image; fusing the registered second images with the first image to obtain a multi-channel image; determining screen defects according to the multi-channel image; the first image acquisition device and the second image acquisition device are different types of image acquisition devices, and the focusing range of the second image acquisition device is larger than that of the first image acquisition device.
Optionally, the second image capturing device is a tilt camera, and the tilt camera includes a schemer ring, and the focusing range of the second image capturing device is adjusted by adjusting an angle of the schemer ring.
Optionally, before the screen defect detecting device registers each second image with the first image, the screen defect detecting device is further configured to: acquiring the definition of each second image and corresponding camera parameters; processing each second image, the definition of the second image and the corresponding camera parameter based on the focusing model to obtain a target adjustment parameter aiming at the camera parameter; adjusting a second image acquisition device corresponding to the second image according to the target adjustment parameter; adopting the adjusted second image acquisition device to acquire the screen image again to obtain a second image acquired again; the reacquired second image satisfies the sharpness condition.
Optionally, the step of constructing the focusing model includes: acquiring a training sample, wherein the training sample comprises a plurality of sample images of a sample screen, and each sample image is marked with an initial camera parameter, a target sample adjustment parameter and the definition of the sample image; the target sample adjustment parameter is the difference value between the calibration camera parameter of the calibration image meeting the definition condition and the initial camera parameter of the sample image; taking a target image of a target shooting angle of a sample screen as a reference, and registering each sample image with the target image; for each registered sample image: inputting the registered sample image into a neural network to obtain corresponding initial sample adjustment parameters; inputting the initial sample adjustment parameters, the sample image and the calibration image into a generative confrontation network to obtain a corresponding pixel difference value; and inputting the pixel difference value and the sample image into a neural network to obtain corresponding secondary sample adjustment parameters so as to train a focusing model until convergence.
Optionally, the screen displays the calibration pattern, and before the screen defect detecting device acquires the plurality of second images, the screen defect detecting device is further configured to: for each second image acquisition device: a second image acquisition device is adopted to acquire a plurality of third images of the screen, and the camera positions and/or shooting angles of any two third images are different; determining a target third image meeting a first condition according to the calibration patterns in the plurality of third images; and adjusting the position and the shooting angle of the second image acquisition device according to the camera position and the shooting angle of the third image of the target.
Optionally, the screen defect detecting device is further configured to, after adjusting the position and the shooting angle of the second image capturing device according to the camera position and the shooting angle of the third image of the target: for each second image acquisition device: acquiring a fourth image and a fifth image of the screen by adopting a second image acquisition device, wherein the fourth image corresponds to the first camera parameter, and the fifth image corresponds to the second camera parameter; determining the definition of the fourth image and the definition of the fifth image respectively; and adjusting the second camera parameter according to the definition of the fourth image and the definition of the fifth image, so that the second image acquisition device acquires the second image according to the adjusted second camera parameter.
Optionally, adjusting the second camera parameter according to the respective corresponding definitions of the fourth image and the fifth image includes: under the condition that the difference value between the definition corresponding to the fifth image and the definition corresponding to the fourth image is larger than or equal to the threshold value, continuously adjusting the second camera parameter according to the direction of adjusting the first camera parameter into the second camera parameter until the preset condition is met; and under the condition that the difference value between the definition corresponding to the fifth image and the definition corresponding to the fourth image is smaller than the threshold value, continuously adjusting the second camera parameter according to the direction of the first camera parameter adjusted by the second camera parameter until the preset condition is met.
According to a second aspect of the present application, an embodiment of the present application provides a screen defect detection method, including: acquiring a plurality of images of a screen, wherein the plurality of images comprise a first image acquired by a first image acquisition device at a target shooting angle and a plurality of second images acquired by a plurality of second image acquisition devices at other different shooting angles; the first image acquisition device and the second image acquisition device are different types of image acquisition devices, and the focusing range of the second image acquisition device is larger than that of the first image acquisition device; taking the first image as a reference, and registering each second image with the first image; fusing the registered second images with the first image to obtain a multi-channel image; and determining the screen defects according to the multi-channel images.
According to a third aspect of the present application, an embodiment of the present application provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method of screen defect detection as in the second aspect or any embodiment of the second aspect.
According to a fourth aspect of the present application, an embodiment of the present application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the screen defect detecting method according to the second aspect or any implementation manner of the second aspect.
The system, the method, the electronic device and the readable storage medium for detecting the screen defects are characterized in that a first image acquisition device is arranged and used for acquiring a first image of a screen at a target shooting angle; the second image acquisition devices are used for acquiring a plurality of second images of the screen at other different shooting angles; the screen defect detection device is used for acquiring a first image and a plurality of second images; taking the first image as a reference, and registering each second image with the first image; fusing the registered second images with the first image to obtain a multi-channel image; determining screen defects according to the multi-channel image; the first image acquisition device and the second image acquisition device are different types of image acquisition devices, and the focusing range of the second image acquisition device is larger than that of the first image acquisition device; thereby when detecting the defect of screen, not only can adopt first image acquisition device to gather the positive image of screen, can also adopt the second image acquisition that the scope of focusing still will be big than first image acquisition device, gather the multi-angle side of screen and shoot the image, guarantee to shoot the image with the screen side in wider range clearly, and fuse multi-angle side and the positive image of shooting, obtain the all clear multichannel image of each angle, then carry out defect detection based on the multichannel image again, just can detect all defects of screen, guarantee the degree of accuracy that the screen defect detected.
The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.
Drawings
FIG. 1 is a schematic view of a focusing range of a tilt-shift camera according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of a screen defect detection system according to an embodiment of the present application;
FIG. 3 is a top view of a first image capture device and a plurality of second image capture devices in an embodiment of the present application;
FIG. 4 is a schematic diagram of a first image in an embodiment of the present application;
FIG. 5 is a schematic diagram of a second image in an embodiment of the present application;
FIG. 6 is a schematic flowchart illustrating a process of determining target adjustment parameters of camera parameters according to an embodiment of the present application;
FIG. 7 is a schematic diagram of a training process of a focusing model in an embodiment of the present application;
FIG. 8 is a schematic flowchart illustrating a method for detecting screen defects according to an embodiment of the present application;
fig. 9 is a schematic diagram of a hardware structure of an electronic device in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The appearance detection of the screen and the shooting are always the most important ring. Some defects can not be shot under the traditional front shooting condition due to the characteristics of the defects and can only be shot from a specific angle. Thus, for such a case, a screen needs to be photographed from a side at a plurality of angles. In the conventional industrial camera, since the lens plane is parallel to the image plane and the object plane is also parallel to the lens plane, the front and rear limits of the depth of field range are naturally parallel to the lens plane, so that the focusing area is limited and the entire imaging area of the screen cannot be covered. The new tilt-shift camera can realize the intersection and the same position of a lens plane, an image plane and a shot object plane by adjusting the Schlemm ring, and as shown in figure 1, the focusing range is a wedge-shaped space, so that a larger focusing range (depth of field) can be realized, and the picture of the side surface of the screen can be shot clearly in a larger range.
Based on this, the present application provides a screen defect detecting system, as shown in fig. 2, including:
a first image capturing device 11 for capturing a first image of the screen 12 at a target shooting angle;
a plurality of second image capturing devices 13 for capturing a plurality of second images of the screen 12 at different other capturing angles;
a screen defect detecting device 14 for acquiring a first image and a plurality of second images; taking the first image as a reference, and registering each second image with the first image; fusing the registered second images with the first image to obtain a multi-channel image; determining the defects of the screen 12 according to the multi-channel image; the first image acquisition device and the second image acquisition device are different types of image acquisition devices, and the focusing range of the second image acquisition device 13 is larger than that of the first image acquisition device 11.
In the embodiment of the present application, the screen 12 is a display area of an electronic device such as a computer, a mobile phone, a tablet, a display, and the like.
In some embodiments, as shown in fig. 3, the first image capturing device 11 is disposed directly above the screen 12, the first image capturing device 11 may be a conventional industrial camera, and the target shooting angle is a front shooting angle, i.e., an angle directly facing the screen 12. The first image has at least a clear image of the middle position of the screen 12.
In some embodiments, as shown in fig. 3, the second image capturing device 13 is disposed at the side of the screen 12, for example, in different orientations, such as front, back, left, right, and so on. The other different photographing angles are different side photographing angles from the front photographing angle.
The focusing range of the second image capturing device 13 is larger than that of the first image capturing device 11, so that when the second image capturing device 13 is disposed at the side of the screen 12, the focusing area can cover the whole imaging area of the screen 12, and when the second image capturing device 13 captures the side-shot image of the screen 12, that is, when the second image capturing device 13 captures the second image, it can be ensured that the second image has the complete screen 12. The second image has at least a sharp image of the edge position of the screen 12.
Screen defect detection device 14 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The screen defect detection device 14 may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices.
The screen defect detecting device 14 is connected to the first image capturing device 11 and the second image capturing device 13 by wire or wirelessly, which is not limited in this embodiment.
After receiving the first image sent by the first image capturing device 11 and the plurality of second images sent by the second image capturing device 13, the screen defect detecting device 14 may register the first image and the plurality of second images for subsequent fusion of the first image and the plurality of second images, because the first image and the plurality of second images are taken from different angles of the screen 12.
In the embodiment of the present application, when the first image and the plurality of second images are registered, since the first image is an image photographed on the front surface of the screen 12 as shown in fig. 4, the shape of the first image is the same as or most similar to the shape of the screen 12. And the plurality of second images are images photographed from respective sides of the screen 12 as shown in fig. 5, so that the shape of the second images may deviate from the shape of the screen 12. Therefore, in the embodiment of the present application, each second image is registered with the first image with reference to the first image.
In order to better achieve the registration of the first image and each second image, in the embodiment of the present application, a calibration pattern may be displayed in the screen 12, as shown in fig. 4 or 5, opposite corners of the calibration pattern are recognizable patterns such as icons or characters, and a frame of the calibration pattern is a straight line.
And fusing the registered second images with the first image to obtain a multi-channel image. Since the first image has at least a clear image of the middle position of the screen 12 and the second image has at least a clear image of the edge position of the screen 12, each position of the screen 12 in the multi-channel image is clear. Based on the multi-channel image, all defects in the screen 12 can be detected.
The screen defect detection system provided by the embodiment of the application is provided with the first image acquisition device and is used for acquiring the first image of the screen at the target shooting angle; the second image acquisition devices are used for acquiring a plurality of second images of the screen at other different shooting angles; the screen defect detection device is used for acquiring a first image and a plurality of second images; taking the first image as a reference, and registering each second image with the first image; fusing the registered second images with the first image to obtain a multi-channel image; determining screen defects according to the multi-channel image; the first image acquisition device and the second image acquisition device are different types of image acquisition devices, and the focusing range of the second image acquisition device is larger than that of the first image acquisition device; thereby when detecting the defect of screen, not only can adopt first image acquisition device to gather the positive image of screen, can also adopt the second image acquisition that the scope of focusing still will be big than first image acquisition device, gather the multi-angle side of screen and shoot the image, guarantee to shoot the image with the screen side in wider range clearly, and fuse multi-angle side and the positive image of shooting, obtain the all clear multichannel image of each angle, then carry out defect detection based on the multichannel image again, just can detect all defects of screen, guarantee the degree of accuracy that the screen defect detected.
In an alternative embodiment, the second image capturing device 13 is a tilt-shift camera, which includes a schemer ring, and the angle of the schemer ring is adjusted to adjust the focusing range of the second image capturing device 13.
In the embodiment of the present application, the tilt-shift camera has a larger focusing range (depth of field) than a common industrial camera, so that it can be ensured that the picture of the side of the screen 12 can be clearly taken in a larger range, and therefore, the second image capturing device 13 can be directly selected as the tilt-shift camera. The axis-shifting camera is provided with the Samsung ring, and the angle of the Samsung ring is adjusted, so that when the second image of the screen 12 is acquired by the second image acquisition device 13, the imaging area of the screen 12 can be covered, and the clear edge image of the screen 12 can be obtained.
In an alternative embodiment, when the object to be detected of the system for detecting defects on the screen 12 changes, for example, a flat panel is changed into a notebook, the camera position, the shooting angle and the camera parameters of the second image capturing device 13 need to be adjusted in time. The focusing of a common industrial camera only needs to adjust the position and the focal length of the camera, but a tilt-shift camera needs to adjust the position, the heading (yaw) angle, the roll angle, the pitch angle, the angle of a schemer ring, the focal length of the camera and other parameters. Therefore, the adjustment of the tilt-shift camera is very complicated and difficult, different focusing ranges correspond to different adjustment schemes, so that not only is the experience of engineers rich, but also the required focusing effect is difficult to be quickly adjusted, and meanwhile, the focusing effect is different from person to person.
Therefore, in the embodiment of the present application, a plurality of images of the screen 12 may be acquired by the second image acquisition device 13, and the camera positions and/or the shooting angles of any two images are different; determining a target image meeting a first condition according to calibration patterns in the plurality of images; and adjusting the position and the shooting angle of the second image acquisition device 13 according to the camera position and the shooting angle of the target image, so that the second image acquisition device 13 acquires the second image at the adjusted camera position and shooting angle. The shooting angles include a camera heading (yaw) angle, a camera roll (roll) angle.
Then, before the screen defect detecting device 14 registers each second image with the first image, the screen defect detecting device 14 is further configured to: acquiring the definition of each second image and corresponding camera parameters; processing each second image, the definition of the second image and the corresponding camera parameter based on the focusing model to obtain a target adjustment parameter aiming at the camera parameter; adjusting a second image acquisition device 13 corresponding to the second image according to the target adjustment parameter; adopting the adjusted second image acquisition device 13 to acquire the image of the screen 12 again to obtain a second image acquired again; the reacquired second image satisfies a sharpness condition.
In the embodiment of the present application, if the second image capturing device 13 is a tilt camera, the camera parameters include a camera pitch (pitch) angle, a salon angle, a camera focal length, and the like. The screen defect detecting device 14 may first process each second image based on the sharpness detection model to obtain the sharpness of each second image before registering each second image with the first image. The sharpness monitoring model may be trained in advance.
Exemplarily, each second image, the definition of the second image, and the corresponding camera parameter are processed based on the focusing model, and a process of obtaining the target adjustment parameter for the camera parameter is shown in fig. 6, where in fig. 6, the diagonal definition confidence represents the definition of the second image. The focusing model may be trained in advance.
In the embodiment of the application, the target adjustment parameters of the camera parameters of the second image acquisition device are generated based on the focusing model, so that the self-adaptive focusing of the second image acquisition device can be realized, the focusing process of the second image acquisition device is more efficient and robust, even if the second image acquisition device is knocked or a detected object is changed in the screen defect detection process, the focusing of the second image acquisition device can be realized only by once adjustment, and the focusing time and the focusing complexity of the second image acquisition device can be reduced.
In an alternative embodiment, the step of constructing the focusing model comprises: acquiring a training sample, wherein the training sample comprises a plurality of sample images of a sample screen, and each sample image is marked with an initial camera parameter, a target sample adjustment parameter and the definition of the sample image; the target sample adjustment parameter is the difference value between the calibration camera parameter of the calibration image meeting the definition condition and the initial camera parameter of the sample image; taking a target image of a target shooting angle of the sample screen as a reference, and registering each sample image with the target image; for each registered sample image: inputting the registered sample image into a neural network to obtain corresponding initial sample adjustment parameters; inputting the initial sample adjustment parameters, the sample image and the calibration image into a generating type countermeasure network to obtain corresponding pixel difference values; and inputting the pixel difference value and the sample image into a neural network to obtain corresponding secondary sample adjustment parameters so as to train a focusing model until convergence.
Illustratively, the training process for the focusing model is shown in FIG. 7. The clear focus picture in fig. 7 is a calibration image satisfying the definition condition. The sharpness of the calibration image is set to 1. The unfocused photograph in fig. 7 is a sample image registered based on the target image. The initial camera parameters include camera pitch angle, gimbal angle, camera focal length. The diagonal sharpness confidence characterizes the sharpness of the sample image.
For each registered sample image: firstly, inputting a neural network to obtain corresponding initial sample adjustment parameters, wherein the sample adjustment parameters comprise a camera pitch angle difference (camera angle difference), a camera focal length difference and a Schlemm ring angle difference. And then inputting the camera angle difference, the camera focal length difference, the Schlemm ring angle difference, the unfocused picture and the clear focused picture into a generation type countermeasure network (GAN) to obtain a pixel difference value with the clear focused picture, inputting the pixel difference value into a neural network as feedback to obtain a secondary sample adjustment parameter, and repeating the steps for multiple times to obtain a final sample adjustment parameter.
In the embodiment of the application, the neural network can be trained more quickly and accurately to obtain the focusing model based on the pixel difference value as the feedback input to the neural network.
In an alternative embodiment, the focusing of the second image capturing device 13 may also be performed by real-time searching. First, the screen 12 displays a calibration pattern, and for each second image pickup device 13: a second image acquisition device 13 is adopted to acquire a plurality of third images of the screen 12, and the camera positions and/or shooting angles of any two third images are different; determining a target third image meeting a first condition according to the calibration patterns in the plurality of third images; and adjusting the position and shooting angle of the second image acquisition device 13 according to the camera position and shooting angle of the third image of the target.
In particular, the first condition may be that the view of the screen 12 is the largest in the second image acquisition device 13, the screen 12 being symmetrical in the third image, i.e. the distance from the upper edge of the third image to the upper edge of the screen 12 is equal to the distance from the lower edge of the third image to the lower edge of the screen 12; the distance from the left edge of the third image to the left edge of the screen 12 is equal to the distance from the right edge of the third image to the right edge of the screen 12.
Then, after the screen defect detecting device 14 adjusts the position and the shooting angle of the second image capturing device 13 according to the camera position and the shooting angle of the third image of the target, the screen defect detecting device 14 is further configured to: for each second image acquisition device 13: acquiring a fourth image and a fifth image of the screen 12 by using a second image acquisition device 13, wherein the fourth image corresponds to the first camera parameter, and the fifth image corresponds to the second camera parameter; determining the definition of the fourth image and the definition of the fifth image respectively; and adjusting the second camera parameter according to the respective corresponding definitions of the fourth image and the fifth image, so that the second image acquisition device 13 acquires the second image according to the adjusted second camera parameter. Therefore, the camera parameters of the second image acquisition device 13 can be adjusted in real time, and the second image acquired by the adjusted second camera parameters can meet the definition requirement.
In an optional implementation manner, adjusting the second camera parameter according to the degrees of sharpness corresponding to the fourth image and the fifth image respectively includes: under the condition that the difference value between the definition corresponding to the fifth image and the definition corresponding to the fourth image is larger than or equal to the threshold value, continuously adjusting the second camera parameter according to the direction of adjusting the first camera parameter into the second camera parameter until the preset condition is met; and under the condition that the difference value between the definition corresponding to the fifth image and the definition corresponding to the fourth image is smaller than the threshold value, continuously adjusting the second camera parameter according to the direction of the first camera parameter adjusted by the second camera parameter until the preset condition is met. The preset condition may be that the sharpness of the image acquired with the adjusted second camera parameter is highest. Therefore, an image with the highest definition can be found by dynamically adjusting the second camera parameters, and the camera parameters at the moment are stored as focusing parameters for completing focusing.
In the embodiment of the application, the second image acquisition device can be focused in real time by adjusting the position, the shooting angle and the camera parameters of the camera in real time so as to acquire the second image meeting the definition condition.
An embodiment of the present application provides a method for detecting a screen defect, as shown in fig. 8, including:
s101, acquiring a plurality of images of a screen, wherein the plurality of images comprise a first image acquired by a first image acquisition device at a target shooting angle and a plurality of second images acquired by a plurality of second image acquisition devices at other different shooting angles; the first image acquisition device and the second image acquisition device are different types of image acquisition devices, and the focusing range of the second image acquisition device is larger than that of the first image acquisition device.
In some embodiments, the first image capturing device is disposed directly above the screen, the first image capturing device may be a conventional industrial camera, and the target shooting angle is a front shooting angle, i.e., an angle directly facing the screen. The first image has at least a clear screen middle position image.
The second image acquisition device is arranged on the side face of the screen, for example, in different directions such as front, back, left and right. The other different photographing angles are different side photographing angles from the front photographing angle.
The focusing range of the second image acquisition device is larger than that of the first image acquisition device, so that when the second image acquisition device is arranged on the side face of the screen, the focusing area can cover the whole imaging area of the screen, and the second image acquisition device acquires the side-shot image of the screen, namely when the second image is acquired, the second image can be ensured to have a complete screen. The second image has at least a sharp screen edge location image.
And S102, taking the first image as a reference, and registering each second image with the first image.
In the embodiment of the present application, since the first image and the plurality of second images are taken from different angles of the screen, the first image and the plurality of second images may be registered so as to be fused subsequently.
And S103, fusing the registered second images with the first image to obtain a multi-channel image.
In the embodiment of the application, each position of the screen in the multi-channel image is clear because the first image at least has a clear screen middle position image and the second image at least has a clear screen edge position image.
And S104, determining the screen defect according to the multi-channel image.
In the embodiment of the application, the detection is carried out based on the multi-channel image, so that the defects in the screen can be detected.
According to the screen defect detection method provided by the embodiment of the application, the first image acquisition device is arranged and used for acquiring the first image of the screen at the target shooting angle; the second image acquisition devices are used for acquiring a plurality of second images of the screen at other different shooting angles; the first image acquisition device and the second image acquisition device are different types of image acquisition devices, and the focusing range of the second image acquisition device is larger than that of the first image acquisition device; therefore, when the defects of the screen are detected, the first image acquisition device can be adopted to acquire the positive shot image of the screen, the second image acquisition device with a larger focusing range than the first image acquisition device can be adopted to acquire the multi-angle side shot image of the screen, the screen side shot image can be shot clearly in a larger range, the multi-angle side shot image and the positive shot image are fused, a multi-channel image with clear angles is obtained, then defect detection is carried out based on the multi-channel image, all the defects of the screen can be detected, and the accuracy of the screen defect detection is ensured.
In some embodiments, the second image capturing device is a tilt camera, the tilt camera includes a scheimpflug, and the focusing range of the second image capturing device is adjusted by adjusting the angle of the scheimlug.
In some embodiments, prior to registering the second images with the first image, the screen defect detection method further comprises:
acquiring the definition of each second image and corresponding camera parameters; processing each second image, the definition of the second image and the corresponding camera parameter based on the focusing model to obtain a target adjustment parameter aiming at the camera parameter; adjusting a second image acquisition device corresponding to the second image according to the target adjustment parameter; adopting the adjusted second image acquisition device to acquire the screen image again to obtain a second image acquired again; the reacquired second image satisfies a sharpness condition.
In some embodiments, the step of constructing the focus model comprises: acquiring a training sample, wherein the training sample comprises a plurality of sample images of a sample screen, and each sample image is marked with an initial camera parameter, a target sample adjustment parameter and the definition of the sample image; the target sample adjustment parameter is the difference value between the calibration camera parameter of the calibration image meeting the definition condition and the initial camera parameter of the sample image; taking a target image of a target shooting angle of the sample screen as a reference, and registering each sample image with the target image; for each registered sample image: inputting the registered sample image into a neural network to obtain corresponding initial sample adjustment parameters; inputting the initial sample adjustment parameters, the sample image and the calibration image into a generating type countermeasure network to obtain corresponding pixel difference values; and inputting the pixel difference value and the sample image into a neural network to obtain corresponding secondary sample adjustment parameters so as to train a focusing model until convergence.
In some embodiments, the screen displays the calibration pattern, and before acquiring the plurality of second images, the screen defect detecting method further includes: for each second image acquisition device: a second image acquisition device is adopted to acquire a plurality of third images of the screen, and the camera positions and/or shooting angles of any two third images are different; determining a target third image meeting a first condition according to the calibration patterns in the plurality of third images; and adjusting the position and the shooting angle of the second image acquisition device according to the camera position and the shooting angle of the third image of the target.
In some embodiments, after adjusting the position and the photographing angle of the second image capturing device according to the camera position and the photographing angle of the third image of the target, the screen defect detecting method further includes: for each second image acquisition device: acquiring a fourth image and a fifth image of the screen by adopting a second image acquisition device, wherein the fourth image corresponds to the first camera parameter, and the fifth image corresponds to the second camera parameter; determining the definition of the fourth image and the definition of the fifth image respectively; and adjusting the second camera parameter according to the definition of the fourth image and the definition of the fifth image, so that the second image acquisition device acquires the second image according to the adjusted second camera parameter.
In some embodiments, adjusting the second camera parameter according to the degrees of sharpness corresponding to the fourth image and the fifth image respectively includes: under the condition that the difference value between the definition corresponding to the fifth image and the definition corresponding to the fourth image is larger than or equal to the threshold value, continuously adjusting the second camera parameter according to the direction of adjusting the first camera parameter into the second camera parameter until the preset condition is met; and under the condition that the difference value between the definition corresponding to the fifth image and the definition corresponding to the fourth image is smaller than the threshold value, continuously adjusting the second camera parameter according to the direction of the first camera parameter adjusted by the second camera parameter until the preset condition is met.
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
FIG. 9 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 9, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 801 executes the respective methods and processes described above, such as the screen defect detection method. For example, in some embodiments, the screen defect detection method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and/or installed onto device 800 via ROM 802 and/or communications unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the screen defect detection method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the screen defect detection method in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present application may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, causes the functions/acts specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to implicitly indicate the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A screen defect detection system, comprising:
the first image acquisition device is used for acquiring a first image of the screen at a target shooting angle;
the second image acquisition devices are used for acquiring a plurality of second images of the screen at other different shooting angles;
the screen defect detection device is used for acquiring the first image and the plurality of second images; registering each second image with the first image by taking the first image as a reference; fusing the registered second images with the first image to obtain a multi-channel image; determining screen defects according to the multi-channel image;
the first image acquisition device and the second image acquisition device are different types of image acquisition devices, and the focusing range of the second image acquisition device is larger than that of the first image acquisition device.
2. The screen defect detecting system of claim 1, the second image capturing device being a tilt camera, the tilt camera comprising a Samm ring, the angle of the Samm ring being adjusted to adjust the focus range of the second image capturing device.
3. The screen defect detecting system of claim 1, the screen defect detecting device, prior to registering each of the second images with the first image, further to:
acquiring the definition of each second image and corresponding camera parameters;
processing each second image, the definition of the second image and the corresponding camera parameter based on a focusing model to obtain a target adjustment parameter aiming at the camera parameter; adjusting a second image acquisition device corresponding to the second image according to the target adjustment parameter; adopting the adjusted second image acquisition device to acquire the screen image again to obtain a second image acquired again; the reacquired second image satisfies a sharpness condition.
4. The screen defect detection system of claim 3, the step of building the focusing model comprising:
acquiring a training sample, wherein the training sample comprises a plurality of sample images of a sample screen, and each sample image is marked with an initial camera parameter, a target sample adjustment parameter and the definition of the sample image; the target sample adjustment parameter is the difference value between the calibration camera parameter of the calibration image meeting the definition condition and the initial camera parameter of the sample image;
taking a target image of a target shooting angle of the sample screen as a reference, and registering each sample image with the target image;
for each registered sample image: inputting the registered sample image into a neural network to obtain a corresponding initial sample adjustment parameter; inputting the initial sample adjustment parameters, the sample image and the calibration image into a generative confrontation network to obtain corresponding pixel difference values; and inputting the pixel difference value and the sample image into the neural network to obtain corresponding secondary sample adjustment parameters so as to train a focusing model until convergence.
5. The screen defect detection system of claim 1, said screen displaying a calibration pattern,
before acquiring a plurality of second images, the screen defect detection device is further configured to:
for each of the second image acquisition devices:
collecting a plurality of third images of the screen by adopting the second image collecting device, wherein the camera positions and/or shooting angles of any two third images are different;
determining a target third image meeting a first condition according to the calibration patterns in the third images;
and adjusting the position and the shooting angle of the second image acquisition device according to the camera position and the shooting angle of the third image of the target.
6. The screen defect detecting system of claim 5, the screen defect detecting device, after adjusting the position and the shooting angle of the second image capturing device according to the camera position and the shooting angle of the third image of the target, is further configured to:
for each of the second image acquisition devices:
acquiring a fourth image and a fifth image of the screen by using the second image acquisition device, wherein the fourth image corresponds to the first camera parameter, and the fifth image corresponds to the second camera parameter;
determining the definition of the fourth image and the definition of the fifth image respectively;
and adjusting the second camera parameters according to the definitions corresponding to the fourth image and the fifth image respectively, so that the second image acquisition device acquires the second image according to the adjusted second camera parameters.
7. The screen defect detecting system of claim 6, wherein the adjusting the second camera parameter according to the degrees of sharpness corresponding to the fourth image and the fifth image respectively comprises:
under the condition that the difference value between the definition corresponding to the fifth image and the definition corresponding to the fourth image is larger than or equal to a threshold value, continuously adjusting the second camera parameter according to the direction of adjusting the first camera parameter to the second camera parameter until a preset condition is met; and
and under the condition that the difference value between the definition corresponding to the fifth image and the definition corresponding to the fourth image is smaller than a threshold value, continuously adjusting the second camera parameter according to the direction of adjusting the second camera parameter to the first camera parameter until a preset condition is met.
8. A screen defect detection method comprises the following steps:
acquiring a plurality of images of a screen, wherein the plurality of images comprise a first image acquired by a first image acquisition device at a target shooting angle and a plurality of second images acquired by a plurality of second image acquisition devices at other different shooting angles; the first image acquisition device and the second image acquisition device are different types of image acquisition devices, and the focusing range of the second image acquisition device is larger than that of the first image acquisition device;
registering each second image with the first image by taking the first image as a reference;
fusing the registered second images with the first image to obtain a multi-channel image;
and determining the screen defects according to the multi-channel image.
9. An electronic device, comprising:
at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the screen defect detection method of claim 8.
10. A computer-readable storage medium storing computer instructions for causing a computer to perform the screen defect detecting method of claim 8.
CN202210344598.0A 2022-03-31 2022-03-31 Screen defect detection system and method, electronic device and readable storage medium Pending CN114913121A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210344598.0A CN114913121A (en) 2022-03-31 2022-03-31 Screen defect detection system and method, electronic device and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210344598.0A CN114913121A (en) 2022-03-31 2022-03-31 Screen defect detection system and method, electronic device and readable storage medium

Publications (1)

Publication Number Publication Date
CN114913121A true CN114913121A (en) 2022-08-16

Family

ID=82763608

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210344598.0A Pending CN114913121A (en) 2022-03-31 2022-03-31 Screen defect detection system and method, electronic device and readable storage medium

Country Status (1)

Country Link
CN (1) CN114913121A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115598136A (en) * 2022-10-28 2023-01-13 深圳市元硕自动化科技有限公司（Cn） Detection apparatus for screen rubber coating quality
CN115953422A (en) * 2022-12-27 2023-04-11 北京小米移动软件有限公司 Edge detection method, apparatus and medium
CN116912233A (en) * 2023-09-04 2023-10-20 深圳市明亚顺科技有限公司 Defect detection method, device, equipment and storage medium based on liquid crystal display screen
CN117056148A (en) * 2023-08-30 2023-11-14 昆山迈致治具科技有限公司 Method for detecting abnormal display of screen
CN117115433A (en) * 2023-10-24 2023-11-24 深圳市磐鼎科技有限公司 Display abnormality detection method, device, equipment and storage medium

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115598136A (en) * 2022-10-28 2023-01-13 深圳市元硕自动化科技有限公司（Cn） Detection apparatus for screen rubber coating quality
CN115598136B (en) * 2022-10-28 2023-08-08 深圳市元硕自动化科技有限公司 Screen gluing quality detection device
CN115953422A (en) * 2022-12-27 2023-04-11 北京小米移动软件有限公司 Edge detection method, apparatus and medium
CN115953422B (en) * 2022-12-27 2023-12-19 北京小米移动软件有限公司 Edge detection method, device and medium
CN117056148A (en) * 2023-08-30 2023-11-14 昆山迈致治具科技有限公司 Method for detecting abnormal display of screen
CN117056148B (en) * 2023-08-30 2024-02-09 昆山迈致治具科技有限公司 Method for detecting abnormal display of screen
CN116912233A (en) * 2023-09-04 2023-10-20 深圳市明亚顺科技有限公司 Defect detection method, device, equipment and storage medium based on liquid crystal display screen
CN116912233B (en) * 2023-09-04 2024-02-13 深圳市明亚顺科技有限公司 Defect detection method, device, equipment and storage medium based on liquid crystal display screen
CN117115433A (en) * 2023-10-24 2023-11-24 深圳市磐鼎科技有限公司 Display abnormality detection method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN114913121A (en) Screen defect detection system and method, electronic device and readable storage medium
US20230059657A1 (en) Multi-camera zoom control method and apparatus, and electronic system and storage medium
EP3641298B1 (en) Method and device for capturing target object and video monitoring device
US20230027389A1 (en) Distance determination method, apparatus and system
CN108668086B (en) Automatic focusing method and device, storage medium and terminal
CN111368717A (en) Sight line determining method and device, electronic equipment and computer readable storage medium
CN110555838A (en) Image-based part fault detection method and device
CN112291473B (en) Focusing method and device and electronic equipment
CN113724391A (en) Three-dimensional model construction method and device, electronic equipment and computer readable medium
CN111191619B (en) Method, device and equipment for detecting virtual line segment of lane line and readable storage medium
CN110658918B (en) Positioning method, device and medium for eyeball tracking camera of video glasses
CN108769538B (en) Automatic focusing method and device, storage medium and terminal
CN109816628B (en) Face evaluation method and related product
JP6283329B2 (en) Augmented Reality Object Recognition Device
US9041689B1 (en) Estimating fingertip position using image analysis
CN111369513B (en) Abnormality detection method, abnormality detection device, terminal equipment and storage medium
JP6483661B2 (en) Imaging control apparatus, imaging control method, and program
CN115578515B (en) Training method of three-dimensional reconstruction model, three-dimensional scene rendering method and device
WO2023273158A1 (en) Method and apparatus for determining operating range of camera in cooperative vehicle infrastructure and roadside device
US20210279907A1 (en) Methods and systems for sensing obstacles in an indoor environment
US20220345621A1 (en) Scene lock mode for capturing camera images
TWI650711B (en) Action recognition methods and systems thereof
US10609311B2 (en) Method and device for increasing resolution of an image sensor
CN113705620B (en) Training method and device for image display model, electronic equipment and storage medium
CN115631249B (en) Camera correction method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination