CN110602355A - an image acquisition method - Google Patents

Abstract

The invention discloses an image acquisition method, comprising the steps of: driving a camera or/and a target to make them move relative to each other, so that the part to be photographed of the target can appear in the clear imaging plane of the camera in a time-divisional manner; Image; process the image captured by the camera to identify the part where the clear imaging surface of the camera coincides with the surface of the target object. The present invention has the advantages of providing an image acquisition system and method, which can still obtain a clear image of the surface of the target object without adjusting the spatial position of the object to be measured/camera in advance.

Description

an image acquisition method

technical field

The present invention relates to an image acquisition system and method.

Background technique

When collecting images of target objects through image acquisition devices such as cameras and video cameras, it is often necessary to adjust parameters such as the focal length and aperture of the camera and the camera, or the relative relationship between the camera, the camera and the target object, such as the direction and distance, according to the optical imaging relationship. Such adjustment steps complicate the task of acquiring images of the target object.

Although the current camera has the function of automatic zoom, due to the limitation of the depth of field of the camera itself, especially in macro photography, the camera cannot obtain a complete and clear image of the surface of an object with a certain length through adjustment operations such as automatic zoom.

SUMMARY OF THE INVENTION

An image acquisition method, comprising the steps of: driving a camera or/and a target to make them move relative to each other, so that the part to be photographed of the target can appear in a clear imaging surface of the camera in a time-divisional manner; collecting an image of the target; processing the camera The image taken to identify the part where the clear imaging surface of the camera coincides with the surface of the target object.

Further, the camera or/and the object are driven to rotate around a rotation axis, and the rotation axis is parallel to the clear imaging plane of the camera.

Further, the camera or/and the object are driven to move in a straight line.

Further, processing the image captured by the camera includes: calculating the peak value of the gradient value of the image parameter of the pixel/pixel group in the image.

Further, processing the image captured by the camera includes: denoising the image.

Further, the image acquisition method further includes: acquiring the position information of the contour of the target object; acquiring the position information of the clear imaging plane of the camera; calculating the distance between the surface of the target object and the camera according to the position information of the contour of the target object and the position information of the clear imaging plane of the camera. Coincidence of clear image planes.

Further, the location information includes coordinate information.

Further, the image acquisition method further includes: emitting laser light, and the area formed by the laser light includes the clear imaging surface of the camera; acquiring the position information of the laser spot in the image, and matching the position information of the laser spot with the image.

Further, acquiring the position information of the laser spot in the image includes: filtering the image by using a channel corresponding to the laser color in the multicolor camera to obtain a filtered image; and acquiring the position information of the laser spot in the filtered image.

Further, the image acquisition method further includes: acquiring several images of the target object; and merging the overlapping parts of the clear imaging surface of the camera and the surface of the target object in the several images.

The present invention has the advantages of providing an image acquisition system and method, which can still obtain a clear image of the surface of the target object without adjusting the spatial position of the object to be measured/camera in advance.

Description of drawings

1A-1C are schematic diagrams of various image acquisition systems of the present invention;

Figure 2 is a schematic diagram of the working state of the scanning camera;

3A is a system block diagram of an industrial inspection system;

3B is a schematic diagram of an industrial detection system;

4 is a schematic diagram of an encoder installation method;

5A is a schematic perspective view of a camera with an offset configuration;

5B is an internal schematic diagram of a camera with an offset configuration;

6A is a schematic diagram of an image acquisition subsystem including 3 cameras;

6B-6D are schematic diagrams showing the positional relationship of the clear imaging planes P of each camera of the image acquisition subsystem including three cameras;

FIG. 7A is a schematic diagram of an image of a part of a flawed edge frosted glass;

FIG. 7B is a schematic diagram of another image of a flawed part of the edge frosted glass;

8A is a schematic diagram of a strip light source of an industrial inspection system;

8B-8D are schematic diagrams of a ring light source of an industrial detection system;

9A is a flow chart of an industrial inspection system including a position-aware system for judging edge flaws of frosted glass edges using images before fusion;

FIG. 9B is a flow chart of an industrial inspection system including a position-aware system for judging edge flaws of edge frosted glass using the fused image.

Detailed ways

Clear imaging plane P

For an image acquisition device, such as a camera, camcorder, scanner, etc., according to the optical principle, if the physical conditions such as the model and location of the optics and electronic components in the image acquisition device are not changed, the distance and range that the camera can clearly image is often fixed. of.

As shown in FIGS. 1A-1C , the area that can be clearly photographed by the camera 101 is defined as the clear imaging plane P. Generally speaking, the shape of the clear imaging plane P depends on the hardware of the camera 101 , especially the shape of the image sensor. In FIGS. 1A-1C , the example is drawn as a rectangle for illustration only. In fact, the clear imaging plane P can be any arbitrary shape. shape. The size of the clear imaging plane P and the distance and angle between it and the camera 101 are determined by the relative relationship between the lens and the image sensor.

Image acquisition system

The image acquisition system at least includes an image acquisition device and a processor, where a camera is used as the image acquisition device.

The clear imaging plane P of the camera is parallel to the surface of the target object

When performing macro photography, the camera is often required to photograph the subject 104 at a close distance.

As shown in FIG. 1A , 102 is a loading device for placing the target object 104 . 103 is a positioning block for fixing the position of the target object 104 . If the part of the object 104 that needs to be photographed is exactly a plane, and is just within the clear imaging plane P, a complete and clear image of the photographed part can be obtained.

However, if the position of the subject 104 changes as shown in FIG. 1B (or there is no plane structure), at this time, if the subject 104 is photographed by the camera 101, the depth of field of the camera for close-up shooting is small, so The part of the object 104 that is not within the clear imaging plane P cannot obtain clear imaging.

In this case, even by adjusting the focal length of the camera 101 or adjusting the relative distance between the camera 101 and the subject 104, a complete and clear image of the photographed part of the subject 104 cannot be obtained.

The clear imaging plane P of the camera is not parallel to the surface of the target object

The present invention provides an image acquisition system including: a camera 101 , a driving device 107 and a processor (not shown in FIG. 1A and FIG. 1B ).

Among them, the camera 101 is used to capture an image of a subject. The camera 101 can be a zoom camera or a fixed focal length camera.

The driving device 107 can drive both the camera 101 and the subject 104, or both. In this embodiment, the driving device 107 is a motor. As a specific solution, the driving device includes an object carrier 102 for contacting, holding or transporting the object 104 to be photographed, and the object carrier 102 is driven by the driving device to drive the movement of the object 104 to be photographed. The advantage of this is that the camera 101 does not have to be adjusted frequently.

As shown in Figures 1B and 1C, when the clear imaging plane P of the camera is not parallel to the surface of the object, no matter how the focal length/object distance is adjusted, the clear imaging plane P of the camera cannot completely overlap the surface of the target object. At this time, there is often only one line of intersection between the clear imaging plane P of the camera and the surface of the target object in terms of the geometric relationship. In this case, only part of the image in the photo taken by the camera is clear, and the clear point is the line of intersection between the clear imaging plane P of the camera and the surface of the target object.

The following is an example of collecting an image of the surface of a cube. The surface of the cube is not parallel to the clear imaging plane P of the camera, and it is considered that the clear imaging plane P of the camera intersects the surface of the target object.

Identify sharp parts in an image by peaks in image parameter gradients

As an optional implementation manner, the steps of collecting and judging the clear part of the cube image are as follows:

Acquire a raw image of the cube surface. In this image, the image of the intersection of the clear imaging plane P of the camera and the surface of the object is clear, and the image of other parts is blurred. The images captured in this step can be grayscale or color images.

The original image of the cube surface is denoised to obtain a denoised image. Denoising can reduce the influence of noise generated by hardware or environment on subsequent image processing work.

Computes the absolute value of the gradient of parameter values between adjacent pixels/groups of pixels in the denoised image. The parameters here can be gray value, brightness value, or other parameters such as contrast and saturation.

According to the absolute value of the gradient value calculated in the preceding steps, a Gaussian function is used to determine the peak point x _max of the gradient. In the same picture, a "blurred" pixel can be regarded as taking the average value of the pixels around the blurred pixel, reducing the contrast between the pixel and the surrounding pixels, resulting in a blurred visual effect. The contrast between the "sharp" pixels and the surrounding pixels is sharper, which can better reflect the details of the pixels, and the visual effect is clearer. Excluding the influence of noise, in the same image, the larger the absolute value of the gradient, the clearer the image there. As an optional solution, the way of judging the peak point of the gradient value may be other calculation methods other than the Gaussian function. The location of the peak point X _max is the clear part of the picture.

In the process of calculating the gradient value, the gradient values can be arranged in order according to the position of the pixel/pixel group in the picture, that is, the calculation result of each gradient value corresponds to a specific position in the picture. At this time, when the peak point X _max is obtained, the position of the peak point in the picture, that is, the specific position of the clear part in the picture, can also be obtained.

This peak judgment can be realized by using the gradient processing unit in the processor.

The method of judging clear parts by the peak value of image parameters can achieve better results when shooting objects with rich surface textures.

Identify sharp parts in an image by calculating intersections

Create a coordinate system

As an optional implementation manner, the line segment representing the clear imaging plane P of the camera and the surface contour of the target object are represented in the same coordinate system. The coordinate system may be a two-dimensional coordinate system, and the plane of the coordinate system is perpendicular to the clear imaging plane P of the camera.

The position of the camera is fixed, and the parameters of the camera's lens and image sensor are unchanged. At this time, the position of the line segment p corresponding to the clear imaging plane P of the camera in the coordinate system is also fixed. If the x-axis, the y-axis and the coordinate origin are selected, each point on the line segment p has a fixed coordinate value. The line segment p can be represented by a linear function within a certain interval. The position of the intersection of the line segment p and the surface contour of the target object corresponds to the position of the clear part in the image.

Obtain the coordinate information of the surface contour of the target object

As shown in FIG. 2 , a scanning camera 201 is used to obtain the coordinate information of the surface contour of the object. The scanning camera is placed above, usually directly above, the target object. The motion mechanism generates relative motion between the scanning camera and the target object, so that the scanning camera can traverse all over the surface of the object. In this schematic diagram, the scanning camera 201 is mounted on the guide rail 202 through a motion mechanism. The shooting direction of the scanning camera is perpendicular to the plane of the coordinate system to obtain the projected contour of the target object in the coordinate system. During the relative movement of the two, the distance between the scanning camera 201 and the plane where the coordinate system is located remains constant. The scanning camera 201 can be a line scan camera or an area scan camera. The relative movement direction between the camera and the target object is defined as the x direction, and the y direction is perpendicular to the x direction.

The line scan camera obtains the contour coordinate information of the object surface

A line scan camera is a camera that uses a line scan image sensor. Compared with the area scan camera, the image obtained by the line scan camera has a higher resolution, but since it can only obtain one line image at a time, in order to obtain a complete image of the object to be photographed, subsequent processing operations are required.

The line scan camera is used to obtain the coordinate information of the surface contour of the object. The specific working steps are as follows:

Line scan camera scanning: The motion range of the line scan camera is fixed, and it scans continuously at a fixed interval distance, and outputs images, each image is a line.

Threshold processing: Make the difference between the image obtained by the line scan camera and the background image when there is no occlusion, and compare the difference or the absolute value of the difference with the preset threshold. When the difference is within a certain range, it is judged that the line scan camera obtained The line of image does not contain the target object, in this case, the gray value of the image is adjusted to a certain fixed value. When the difference value exceeds a certain range, the image here is considered to be the image of the target object. At this time, the gray value of the image here is adjusted to another fixed value and the image of the target object (line image) obtained by the line scan camera is used. The two endpoints are recorded. Through such an operation, an image of the area covered by the object can be obtained, and the image is a binary image. Binary images are easier to process. And the image information only records the two end points of the line image, which is used to form the outline of the object, and other useless information is discarded, which reduces the storage capacity of the system.

X-direction coordinate acquisition: The motion mechanism causes the line scan camera to move relative to the target object, and the displacement information of the motion mechanism at any time is obtained by using a displacement acquisition device such as an encoder, and the displacement information of the target object at that moment is also obtained. The encoder outputs the displacement information corresponding to this line of images. Since the shooting direction of the line scan camera is perpendicular to the plane of the coordinate system, the position information of the encoder corresponding to the line image is the x-coordinate information of the line image.

The y-direction coordinate acquisition: the length of the scan line of the line scan camera is fixed, and the y-coordinate of one end of the scan line is set to 0, then the y-coordinates of the two endpoints in the line image are calculated by calculating the distance between the two endpoints and the endpoint of the scan line. distance obtained.

Smoothing: splicing the front and back multi-frame images obtained by the line scan camera, and performing smooth filtering on the spliced contour information to obtain the curve information of the entire object edge. Smooth filtering can eliminate the error caused by single-frame image processing, so that the contour of the target object can reflect the real information more accurately, which is beneficial to subsequent processing.

The area scan camera obtains the contour coordinate information of the object surface

Area scan cameras use area image sensors. The imaging area is a surface, and a complete image of the object to be shot can be obtained by one shot.

Using an area array camera to obtain the coordinate information of the surface contour of the object, the specific working steps are as follows:

Area scan camera shooting: The target object is placed within the shooting range of the area scan camera to ensure that the area scan camera can obtain a complete image of the target object in one shot. If the image collected by the area scan camera is not a complete image, you can refer to the working steps of the line scan camera to collect the complete image of the target object.

Thresholding: The steps of thresholding are similar to line scan cameras.

Coordinate acquisition in x and y directions: the position of the area array camera is fixed, and the shooting range is also fixed. The coordinates of a boundary point that defines the shooting range of the area scan camera are (0, 0), then the x and y coordinate information of the outline point represented by the binary image can be obtained by calculating the distance between the outline point and the boundary point. .

Smoothing: The steps for smoothing are similar to those of the line scan camera described above.

Calculate the intersection to get a clear location in the image

As an optional implementation manner, the steps of collecting and judging the clear part of the cube image are as follows:

Acquire a raw image of the cube's surface. The captured image can be a grayscale image or a color image.

The scanning camera scans the image of the target object to obtain the coordinate information of the edge contour of the target object. The coordinate information of the line segment p projected by the clear imaging plane P of the camera in the coordinate system is pre-calibrated.

Calculate the intersection between the line segment p and the edge contour of the target object. The image at this intersection is the clear part of the original image. At this time, the processor obtains the relative position information of the clear imaging part in the target object.

Use laser to mark clear imaging surface to identify clear parts of image

In the case where the clear imaging plane of the camera is not parallel to the surface of the target object, if there are still clear parts in the image captured by the camera at this time, the clear imaging plane P of the camera and the surface of the target object can be approximately geometrically related It is assumed that there is a line of intersection.

Using the laser to mark the position of the clear imaging surface, the place where the surface of the target object is illuminated by the laser at any time is the intersection of the clear imaging surface P of the camera and the surface of the target object. When reflected in the image captured by this camera, the spot in the image illuminated by the laser is where the clear part of the image is located.

As shown in FIG. 1B , as an embodiment, the laser emitting device 108 is disposed on the side of the clear imaging plane P of the camera, and includes several laser emitters. The laser beams emitted by the several laser emitters are parallel to each other, and each beam The lasers are located on the same plane to form a laser area, and the laser area covers the clear imaging plane P of the camera. As another optional implementation, as shown in FIG. 1C , the laser emitting device 108 is arranged above the clear imaging plane P, and the laser emitter emits planar laser light from top to bottom. Since the laser light has a certain width, the target object Due to the diffuse reflection of the surface itself, the laser spot will still appear on the surface of the object and will not be completely blocked by the object. The planar laser area covers the clear imaging plane P of the camera. In this embodiment, the laser is a red laser with a wavelength of 650 nm. As other optional implementations, the color of the laser light can also be green or other colors.

As a way of judging the clear part in the image according to the laser spot, the camera adopts a multi-color camera, including a first channel and a second channel. In this embodiment, the camera adopts a color camera, including three RGB channels. In this embodiment, the laser adopts a red laser with a wavelength of 650 nm, the first channel is R channel, and the second channel is G channel or B channel. It is worth mentioning that both the first channel and the second channel defined here may include several channels. For example, in this embodiment, the second channel may be a GB channel. The processor includes a channel selection unit for selecting channels of the polychromatic camera. The processor extracts the first channel image collected by the multi-color camera, and determines the laser spot area in the first channel image. Then, the second channel image is extracted, and the laser spot area in the first channel image is matched with the second channel image. The laser spot area in the first channel image is the area of the clear image in the second channel image. As an optional embodiment, when the color of the laser light is green, the first channel is the G channel. Using a multi-color camera to judge the laser spot area and the position of the clear image respectively can avoid the interference of the laser irradiation on the image information, and it is convenient to use the image for subsequent operations such as defect detection.

As an optional implementation manner, optical elements such as gratings are used to make the light intensity distribution of the laser region uniform. The uniform laser light intensity distribution will not bring too much interference to the surface information of the target object. In this case, there is no need to filter the laser light with a polychromatic camera.

The method of laser marking the clear imaging surface can also be adapted to different shooting scenes, and equipment such as scanning cameras is omitted.

image fusion

In order to obtain a complete image of the object surface, a processor is used to fuse clear images corresponding to different positions on the object surface to form a complete clear image of the object surface. Before performing image fusion, the processor has obtained the relative positional relationship of each clearly imaged part relative to other clearly imaged parts in the same image according to the above system and method.

As an embodiment, the camera is mounted on the driving device, and the shooting position can be changed along with the driving device. As an embodiment, the driving device includes a mounting seat, a stepping motor, a transmission mechanism, a wheel, and a guide rail. The camera is fixedly connected to the mount. The stepper motor can be set inside the mount, or it can be mounted outside the mount. One end of the transmission mechanism is connected with the wheel, and the other end is connected with the stepping motor. The stepping motor drives the transmission mechanism to drive the wheels to rotate. The wheel can move along the guide rail, and its movement trajectory is determined by the guide rail. During a shooting process, the driving device moves in a certain direction according to the movement track of the guide rail. Driven by the stepping motor, the camera changes the same distance each time. The camera takes pictures of the surface of the object at different positions according to the walking sequence of the driving device. The processor receives these pictures, obtains clear parts in these pictures by judging gradient peaks or calculating intersection points, and splices the clear parts in these pictures in order. (D in the attached figure represents the relative movement direction of the object and the camera)

As an optional embodiment, the motor of the driving device can be a servo motor.

As another embodiment, the driving device can drive the target object or the camera to rotate around a rotation axis, and the rotation axis is parallel to the clear imaging plane P of the camera. In this way, images of different parts of the surface of the target object can also be acquired.

As another optional implementation, the position of the camera is fixed, and the driving device drives the object to be photographed to move. The camera takes multiple pictures at different positions of the target object. The processor receives these pictures, obtains clear parts in these pictures by judging gradient peaks or calculating intersection points, and splices the clear parts in these pictures in order. At this time, the driving equipment may be an industrial production line, a roller, a transport vehicle, or the like.

In a word, the driving device is used to drive the camera or/and the target object, so that the part to be photographed of the target object can appear in the clear imaging plane P of the camera in time division. As an optional implementation manner, the image acquisition system may further include a positioning device, which forms a data connection with the driving device and the processor respectively, so that the processor can match the clear part in the image with the part corresponding to the target object.

As an alternative, the camera may be connected to the processor. The camera transmits the captured picture to the processor, which processes the image to obtain information on the surface of the object contained in the image. The information on the surface of the object includes defect information on the surface of the object, text on the surface of the object, logo information, color, texture, pattern, and the like on the surface of the object. Wherein, the defect information includes at least: information on whether there is a defect in the image; location information of the defect in the image.

By calculating the position of the intersection of the clear imaging plane and the surface of the object to determine the clear part of the image, it can adapt to various shooting scenes and will not be affected by lighting conditions or surface texture of the object.

Shooting with multiple cameras

Several cameras can be included in the image acquisition system to improve the shooting efficiency. For example, when the rotary driving device 107 is used to drive the object to rotate, if there is only one camera, the rotary driving device usually needs to rotate 360° to capture a complete image of the surface of the target object. If two oppositely set cameras are used, the target object is placed between the two cameras, and the clear imaging surfaces of the two cameras are perpendicular to the same plane. At this time, the driving device only needs to rotate 180° to complete the shooting, which reduces the time by half. If three cameras are used, the angle between the center lines of the two cameras is 120°, and the planes where the clear imaging planes of the two cameras are located obliquely intersect. At this time, the driving device only needs to rotate 120° to complete the shooting.

Online detection system

Industrial testing often has higher testing standards. If the artificial eye detection is replaced by image acquisition detection, the camera needs to take a close-up shot of the object to be detected to obtain finer object surface image information.

As shown in FIGS. 3A and 3B , an industrial inspection system based on image acquisition includes an industrial production line 30 , a position acquisition subsystem 31 , an image acquisition subsystem 32 , and a processor 33 .

The following takes the glass edge online detection system as an example to specifically introduce the online detection system including the image acquisition system of the present invention. In this embodiment, the glass to be tested 39 is sheet glass. As an optional embodiment, the detection system can also detect surface defects of objects other than glass, such as wood surface defects, steel surface defects, stone surface defects, and the like.

The position learning subsystem 31 is used to obtain the coordinate information of the edge of the glass to be tested. When identifying the clear part in the image by calculating the gradient peak value of the image parameter or marking the clear imaging surface with laser, the position learning subsystem can be omitted. Image acquisition subsystem 32 includes one or more cameras for acquiring images of the edge of the glass. The processor 33 forms a data connection with at least the position acquisition subsystem 31 and the image acquisition subsystem 32 for performing various image and data processing operations.

The industrial inspection system can obtain a clear image of the surface of the object to be inspected through the machine. Save labor and improve detection accuracy. With this industrial inspection system, the object to be tested can be placed in any position and direction on the industrial production line.

Industrial production line

The industrial line 30 is used to place the glass to be photographed. The industrial production line can at least transport the object to be detected on a transport surface in a transport direction along a transport line. In this embodiment, the industrial production line drives the glass placed on the industrial production line to move.

Industrial lines include transport units for transporting the product to be tested.

As an optional embodiment, the industrial production line includes a roller frame and several conveying rollers, the several conveying rollers and the roller frame are rotatably connected, and the axes of the several conveying rollers are located on the same plane.

As other optional embodiments, the industrial production line may also include conveying devices such as conveyor belts and conveying vehicles.

The surface of the industrial production line can be made of non-slip rubber material, or a suction cup can be provided to increase the friction between the industrial production line and the object to be transported, thereby increasing the friction between the industrial production line and the glass, and improving the transportation efficiency of the industrial production line. .

Location Awareness Subsystem 31

The position learning subsystem is used to obtain the coordinate information of the product to be tested. When identifying the clear part in the image by calculating the gradient peak value of the image parameter or marking the clear imaging surface with laser, the position learning subsystem can be omitted.

The coordinate system is established by taking the transportation surface of the industrial production line as the plane where the coordinate system is located. The coordinate system may be a two-dimensional coordinate system. The outline of the object placed on the industrial production line is projected into a closed figure in this coordinate system. Take the transportation direction of the industrial production line as the x direction, and the y direction perpendicular to the transportation direction. The position learning subsystem at least needs to know the relative position between the product to be inspected and the camera.

front camera

As shown in FIG. 3B , the front camera 312 is disposed above the industrial production line 30 , and is usually disposed directly above the industrial production line 30 . Of course, as an optional implementation manner, the front camera 312 can be set according to actual needs. However, no matter what the setting is, it must be ensured that the imaging range of the front camera 312 can cover a suitable industrial production line area. The front camera 312 is separated from the industrial production line 30 by a certain distance. The lens of the front camera faces the industrial production line 30, and its optical axis is perpendicular to the plane where the axes of the several conveying rollers are located.

The front camera may be a line scan camera (line scan camera), or an area scan camera. The scanning camera is used to obtain the coordinate information of the object to be measured.

As an optional implementation manner, the front camera 312 is a line scan camera.

Acquisition of industrial production line position information (glass profile x coordinate value)

The glass to be tested is placed on the industrial production line and moves with the industrial production line. From the coordinate plane, at any moment, as long as the position information of the industrial production line is obtained, the x-coordinate value of the glass to be tested at that moment can be obtained.

As an implementation manner, the image acquired by the front camera is judged, and the x-coordinate value of the part of the same piece of glass to be tested that is first acquired by the front camera is set to 0.

As an optional embodiment, a position detection device is provided in the y direction of the industrial production line. As an embodiment, the position detection device is a photogate, including a signal transmitting device arranged on one side of the industrial production line and a signal receiving device arranged oppositely, and the connection between the signal transmitting device and the receiving device is perpendicular to the output of the industrial production line. Straight line. Set the x-coordinate of any point on the line to 0. In the case of no shielding, the receiving device of the photogate can always receive the light signal sent by the light-emitting device of the photogate. In the case of blocking, the receiving device of the photogate cannot receive the light signal sent by the light-emitting device of the photogate.

The photogate is connected to the processor, and the distance between the photogate and the front camera is kept constant. At this time, the x-coordinate information of the scanning area of the front camera can also be obtained.

The encoder gets the coordinates

As a displacement detection device, the encoder includes an encoder disc and an encoder reading unit, which are used to convert angular displacement or linear displacement into electrical signals.

As shown in FIG. 3B and FIG. 4 , the industrial detection system based on image acquisition includes an encoder 311, and the encoder may be a rotary encoder, including an absolute encoder and an incremental encoder.

The encoder's code disc can be connected to the motor via a coupling, or it can be connected to a transmission for direct industrial line motion. When the encoder is connected to the motor, the encoder directly obtains the rotation information of the motor, and the corresponding motion coordinates of the industrial production line need to be converted by the reduction ratio.

The encoder is connected to the processor for sending information to the processor. According to the image acquired by the scanning camera for the first time or the starting point of the photogate, the motion coordinates of the industrial production line recorded by the encoder at any time are the x coordinates of the point in front of the glass.

Because obtaining the position information of the industrial production line is the original step of the whole inspection system inspection process, it is also a key step. If this step produces an error, the output of the various subsequent steps may be wrong. Therefore, it is necessary to ensure the accuracy of the information obtained by the encoder.

As an alternative to photogates, the processor calculates the real-time speed of the industrial production line based on the data transmitted by the encoder. At the moment when the real-time speed of the industrial production line suddenly drops, it is judged that glass is placed on the industrial production line. The motion coordinate of the industrial production line at the moment of the sudden drop in speed is marked as 0. At this time, the x coordinate of the point in front of the glass is also 0.

As an optional embodiment, the glass edge detection system includes a plurality of encoders, and the plurality of encoders are installed in different positions or different parts of the industrial production line. For example, in the case of using two encoders, the main encoder 311a and the sub-encoder 311b are respectively connected to the motor and the transmission mechanism of the industrial production line. If the information obtained by the sub-encoder 311b is consistent with the information obtained by the main encoder 311a, or the error is within a certain range, it is considered that the information obtained by the encoder at this time is accurate. The setting of such main and sub encoders can timely detect encoder faults and improve detection accuracy.

Use brushless motor parameters to obtain coordinates

The brushless motor is controlled by electronic commutation commands, and its speed can be controlled by a pulse width modulated signal. Due to this characteristic of the brushless motor, the brushless motor controller can obtain the rotation angle of the brushless motor by recording the number of control signals, thereby obtaining the motion information of the industrial production line.

When the photoelectric sensor outputs zero coordinates, the glass enters the detection area, and the brushless motor controller records the pulse width and number of the control signal of the brushless motor, and calculates the pulse width and number of the industrial production line at any time according to the pulse width and number of the control signal. The motion coordinate, that is, the x coordinate of the point in front of the glass.

Time interval between front camera shots

Whether using an encoder or a brushless motor, the real-time speed of an industrial production line can be obtained. Due to the different loads placed on the industrial production line, or due to the difference in the lubrication environment of the industrial production line machinery, the industrial production line often does not move at a uniform speed. At this time, the processor calculates the real-time movement speed of the industrial production line. Calculate the time required for the industrial production line to move at fixed intervals at this speed, and trigger the front camera to shoot at the corresponding time point. Using this triggering method at the shooting moment of the front camera can ensure that the pictures shot by the front camera meet the requirements of post-processing. For example, when the front camera adopts a line scan camera, it can ensure that the movement interval of the industrial production line corresponding to each shot of the line scan camera is constant; when the front camera adopts an area scan camera, it can ensure that the area scan camera each time Shooting can get a complete image of the glass surface.

The specific shooting interval setting when using a line scan camera is subject to the requirements under different imaging quality requirements. For example, for the glass to be tested with a larger size, the shooting frequency of the line scan camera is relatively reduced, which reduces energy consumption and reduces the heating of the line scan camera. Of course, the shooting frequency of the line scan camera can also be increased to increase the fineness of imaging.

After the front camera is completed, the coordinate information of any point on the edge of the glass at any time can be obtained in combination with the coordinate information of the industrial production line corresponding to the control parameters of the encoder or brushless motor.

Image Acquisition Subsystem 32

The image acquisition subsystem acquires images, and the processor identifies clear parts in the acquired images by judging the gradient peaks of image parameters, calculating intersection positions or marking clear imaging surfaces with lasers. In this embodiment, the image acquisition subsystem 32 is used to acquire an image of the edge of the glass to be tested. Image acquisition subsystem 32 may include multiple cameras 321 .

As an embodiment, the image acquisition subsystem 32 is disposed after the position acquisition subsystem 31 . When the glass to be tested is transported, it first passes through the position acquisition subsystem 31 , and then passes through the image acquisition subsystem 32 . Such an arrangement enables the coordinate information of the edge of the glass to be measured to be known when the image acquisition subsystem 32 acquires an image, thereby facilitating subsequent processing operations.

The working process of a single camera in the image acquisition subsystem

In the case of detecting glass edge defects, the position and orientation of glass placed on the industrial production line are often arbitrary.

As an embodiment, as shown in the figure, the position of the camera is fixed, the center of its imaging range is kept horizontal with the industrial production line, and the projection of the clear imaging plane P of the camera from directly above the industrial production line at least partially covers the industrial production line. During the movement of the industrial line, the position of the camera remains unchanged.

The glass to be tested is placed on the industrial production line in any orientation. In this case, the glass edge is often not parallel to the camera's image. In the glass edge image captured by the camera, the image at the intersection of the glass edge and the clear imaging plane P of the camera is clear, and other parts of the image other than the intersection point are not clear.

As an optional implementation manner, the processor controls the shooting of the camera according to the real-time position of the glass to be measured obtained by the position learning subsystem. When the front point of the glass to be tested is about to enter the camera shooting area or has entered the camera shooting area, the processor controls the camera to start shooting.

In order to ensure that during the movement of the glass to be tested, every point on the edge of the glass can intersect with a certain clear imaging plane P of the image acquisition subsystem 32, as an optional implementation, it can be set at the edge of the industrial production line The limit mechanism prevents the edge of the glass to be tested from exceeding the width of the industrial production line; as another optional implementation, an alarm device/function can be introduced: the front camera 312 judges that the shape and position information of the glass to be tested exceeds the industrial production line. If the width of the line is within the range, an alarm will be issued.

Take multiple pictures and perform image fusion

The camera has a depth of field, and the sharp parts in the photo taken by the camera have a certain range.

In order to obtain a complete image of the glass edge, the camera continuously takes multiple photos at a specific moment, and the clear parts in each image can be stitched together to form a complete glass edge image.

Denote the horizontal center point of the clear imaging area in the i-th frame image as _xi .

Taking _xi as the center, the range W _x and W _y of the image is taken horizontally and vertically. where x _i , W _x and W _y are all in pixels. The clear range in the image captured by the camera is: [x _i -W _x /2, x _i +W _x /2].

The representation of each parameter in the following formula includes: image distance: u, object distance: v, belt moving speed: V, the distance that the object moves perpendicular to the optical axis of the camera: S _v , the distance that the image moves perpendicular to the optical axis of the camera: S _u , the abscissa of the image on the sensor plane: x (corresponding to the pixel horizontal position of the point on the picture), the sensor pixel size: d, the glass thickness: T, the lens focal length: f, the sensor tilt angle: θ, the reference value x ₀ and u ₀ , given the horizontal coordinate x ₀ on the sensor, the image distance of the corresponding lens is u ₀ .

In the method of calculating the clear part of the image through the image gradient peak points, it is necessary to calculate the area of the clear range in the image. The moving speed V of the belt and the angle α of the optical axis of the camera relative to the belt; the exposure interval t of the two frames of images.

In the method of calculating the clear part of the image through the image gradient peak point,

W _x =S _u /cos(θ)/d, W _y =(u*T)/(v*d)

Wherein, S _u =u*S _v /v, v=1/(1/f–1/u), u=u ₀ +Δu; S _v =V*sin(α)*t; Δu=(xi _i -x ₀ )*sin(θ) The camera interval is t.

In the algorithm that obtains the sharp part of the image by calculating the intersection position,

W _x =S _u /cos(θ)/d and W _y =(u*T)/(v*d)

Wherein, S _u =u*S _v /v, v=1/(1/f–1/u), u=u ₀ +Δu; S _v =V*sin(α)*t

In this method, the rear camera selects an appropriate time to take a picture according to the glass edge position information and the encoder reading provided by the front camera.

The processor controls the photographing work of the image acquisition subsystem according to the known relative position between the product to be detected and the camera. The timing of the clear position x _i of the image captured by the camera is specifically calculated by the following method:

Assuming that the distance between the encoders for the two camera shots before and after is C _i-1 +C _pb and C _i +C _pb , the change of the glass position for the two shots is (C _i -C _i-1 )*S _c .

Because the camera sensor is inclined at an angle θ, the pixel size in the x-axis direction of the image plane is d*Cos(θ), and the pixel size in the y-axis direction of the image plane is still d.

The distance that the object moves in the direction perpendicular to the optical axis of the camera S _v =(C _i -C _i-1 )*S _c *Sin(α).

Then, according to the position information (X _i , Y _i +S _pb ) and D of the edge points provided by the front camera, the object distance v=(D+X _i )/sin(α) of the camera can be obtained. The image distance u can also be obtained according to the lens focal length formula.

Further, x _i =(uu ₀ )/sin(θ)+x ₀ is obtained.

The controller controls the camera to shoot according to the above calculation results and the information obtained by the position knowing subsystem, so that the images captured by the camera twice at the interval time/distance can meet the requirements of image stitching.

By fusing multiple clear images corresponding to different positions of the glass edge, a complete glass edge image in the shooting range of a single camera is obtained.

If the current glass moving direction is from right to left: the [x-D/2, x+D/2] image of the current frame is captured and placed on the right side of the output image.

If the current glass moving direction is from left to right: the [x-D/2, x+D/2] image of the current frame is captured and placed on the left side of the output image.

Repeat the previous two steps until a complete clear image is obtained.

Sensor/Lens Tilt

When detecting surface defects of objects, the object to be tested can be placed on the industrial production line in any direction. The image acquisition and detection system of the present invention collects the image of the object surface by obtaining a clear image at the overlap between the object surface and the clear imaging plane P of the camera. Image acquisition devices, including cameras, video cameras, scanners, etc., all have a certain length of the clear imaging plane P. The object surface defect detection system requires a large depth of field range of the camera, so as to cover a wide enough area on the industrial production line.

In order to obtain a larger depth of field range, the method of replacing the lens and/or the image sensor can be used. But a larger depth of field range means more cost.

As an optional implementation, the image acquisition subsystem in the industrial inspection system adopts a camera with an offset configuration.

As shown in FIG. 5 , the camera with the offset configuration includes a housing 501 , a lens lens 502 and an image sensor 503 , and the image sensor is connected to a mounting shaft 504 , which is rotatably mounted to the housing 501 . The part of the installation shaft is exposed from the upper part of the casing 501 , and the rotary knob 505 is fixedly connected with the part of the installation shaft exposed from the casing. As an optional implementation manner, an angle code wheel 506 is provided around the rotary dial, which is used to indicate the rotation angle of the image sensor. The lens 502 defines a main optical axis, the image sensor 503 defines a sensing plane, and a line passing through the mounting axis 504 and perpendicular to the main optical axis of the lens forms an included angle α with the sensing plane. The rotary knob 505 is operably rotated, so that the installation shaft 504 is rotated, and the installation shaft 504 drives the image sensor 503 to rotate. Through the angle code wheel 506 arranged around the rotary dial, the current tilt angle of the image sensor can be known intuitively.

As an alternative embodiment, the lens 502 may also be rotated, thereby changing the tilt relationship between the image sensor and the lens.

Taking the image sensor tilt as an example, the focal length range of the sensor is expressed by the following formula:

[(u+sin(α)*x)/(u+sin(α)*x-1),u/(u-1)]

Among them, u is the image distance, α is the rotation angle of the image sensor, x is the normalized size of the image sensor in the radial direction of the rotation axis corresponding to the focal length (x=X/f, where X is the real size), sin(α)*x is the projection distance of the sensor in the direction of the optical axis.

It can be seen from the formula that, corresponding to the given x and u, the larger the α, the smaller the (u+sin(α)*x)/(u+sin(α)*x-1), which means the greater the depth of field. big. When the range of the α angle is between 5° and 65° (it can also be expressed as the range of the angle formed by the intersection of the main optical axis of the lens and the sensing plane between 25° and 85°), a better performance can be obtained. Imaging effect.

According to the imaging principle of the lens, a camera with an offset configuration has a longer length of the clear imaging plane P than an ordinary camera using the same lens and image sensor, which also corresponds to a larger depth of field range.

The longer length and depth of field of the clear imaging plane P enable the imaging range of the camera with offset configuration to cover as many areas as possible on the industrial production line. Since the position of the glass to be tested on the industrial production line is uncertain, using a camera with an offset configuration can ensure that any point on the industrial production line can be moved as much as possible under the same hardware cost. Coincidence with a certain point on the clear imaging plane P of the camera at a certain moment also ensures that any position of the glass edge can be clearly imaged in the camera with the offset configuration at a certain moment.

At the same time, with a larger depth of field range, more clear images can be obtained when shooting surface images of objects with a certain depth. For example, when shooting glass with a curvature, a camera with an offset configuration can shoot a clear macro image of the entire glass surface at one time under a certain shooting angle. However, traditional macro cameras, due to the relatively small depth of field, may not be able to obtain clear images of different positions of the curved glass at the same time.

camera with offset configuration camera with offset configuration camera with offset configuration camera with offset configuration

Shooting with multiple cameras

Objects often have multiple surfaces, and there may be mutual occlusion between multiple surfaces of an object or different positions on the surface. At this time, a single camera cannot acquire images of multiple surfaces of an object at the same time.

The image acquisition subsystem 32 is used to acquire a complete image of the glass edge. Image acquisition subsystem 32 includes a plurality of cameras. Multiple cameras can be set on the same level. Usually, multiple cameras are distributed on the periphery of the industrial production line, and are arranged on both sides of the transportation line of the industrial production line. The clear imaging planes P of any two cameras do not overlap.

As shown in FIG. 6A, as an optional implementation, multiple identical cameras are evenly distributed around a certain point (the point is called the center point) on the center line of the width direction of the industrial production line, and any two adjacent cameras are connected to The included angles formed by the lines connecting the center points are consistent, and it can also be described that the multiple cameras are evenly distributed on the circumference of a circle with the center point as the center. In this way, each camera is center-symmetrical according to the center point. The center-symmetric relationship makes the installation and replacement of the entire image acquisition subsystem 32 easier and more convenient.

As an improvement of the above-mentioned embodiment, the number of the plurality of cameras is an even number. An even number of cameras are evenly distributed on both sides of the industrial production line transportation line. At this time, the cameras are not only centrally symmetric, but also axially symmetric. The setting of an even number of cameras further reduces the difficulty of assembling and replacing the image acquisition subsystem.

Using multiple cameras to shoot at the same time, it is possible to obtain images of different edges of the glass at the same time. If the number of cameras used is too small, it cannot meet the requirements of acquiring images of different edges of the glass. If too many cameras are used, the installation method of these units relative to the industrial production line will become complicated, and the corresponding cost will also increase due to the increase of the number.

Each camera has a certain shooting angle β when shooting the glass to be tested at different positions, and the camera can work normally only when β>0°. When multiple cameras are used, the shooting angle β>0° of each camera is required when shooting. Considering the case where multiple cameras are evenly distributed, the relationship between the number of cameras n and the limit shooting angle β _max is expressed by the following formula:

β _max =90°-180°/n

The camera can work properly only when β _max >0°. At the same time, since n must be an integer, it can be known from this formula that n ≥ 3, that is, at least 3 cameras are required to meet the shooting requirements. At this time, at least two cameras are located on the same side of the transport line.

β _max can also be regarded as the incident angle of the camera. The larger the β _max is, the more perpendicular the camera’s shooting direction is to the surface of the object, the better the shooting effect.

As can be seen from the table below, the larger the n, the larger the β _max and the higher the average image quality. That is to say, the more the number of cameras, the better the imaging effect.

Number of cameras n β_max Angle improvement 3 30 - 4 45 50% 5 54 25% 6 60 11% 7 64.3 7%

But more cameras means higher cost and more complicated assembly difficulty.

According to the results of many experiments, the absolute value of the imaging quality, the improvement value of the imaging quality and the hardware cost of the whole system when different numbers of cameras are used are comprehensively considered. The optimal state b of the whole system is expressed by the following formula:

b=8n-n ²

It can be seen from this formula that when n=4, b takes the maximum value.

When n is 4, the camera can be set on both sides of the industrial production line to ensure that it is on the same level as the glass to be tested. Moreover, since the distance between each camera disposed on both sides of the industrial production line and the industrial production line is the same, the difficulty of assembly can be reduced.

With 4 cameras, the angle is improved by 50% compared to 3 cameras. According to the experimental results, when the number of cameras in the image acquisition subsystem is from 3 to 4, the improvement of image quality is the most.

Where to install the camera

As shown in Fig. 6A, taking three cameras as an example, Figs. 6B-6D show the relationship between the clear imaging surfaces of each camera in different image acquisition subsystems.

As an embodiment, the main optical axes of the three cameras are parallel to the same plane. Such an arrangement can prevent the distortion of the captured picture due to the tilt of the camera in the longitudinal direction.

The clear imaging plane P of each camera must satisfy: in the process of the industrial production line driving the target object and the camera to carry out relative motion, every point on the surface of the target object can intersect with a certain clear imaging plane P, that is: with The plane perpendicular to the clear imaging plane P is the projection plane (the transport plane in this embodiment can be used as the projection plane), and the projection of the clear imaging plane P on the projection plane is approximately regarded as a projection line segment, then any point on the target surface is The motion trajectories on the projection plane all have intersections with the projection line segments of the clear imaging plane P.

The target is placed on the industrial production line, and the edge of the target will not exceed the edge of the industrial production line when viewed from the top. This setting can ensure that no matter what direction and position the target is placed on the industrial production line, the edge of the target will not exceed the plane of the industrial production line, that is, there will be no objects on the surface of the target that cannot be imaged at all. (for example, the part beyond the plane of the industrial production line cannot be imaged at all).

In addition, regardless of human eye observation or device imaging, there will be a problem of front and rear occlusion of the observed object itself. In the scene of shooting glass edge images, as shown in Figures 6B-6C, when the glass is placed in a specific position, it will be blocked by the front and rear, so that the complete image of the glass edge to be tested cannot be obtained. As shown in FIG. 6B , in this scenario, the image acquisition subsystem cannot acquire the image of the right edge of the rectangular glass 39 to be measured; in the scenario of FIG. 6C , the image acquisition subsystem cannot acquire the image of the left edge of the rectangular glass 39 to be measured.

According to multiple experiments, as shown in Figure 6D, in the case where a closed area is formed between the line segments of the multiple clear imaging planes P on the projection plane, as long as the inspected product transported on the industrial production line has a good quality on the transport plane Any point in the projection pattern can pass through the closed area, that is, the product to be inspected can completely pass through the closed area during transportation, so a complete and clear image of the product to be inspected can be obtained.

Multiple cameras that meet the above conditions can obtain a complete image of the edge of the glass, regardless of the orientation and position of the glass placed on the industrial production line.

The best camera choice

The positions of the plurality of cameras are fixed, that is, the shooting angles of the respective cameras are fixed. Since the edges of the glass may be occluded, and the glass is placed on the industrial production line at any angle, usually, not every camera can completely capture a certain or a certain edge of the glass image.

After obtaining the glass edge image and the corresponding position information through the position learning subsystem, the glass edge image is processed to obtain the normal vector anywhere on the glass edge image. The position pointed to by the normal vector is the best shooting angle for this point.

If the glass has multiple edges, the average value of the normal vectors on each glass edge is calculated respectively, and the best shooting angle is determined according to the average value of the normal vectors corresponding to each edge to select the best camera.

If the shape of the glass edge is a closed curve, the normal vectors at the fixed intervals of the glass edge can be sorted according to the angle between them and a certain reference direction, and the best camera can be selected according to the angle of the normal vector there.

As an optional implementation manner, the image of the glass edge is a closed figure, and the normal vector of each point on the closed figure is obtained. The angular ranges of the normal vectors corresponding to each camera can be the same or different, but the sum of the angular ranges of the normal vectors corresponding to all cameras should not be less than 360°, to ensure that there is at least one angular range on any glass edge image. corresponding camera. When collecting the glass edge image, the image collected by the camera corresponding to the normal vector at the glass edge is the optimal image.

As an optional embodiment, a plurality of cameras are evenly distributed around a certain point on the center line in the width direction of the industrial production line (this point is called a center point), and a clip formed by a line connecting two adjacent cameras and the center point The angles remain the same, which can also be described as the multiple cameras are evenly distributed on the circumference of a circle with the center point as the center. In this way, each camera is center-symmetrical according to the center point. In the case of including n cameras, each camera selects the corresponding best normal vector in the range of 360°/n. Such a setting can make the angle range of the normal vector corresponding to each camera the same, that is, the workload of each camera is the same, which is beneficial to the stable operation of the image acquisition subsystem. As shown in Fig. 3B, as an embodiment, the industrial inspection system based on image acquisition adopts the center symmetrical arrangement of 4 cameras, which are evenly distributed on both sides of the industrial production line transportation line, and the clamp between the center axes of the two cameras The angle is 90°.

The corresponding camera is selected according to the normal vector of the glass edge, and the glass edge image with the best imaging quality can be obtained while controlling the working time of the camera to reduce the workload of the camera.

Glass edge defect judgment

Figures 7A-7B show the glass edge images taken when the edge of the glass to be tested has flaws. For different types of glass, the form of flaws can be different.

As an example, in this embodiment, the edge of the glass to be tested should be covered with frosting under normal circumstances. At this time, the main types of glass edge flaws include "uncovered matte", and the flawed part produces a specular reflection. The light and dark of the image obtained by the camera shooting the uncovered frosted flaw depend on the angle corresponding to the specular surface and the incident direction of the light source.

If there is no light reflection relationship between the light source, the flawed part and the camera, at this time, the non- flawed part will have diffuse reflection, and part of the light of the light source will be reflected to the camera; the flawed part will have specular reflection, because the direction of the reflection angle is not aligned. The camera, the light from the light source enters the camera without specular reflection. As shown in FIG. 7A , in this case, the visual effect of the defective part is darker than that of the non-defective part, so the glass of the non-frosted defective part appears black.

Figure 7B shows an image of another case where the edge of the glass to be tested has flaws. At this time, a geometric reflection relationship is formed between the light source light, the flawed part and the camera, and all the light source light is reflected into the camera. Non-blemish parts are still diffuse, reflecting a portion of the light from the light source to the camera. In this case, the visual effect of the defective part is brighter than the visual effect of the non-defective part. Therefore, with this type of light source, there may be "highlight" and "darkness" in the "uncovered frosted" flawed part.

as an optional solution. When judging glass edge defects, two different judgment thresholds are set, including dark threshold and bright threshold. When it is judged that the brightness value of a certain part of the glass edge is less than the dark threshold, it is judged to be a defect; or when it is judged that the glass edge is a defect When the brightness value of a certain place is greater than the bright threshold, it is also judged that the place is a defect. as an alternative solution. First calculate the average brightness value of the entire glass edge. Then the brightness values at different positions of the glass edge are subtracted from the average brightness value, and the absolute value of the subtracted result is obtained. The absolute value is judged: if the absolute value is greater than a certain threshold, the brightness of the position is too bright or too dark, that is to say, the position is a specular reflection position. At this time, it is judged that there is a defect in the position. The brightness value mentioned here includes not only the brightness value in the color image, but also the gray value in the black and white image.

light source

When the above judgment method is used to judge the defect, if the difference between the brightness of the defect and the brightness of the non-defect is too small, the selection of the threshold value is difficult, and the judgment of the defect is not accurate enough.

In order to solve this problem, as an optional implementation, referring to FIG. 8A , a strip light source 81 is arranged above the clear imaging plane P of the camera. From a top view, each strip light source can cover the clear imaging plane P of one or more image acquisition units in the image acquisition subsystem 32 . The strip light sources may include point light sources or surface light sources that are uniformly distributed on the strip light sources. This setting can ensure that when the image acquisition subsystem collects images, the clear parts of the image can be illuminated by the light source of the same brightness and angle, so that the absolute difference between the brightness of each defect and the non-defect can be achieved. Increasing the value improves the brightness contrast between flaws and non-blemishes, allowing for more accurate detection of flaws. On the premise of not interfering with the movement of objects on the industrial production line, the strip light source can be as close as possible to the industrial production line to reduce the loss of brightness.

There may also be flaws where the edge of the frosted glass meets the glass surface, which is called a "blowout".

As another optional implementation manner, the light emitted by the light source is parallel light. The parallel light covers the clear imaging plane P of the camera. The light source can be placed next to the camera, on the same side of the industrial line as the camera. It can also be located on both sides of the industrial production line separately from the camera. With such a light source setting, it is possible to irradiate the place where the edge of the glass meets the surface of the glass, so that the "explosion" of the glass can be detected, and the application range of the online detection system is improved.

As another optional embodiment, referring to FIG. 8B , the light source 82 is a ring-shaped surface light source, and the shape of the surface light source is similar to the side surface of a cylinder or the side surface of a rectangular parallelepiped. The light source surrounds the glass to be tested, and the entire inner wall of the light source emits uniform light. Such a light source setting method can ensure that a specular reflection relationship can always be formed between the light emitted from the inner wall of the light source, the defect on the edge of the glass, and a certain camera in the image acquisition subsystem. At this time, no matter what direction the glass to be tested is placed on the industrial production line, the flawed part of the glass edge can always reflect the light from somewhere on the light source into a camera in the image acquisition subsystem through specular reflection, so that the The visual effect of the image of the flawed part of the glass edge can always be highlighted.

As an optional embodiment, the light source shape is a ring-shaped surface light source plus a cover surface light source. Compared with the case where the light source structure only includes the ring light source, the detection effect of "explosion edge" can be improved.

When setting the light source, the lower edge of the light-emitting part of the light source must at least be consistent with the position of the lower end of the glass edge or lower than the lower end of the glass edge to ensure the effect of the light source. But at the same time, such a light source has the potential to hinder the movement of glass on industrial production lines.

As an optional embodiment, as shown in FIG. 8C , the side view of the industrial production line 30 is in the shape of a small upper and a lower large trapezoid, and the industrial production line 30 has a highest plane during operation. Ring lights or capped ring lights are arranged around the highest plane of an industrial line. The position of the lower end of the ring light source 83 may be flush with the surface where the highest plane of the industrial production line 30 is located or slightly lower than the surface where the highest plane of the industrial production line is located. At the same time, the lower end of the ring light source keeps a certain distance from the surface of the industrial production line 30 , and the distance is greater than the maximum thickness of the glass 39 to be tested. When the glass to be tested 39 is transported to the highest plane of the industrial production line, the ring light source or the covered ring light source surrounds the glass to be tested. Since the distance between the lower end of the ring light source 83 and the industrial production line 30 is greater than the maximum thickness of the glass, it can ensure that the glass does not need to change the position of the ring light source when moving on the industrial production line, and the movement of the glass will not be interfered by the light source. It can also ensure that the lower edge of the light-emitting part of the light source is flush or lower than the lower end of the glass edge, thereby ensuring the integrity of the incident light angle of the light source.

As shown in FIG. 8D , as another optional implementation manner, the ring light source 84 can move up and down. The upper end of the ring light source is provided with a telescopic rod 841 that can move up and down, and the telescopic rod can be driven by an air cylinder, hydraulic pressure or motor. When the glass to be tested on the industrial production line 30 moves below the light source 84, the movement of the industrial production line stops, at this time, the telescopic rod 841 descends so that the lower edge of the light-emitting part of the light source is flush with the lower end of the glass to be tested 39. After completing the acquisition of the glass edge image, the telescopic rod 841 rises, and the industrial production line resumes motion. Correspondingly, the industrial production line 30 can be driven by a servo motor, and a fixed area for placing the glass to be tested 39 is set on the industrial production line. Each operation of the servo motor moves the industrial line 30 a fixed distance. Such an arrangement can make the light source 84 rise and fall at the same time interval, thereby simplifying the procedure and reducing the error probability. At the same time, a pressure feedback device or a distance measuring device may be provided on the light source to ensure that the glass to be tested 39 will not be crushed when the light source descends.

Since the light source itself is opaque, in order to acquire an image of the glass to be tested, the image acquisition subsystem 32 usually needs to be installed between the light source and the glass to be tested. At the same time, since the camera in the image acquisition system 32 is usually opaque, the camera often blocks the light of a certain part of the light source. If there are multiple cameras, and a reflection relationship is formed between some two cameras and the flaws on the glass edge, the resulting image of the flawed part of the glass edge will appear "dark" again.

In order to solve the above problem, as an optional solution, a light source is provided on the camera 321 . The light source can be arranged on the side of the camera lens, and the light source can be arranged in the form of a surface light source, so that the angle of the outgoing light can at least compensate for the light of the ring light source blocked by the camera. The light source can also be annular and installed around the camera lens. Such an arrangement can further ensure that the light emitted by the light source can compensate for the light blocked by the camera.

Defect detection

After obtaining a clear image of the edge of the glass, processor 33 detects imperfections from the image.

Defect detection method including front camera

1. Defect detection method without image stitching

The flaw detection of image stitching is not performed, and flaw detection is performed on the clear image obtained by the image acquisition subsystem at each shooting moment, so as to judge whether the image at the edge of the glass obtained at the shooting moment is flawed or not. At the same time, since the detection system including the front camera 312 can directly obtain the position of the clear image at the edge of the glass at any time, when a defect is detected at the clear image of the glass edge at any time, it can also know the spatial position of the defect. . In this way, it is possible to know at the same time whether the glass edge has a flaw and the position of the glass edge where the flaw is located.

The specific judgment process is shown in FIG. 9A .

In step S911, a reference value of luminance or gradation is determined. The determination of the reference value, as an embodiment, may be the average value of the edge brightness or gray level of the previous piece of glass. As another embodiment, the average value of brightness or grayscale may be determined by flawless glass prior to testing.

In step S912, the absolute value of the difference between the average value of the brightness or gray level of the current picture and the reference value is calculated.

In step S913, it is determined whether the absolute value exceeds a reasonable error range. If the absolute value of the calculation result does not exceed a reasonable error range, it is considered that there is no defect at this position of the glass edge. If the absolute value of the calculation result exceeds a reasonable error range, it is considered that there is a defect at the position of the glass edge, and the process proceeds to step S914.

In step S914, the spatial information of the glass edge corresponding to the defective picture is obtained. The spatial information can be obtained by the position learning subsystem 31 .

In step S915, feedback is performed. The way of feedback can be alarmed by sound and light, or it can be fed back to the work computer of the quality inspector through a wired connection, or sent to an external mobile terminal through a wireless connection, such as smart watches and smart phones.

Directly determine whether the clear image obtained at any time is flawless, eliminating the need for image stitching steps, efficient and fast.

2. Defect detection method after image stitching

The defect detection method without image stitching, each detection is relatively independent. Therefore, the detection result is greatly affected by the previous steps. For example, the reference value depends on the brightness and grayscale information of the glass before detection, and there will be differences under the influence of different ambient lighting. Therefore, with the change of natural lighting, the accuracy of flaw detection will be affected. And the position information of the clear image at any time completely depends on the front camera 312 and the encoder 311. If the information obtained by the front camera 312 or the encoder 311 itself is deviated, the position information obtained will be deviated.

As another glass edge defect detection method, as shown in Figure 9B, the steps are as follows:

In step S921, the images are stitched to form a clear image of the complete glass edge.

In step S922, the average value of the brightness or gray level of the complete image obtained in step S921 is calculated. When the obtained picture is a color picture, the average value is the brightness average value; when the obtained picture is a black and white picture, the average value is the grayscale average value.

In step S923, according to a specific calculation sequence, the brightness or grayscale of each pixel/pixel group in the complete image obtained in step S921 is calculated, and compared with the average value of the brightness or grayscale obtained in step S921. The comparison method can be to calculate the absolute value of the difference between the brightness or gray level of the pixel and the average value. It can also be calculated by calculating the absolute value of the difference between the brightness or gray level of the pixel and the average value divided by the average value, and dividing the difference value by the average value can effectively eliminate the influence of lighting conditions on the judgment of defective parts. If the result of comparing the brightness or gray level of the pixel with the reference value is greater than a certain threshold, the pixel position is considered to be a defect position.

In step S924, according to the order of calculating the flaw obtained in step S923, the relative position of the flawed portion on the edge of the glass is obtained. At this time, the spatial information obtained by the position learning subsystem 31 can also be received, and integrated with the relative position obtained before.

In step S925, the relative position is fed back to the quality inspector. The way of feedback can be sent to the work computer of the quality inspector through a wired connection, or sent to an external mobile terminal, such as a smart watch or a smartphone, through a wireless connection.

Defect detection method without front camera 312

1. Defect detection method without image stitching

The flaw detection of image stitching is not performed, and flaw detection is performed on the clear image obtained by the image acquisition subsystem at each shooting moment, so as to judge whether the image at the edge of the glass obtained at the shooting moment is flawed or not. Since it is difficult to obtain the position of the clear image at any time in the flaw detection method that does not include the front camera 312 , such a detection method is usually used to determine whether the edge of the piece of glass to be tested is flawed.

Directly determine whether the clear image obtained at any time is flawless, eliminating the need for image stitching steps, efficient and fast.

2. Defect detection method after image stitching

Defect detection after image splicing can improve the detection accuracy on the one hand, and on the other hand, obtain the position information of the defective part by obtaining the positional relationship of the defective part compared with other parts of the object to be measured. With this method, the position of the defective part can be obtained without the position information provided by the position learning system.

The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above-mentioned embodiments do not limit the present invention in any form, and all technical solutions obtained by means of equivalent replacement or equivalent transformation fall within the protection scope of the present invention.

Claims (10)

1. An image acquisition method, characterized by:

the image acquisition method comprises the following steps:

driving a camera or/and a target object to make them move relatively so that a part to be shot of the target object can appear in a clear imaging plane of the camera in a time-sharing manner;

collecting an image of a target object;

processing an image captured by the camera to identify a portion of the camera where a sharp imaging plane coincides with the target object surface.

2. The image acquisition method according to claim 1, characterized in that:

the camera or/and the target are driven to rotate around a rotation axis, and the rotation axis is parallel to a clear imaging plane of the camera.

3. The image acquisition method according to claim 1, characterized in that:

and driving the camera or/and the target object to move along a straight line.

4. The image acquisition method according to claim 1, characterized in that processing the image taken by the camera comprises:

the peak value of the gradient values of the image parameters of a pixel/group of pixels in the image is calculated.

5. The image acquisition method according to claim 1, characterized in that processing the image taken by the camera comprises:

and denoising the image.

6. The image acquisition method according to claim 1, further comprising:

acquiring position information of the contour of a target object;

acquiring position information of a clear imaging surface of a camera;

and calculating the coincidence position of the surface of the target object and the clear imaging surface of the camera according to the position information of the contour of the target object and the position information of the clear imaging surface of the camera.

7. The image acquisition method according to claim 6, characterized in that:

the location information includes coordinate information.

8. The image acquisition method according to claim 1, further comprising:

emitting laser light, wherein the region formed by the laser light comprises a clear imaging surface of the camera;

and acquiring the position information of the laser spot in the image, and matching the position information of the laser spot with the image.

9. The image acquisition method according to claim 8, wherein acquiring laser spot position information in the image comprises:

filtering the image by adopting a channel corresponding to the laser color in the multi-color camera to obtain a filtered image;

and acquiring the position information of the laser spots in the filtered image.

10. The image acquisition method according to claim 1, further comprising:

acquiring a plurality of images of the target object;

and fusing the parts of the plurality of images, which are overlapped by the clear imaging surfaces of the cameras and the surface of the target object.