CN111060004B

CN111060004B - Method for determining position of moving object in machine vision, position determining device and position determining system

Info

Publication number: CN111060004B
Application number: CN201911183667.9A
Authority: CN
Inventors: 林卿
Original assignee: Wuhan Lingyun Photoelectronic System Co ltd
Current assignee: Wuhan Lingyun Photoelectronic System Co ltd
Priority date: 2019-11-27
Filing date: 2019-11-27
Publication date: 2022-07-12
Anticipated expiration: 2039-11-27
Also published as: CN111060004A

Abstract

The invention discloses a method for determining the position of a moving object in machine vision, a position determining device and a position determining system. The method is to shoot the first image of the moving object at the first set position, and to determine the actual position of the moving object by using the moving speed of the moving object and during shooting. The method can accurately calculate the difference value between the preset shooting coordinate and the real shooting coordinate caused by the camera shutter delay, and provides a reference value for subsequent accurate positioning calculation.

Description

Method for determining position of moving object in machine vision, position determining device and position determining system

Technical Field

The invention belongs to a machine vision application technology, and particularly relates to a technology for determining object position information by using a machine vision technology.

Background

During the processing and braking process of the parts, the position information of the parts can be determined by adopting machine vision so as to accurately position the parts. CN 109448054A discloses a step-by-step target positioning method, application, device and system based on visual fusion, the method collects a first image containing target feature points to realize coarse positioning of the target feature points, collects a second image containing target feature point information according to coordinate information contained in the first image, carries out fine positioning, obtains coordinate information contained in the second image, and obtains coordinate transformation quantity of the target feature points based on the same coordinate system according to transformation relations among different coordinate systems, thereby positioning the target feature points. The acquired images are all image coordinate information of static objects and are not suitable for determining the positions of moving objects.

CN 109300829 a discloses a crystalline silicon photovoltaic solar cell conveying and positioning device and method based on machine vision, which discloses a calibration method using machine vision, comprising: placing a workpiece carrying a marker on a punching belt, and enabling the workpiece to be in the visual field of a front station camera, triggering the front station camera to take a picture, identifying the marker, and calculating the position of the current marker in a unified coordinate system in a visual system; the lower computer sends a plurality of pulses to the motor to drive the punching belt to convey the workpiece to the visual field of the rear station camera; after triggering, photographing by a station camera, identifying a marker, calculating the position of the current marker in a unified coordinate system in a vision system, simultaneously recording the number of motor pulses used for the transmission, and then taking away a workpiece; repeating the step for a plurality of times, and then calculating the functional relation between the number of the motor pulses and the transmission distance of the perforated belt; in both shooting operations, the subject is in a stationary state. And is also not suitable for determining the position of a moving object.

The conventional method is also shown in fig. 1, and comprises a camera 101, a shutter 102, a trigger control 103, and a workpiece 104. The workpiece 104 can move on the X-Y horizontal plane, the camera starts shooting after the workpiece 104 is moved to the shooting position and waits for the workpiece to completely stop moving, and the center coordinate of the shot picture can be obtained through the coordinate calculation of the X-Y moving platform. And subsequently, the workpiece can be accurately moved to the specified position by calculating the absolute values of the shooting position and the processing position and the picture center coordinate. The method is a machine vision auxiliary position positioning method commonly used in automatic processing. However, the method has a problem of low working efficiency, which is mainly caused by the fact that the workpiece 104 needs to be stopped at a photographing position and wait for complete standstill, if the workpiece is not completely stopped, photographing is started, which causes the central position of the picture to be misaligned with the preset photographing position of the X-Y moving platform (as shown by a dotted line 105, the central position of actual photographing and a dotted line 106, the central position of ideal photographing), which causes a large error in subsequent calculation, and even when the speed is fast, the workpiece 104 moves out of the field of view of the camera, and a correct picture cannot be taken.

In order to solve the above problems, a conventional method is proposed, as shown in fig. 2, which is composed of a camera 201, a shutter 202, a trigger control 203, a workpiece 204, and a position sensor 207. A position sensor 207 is added at the position where the workpiece 204 is to be photographed, and a signal for photographing trigger is given to the camera shutter 202 when the workpiece 204 passes the sensor rapidly, and the workpiece does not need to stay at the photographing position. By adopting the method, the problem that the triggering control by adopting software in the traditional method 1 cannot be matched with the high-speed movement speed is solved. Meanwhile, with the electronic technology, modern industrial COMS cameras largely adopt all electronic shutters to replace the traditional rolling shutter and the application of high-brightness illumination LEDs to improve the light sensitivity, so that the exposure time of the industrial cameras is greatly shortened to a few milliseconds, the problems of smear and blurring in the process of shooting moving objects can be avoided, and the motion position error caused by the exposure time is also reduced to a few micrometers so as to reach the extent that the mechanical processing is negligible. However, this method still has technical defects in practice, and the biggest problem is that there is a time delay from the receipt of the trigger shooting signal to the opening of the shutter, which causes a large difference between the ideal shooting position and the actual shooting position (ideal shooting position 206, actual shooting position 205). Particularly cmos cameras that employ electronic shutters almost exclusively in industrial automation applications. The shutter is characterized in that the frequency of opening and closing the shutter can be very high, such as thousands or even ten thousand times per second, but the delay from the receiving of the actuating signal to the opening of the shutter is extremely serious, and is often hundreds of milliseconds. The electronic shutter camera is more intuitive, and the traditional digital camera and the mobile phone almost adopt the electronic shutter camera, so that the wonderful moment is often missed when the capture of a fast moving object is not forcefully pressed down a shooting key. When the workpiece moves rapidly in automatic processing, the center of the actually shot picture and the preset shooting position form a remarkable distance difference, and even the shot workpiece moves out of the shooting view. For some machine vision applications, such as color recognition, quality judgment, contour recognition, character recognition, license plate identification, face recognition, etc., the problem that the absolute value of the position is not concerned is not a problem, but the influence caused by the distance difference in positioning application cannot be ignored. Another technical drawback is that the actual motion is usually a linear interpolation motion in which the X-Y axes move simultaneously, and in this way, there is a slight difference between the actual positioning coordinates and the preset coordinates, and when the speed is higher, the difference is larger, and the conventional position sensor or photoelectric switch mode is adopted, the problem that the X, Y axes cannot trigger the switches simultaneously may occur, which may cause failure in starting the camera.

Disclosure of Invention

The invention aims to provide a method, a device and a system for acquiring images of moving objects and determining position information of the moving objects by using machine vision.

One of the technical schemes for realizing the purposes of the invention is as follows: the method for determining the position of a moving object in machine vision is that,

shooting a first image of a moving object at a first set position, and determining the actual position of the moving object by using the moving speed of the moving object and the shooting time.

The method compensates the coordinate information of the ideal (target) position when the moving object reaches the set position by utilizing the moving speed of the moving object and shooting, obtains the actual position information of the moving object during shooting, and realizes the accurate positioning of the position of the moving object. The speed of the known moving object may be a known set moving speed, or may be a speed of movement of the object detected other than by using a machine recognition method.

Shooting a first image of a moving object at a first set position, shooting a second image of the moving object at a second set position, and determining the actual position of the moving object by using the moving speed of the moving object, the time interval of two times of shooting and the time for shooting.

The method not only compensates the fixed ideal (target) position coordinate information by utilizing the moving speed of the moving object and the shooting time, but also compensates by combining the interval time of two times of shooting, is suitable for the situation that the speed of the moving object is unknown, and identifies the detected moving speed of the object through machine vision. The accurate positioning of the position of the object with plane motion is realized.

Further preferred methods are: the first image and the second image are shot in the same visual field range. The shooting is carried out in the same visual field, the time interval of the two times of shooting is reduced, on one hand, the accuracy of determining the position information can be improved, and on the other hand, the position determination of a high-speed moving object is met.

Further preferred methods are: the shooting of the first image is as follows: when a moving object is sensed, the shooting equipment is triggered to shoot, and shooting time and the first shooting starting time are recorded.

Further preferred methods are: the shooting of the second image is: when the moving object is sensed, the shooting equipment is triggered to shoot, and the second shooting starting time is recorded.

Further preferred methods are: when the second image is shot, the counting device for recording the first shooting starting time is triggered to stop timing, and the timing stopping time is recorded.

The triggering mode is beneficial to setting of system equipment and determining of control accuracy.

Compared with the traditional method 1, the method can shoot without stopping the workpiece; compared with the traditional method 2, the method can accurately calculate the difference value between the preset shooting coordinate and the real shooting coordinate caused by the shutter delay of the camera, and provides a reference value for the subsequent accurate positioning calculation.

The second technical scheme for realizing the purpose of the invention is as follows: an apparatus for determining the position of a moving object in machine vision, comprising:

the moving object position sensing unit is used for sensing a moving object and outputting the sensed position information of the moving object;

the position information comparison unit is used for comparing the sensed position information of the moving object with the set position information and outputting a switching value signal;

the timing unit is used for receiving the switching value signal and recording time;

the shooting trigger unit of the shooting equipment is used for receiving the switching value signal and outputting a shooting starting signal;

and the storage unit is used for storing the position information.

The position sensing unit of the moving object in the unit has a storage unit which is a physical entity device, and other units can be realized in a physical entity device or a processor by adopting a software program.

The further preferred technical scheme is as follows: the timing unit comprises a shooting time counting unit and a two-time shooting interval timing unit.

The further preferred technical scheme is as follows: the set position information is information of one direction of plane coordinate information; the information of the other direction of the plane coordinate information is sensed by the moving object position sensing unit.

The further preferred technical scheme is as follows: it also includes a data processing unit for calculating the actual position information of the moving object shot.

The data processing unit can be an industrial personal computer, a PC (personal computer) or a CPU (central processing unit).

The third technical scheme for realizing the aim of the invention is as follows: a system for determining the position of a moving object in machine vision, comprising the above apparatus, and further comprising:

the shooting equipment starts shooting when receiving the shooting starting signal and outputs shooting ending information;

the object moving device realizes plane or linear motion of the object.

The system is an application scene, and the moving object can be a processed workpiece, moves on a moving platform, and can also be a moving object with power.

Drawings

Fig. 1 is a schematic view of prior art 1.

Figure 2 prior art 2 schematic.

FIG. 3 is a schematic view of the system of the apparatus of the present invention.

Detailed Description

The following detailed description is provided for the purpose of explaining the claimed embodiments of the present invention so that those skilled in the art can understand the claims. The scope of the invention is not limited to the following specific implementation configurations. It is intended that the scope of the invention be determined by those skilled in the art from the following detailed description, which includes claims that are directed to this invention.

The first embodiment is a device and a system for determining the actual position information of a moving object based on two-time shooting, the following device is a circuit principle structure, the specific implementation mode can be a digital circuit and an analog circuit, and the digital circuit can be an integrated chip and software implementation.

As shown in fig. 3, the register consists of register X1301, comparator a 302, raster scale X303, register X2304, comparator B305, raster scale Y306, latch a 307, latch B308, register Y1310, register Y2311, camera 312, timer B313, and timer a 314. The grating ruler is a moving object position information sensing device, the register is a storage unit, and the storage unit can be a storage unit only storing data information related to the invention and can also be a storage unit storing data information and other information of the invention; the timer is a timing unit, can be an independent timer or an integrated circuit; the comparator is a position information comparing unit, and can be an independent comparator circuit or an integrated circuit.

The two comparison input ends a and B of the comparator a 302 are respectively connected to the register X1301 and the raster ruler X303, and the outputs thereof are respectively connected to the latch trigger input end i of the register a 307, the operation input end a of the logic operator 309, the start timing input end s of the timer a314, and the start timing input end s of the timer B313.

Two comparison input ends a and B of the comparator B305 are respectively connected to the register X2304 and the raster ruler X303, and the outputs thereof are respectively connected to the latch trigger input end i of the register B308, the operation input end B of the logic operator 309, and the stop timing input end t of the timer B313. The output of the raster scale Y is connected to the data inputs a of both register a 307 and register B308. The input of register Y1310 is connected to the data output o of latch A307 and the input of register Y2311 is connected to the data output o of latch B308. The output o of the logic operator 309 is connected to the photographing start terminal s of the camera 312. And the shooting completion signal output end d of the camera is connected with the timing stopping input end t of the timer A.

The register X1301, the register X2304, the register Y1310 and the register Y2311 are devices for storing numerical values, and the specific implementation mode of the devices is irrelevant to the method.

The grating ruler X303 and the grating ruler Y306 are devices for measuring relative values or absolute values of positions, can convert linear position coordinates into digital values to be output, and can also be grating discs for converting the relative linear position coordinates into angles and rotation turns. The measurement directions of the grating ruler X303 and the grating ruler Y306 are mutually vertical, and the coordinates of any point on a plane can be accurately measured. The grating ruler X303 and the grating ruler Y306 can be arranged in a linear motor, a screw rod guide rail or a servo motor, and the specific measurement principle is independent of the method.

The comparator a 302 and the comparator B305 according to the present invention are a numerical absolute value comparing device, and the output terminal o outputs a high level when the values of the input terminal a and the input terminal B are equal, and outputs a low level when the values of the input terminal a and the input terminal B are not equal.

The camera 312 of the present invention is a general two-dimensional image pickup device having at least one shutter start input terminal s and one photographing completion output terminal d. Other resolutions, colors, target size, interfaces, pixel sizes, lens magnifications, etc. are not relevant to the method of the invention.

The latch a 307 and the latch B308 according to the present invention are a device for transferring a value from the input terminal a to the output terminal o at a specific time point. When the input latch trigger terminal l has a transition from low level to high level (at the time of rising edge), the value of the input terminal a is transmitted to the output terminal o, and the output terminal o remains unchanged at other times.

The logic operator 309 according to the invention is a binary operator which has the effect that a transition from low to high is output at the output o when the transition from low to high occurs at the instant (rising edge) of either of the inputs a and b.

The timer a314 and the timer B313 are timing devices having two input ports, wherein the input port s is a start timer and the input port t is a stop timer. Starting the timer when the input port s receives the instant (rising edge) of the transition from low level to high level; the timer stops counting when the input port t receives the instant (rising edge) of the transition from low level to high level. The timer B313 is used for measuring the interval Td between two times of shooting, and the timer a314 is used for measuring the start delay Tg of the camera during shooting.

The object moving device may be a workpiece conveying device such as a conveying device having an X-Y running track, a linear motion transfer device, or the like.

The data processing unit may be a CPU, and calculates the actual position coordinates of the workpiece using the following equation.

The method comprises the following implementation steps:

the photographed workpiece is fixed on an X-Y two-direction moving stage (e.g., a linear motor, a lead screw stage), the register X1 value is preset as the X-axis coordinate of the workpiece when the camera 312 photographs for the first time, and the register X2 value is preset as the X-axis coordinate of the workpiece when the camera photographs for the second time. The X-Y motion stage motion path is set so that the workpiece can remain in a straight line through the camera 312.

The set position information is information of one direction of the plane coordinate information; the information of the other direction of the plane coordinate information is sensed by the moving object position sensing unit. Since the actual motion path may not be completely straight or oblique (the interpolation of the straight motion inevitably has errors), the precise coordinates can be photographed only by presetting a direction (for example, the X-axis is preset in this embodiment, if the Y-axis is also suitable for the method of the present invention), and the actual position, that is, the actually detected position information, is obtained by the latch a 307 and the latch B308 in the other motion direction.

Certainly, we can completely set a (X, Y) coordinate position, when the workpiece moves to the position and is sensed, the camera can be triggered to shoot, and at this time, the moving object position sensing unit can be a position sensor, a two-dimensional code scanning device, and the like.

When the workpiece moves from the loading position to the first shooting position, the output port o of the comparator a 302 outputs a transition (rising edge) from low level to high level when the position output value of the grating ruler X303 is the same as the coordinate value prestored in the register X1301. The timer A314 and the timer B313 start to time after receiving the rising edge jump; the input end i of the latch A307 receives the rising edge jump and stores the position value of the raster ruler Y306 into a register Y1; the input end a of the logic operator 309 receives the rising edge jump, and the output end also outputs a jump (rising edge) from low level to high level, and the start input end s of the camera 312 receives the rising edge and starts shooting; after the camera 312 finishes shooting, it sends a completion signal to the stop timing input terminal t of the timer a314 and stops timing to obtain a time value Tg from the start of shooting to the completion of shooting.

The shooting time (shooting time) of the camera may also use parameters of the camera itself, such as the shooting time corresponding to the shutter speed of the camera when shooting. In which case timer a314 may not be used in the circuit configuration.

When the workpiece continues to move to the second photographing position, the position output value of the grating scale X303 is the same as the coordinate value prestored in the register X2304, and the output port o of the comparator B305 outputs a transition (rising edge) from low level to high level. A timing stopping input port t of the timer B313 stops timing after receiving rising edge jump to obtain a time interval Td between the first shooting and the second shooting, and a latch request input end i of the latch B308 stores the position value of the raster ruler Y306 into a register Y2311 after receiving the rising edge jump; the output terminal o of the logic operator 309 also outputs a transition (rising edge) from low level to high level after the input terminal b receives the rising edge transition; the camera 312 starts shooting again after the start input terminal s receives the rising edge signal.

The ideal coordinates obtained by the above steps are (X1, Y1) for the first shot and (X2, Y2) for the second shot. Because there is a time difference Tg between triggering the camera shutter and completing shooting and the shooting work in the process keeps moving continuously, there is a difference between the shooting preset coordinate and the actual shooting coordinate. Since the first shot and the second shot are generally short, they can be considered to be uniform motion at a speed V. Therefore, the actual shooting coordinates and the preset shooting coordinates have the following relationship:

RX1, RY1 are first actual shooting coordinates; RX2, RY2 are second actual shooting coordinates; x1 and Y1 are first preset shooting coordinates; x2 and Y2 are preset coordinates for the second shooting; vx is the velocity component in the X direction; v_yIs the velocity component in the Y direction. The time difference Td between the first shooting and the second shooting can be obtained by the timer B313 and thus the velocity components in the X and Y directions can be calculated as follows:

vx (X2-X1)/Td formula 3

Vy is (Y2-Y1)/Td formula 4

Substituting equations 3 and 4 into equations 1 and 2 yields the following equation:

RX1＝X1+Tg*(X2-X1)/Td

RX2＝X2+Tg*(X2-X1)/Td

RY1＝Y1+Tg*(Y2-Y1)/Td

RY2＝Y2+Tg*(Y2-Y1)/Td

the grating ruler X303 and the grating ruler Y306 belong to essential devices for a conventional linear motor, and output signals of the conventional linear motor can be directly utilized without additional increase; the angle and the number of turns of a grating disc encoder in the servo motor can be measured by the screw rod platform device driven by the servo motor to indirectly obtain the real position coordinate without additionally adding devices. Other comparators, registers, timers, logic calculators and other devices can be realized by idle units in a Programmable Logic Controller (PLC) in automatic control, and a conventional industrial camera generally has shutter input signals triggered externally and shooting completion feedback signals and only needs to connect the signals with functional units in the PLC. By presetting the X-axis shooting coordinate and latching the Y-axis coordinate, the problem that the X, Y axis coordinate cannot be equal to the preset coordinate at the same time and shooting cannot be started due to insufficient actual positioning precision in the linear interpolation motion can be effectively solved.

The system based on the device can be as follows: the camera adopts a Haekwondo MV-CA013-21UM type USB3.0 interface industrial camera which has external shutter input and finishes shooting and outputs I/O signals, a grating ruler is replaced by a loose A6 series MSMF042A1U2 servo motor and a grating encoder of a driver MBDLN25SE, the signal output of the encoder A, B, Z is connected with the input ports of R500, R501 and R502 of a KV-N60T small programmable controller of Keynes to perform position counting, and the programmable comparator and the storage space function in the encoder are utilized again to realize the content of the invention.

In the second embodiment, unlike the first embodiment, the speed of the movement of the workpiece is calculated by using the interval time between two times of photographing and the positions of the two times of photographing by the camera, and then the compensation calculation is performed. In this embodiment, the speed of the motion of the workpiece is already known, so that only one shot needs to be taken, and the specific method is as follows:

the moving workpiece moves to a calibration position, is sensed, and triggers a camera to shoot to obtain an image of the moving object; record the shooting time of the camera.

And calculating to obtain the actual position information of the moving workpiece during shooting by utilizing the moving speed of the moving object and the shooting time. The algorithm is as follows:

RX1＝X1+Tg*Vx

RY1＝Y1+Tg*Vy

the device based on the above method can reduce the number of the comparator B305, the timer B313, the latch B308 and the register X2304 compared with the first embodiment.

When the camera parameters are used, such as the photographing time corresponding to the shutter speed of the camera when photographing. The device comprises the following components: the system comprises a moving object position sensing unit, a position information comparison unit, a storage unit and a timing unit, wherein the timing unit can be stored photographing time information corresponding to the shutter speed of the camera, has no physical entity and can also be a shutter and control unit physical entity of the camera.

Claims

1. A method of determining the position of a moving object in machine vision, characterized by:

when the workpiece moves to a first shooting position, the position output value of the first grating ruler is the same as a first coordinate value prestored in the first register, and the output port o of the first comparator outputs a rising edge jump from a low level to a high level;

the first timer and the second timer start timing after receiving the jump of the rising edge;

after receiving the rising edge jump, an input end i of the first latch stores the position value of the second grating ruler into a third register;

the camera starting input end s starts shooting after receiving the rising edge jump output by the logic arithmetic unit;

sending a completion signal to a timing stop input end t of the first timer after the camera finishes shooting and stopping timing to obtain a time value Tg from the start of shooting to the completion of shooting of the camera;

when the workpiece continues to move to the second shooting position, the position output value of the first grating ruler is the same as a second coordinate value prestored in the second register, and the output port o of the second comparator outputs a rising edge jump from a low level to a high level;

the input port t of the second timer stops timing after receiving the rising edge jump to obtain a time interval Td between the first shooting and the second shooting;

a latch request input end i of the second latch stores a second raster position value into a fourth register after receiving rising edge jump;

the input end b of the logic arithmetic unit receives the rising edge jump and then the output end o outputs a rising edge jump signal from low level to high level; the camera starting input end s starts shooting again after receiving the rising edge jumping signal output by the logic arithmetic unit;

based on the obtained ideal coordinates of the workpiece shot for the first time and the ideal coordinates shot for the second time; the time for shooting and the time interval of two times of shooting determine the actual position of the moving object.

2. An apparatus for determining the position of a moving object in machine vision, comprising:

two comparison input ends of the first comparator are respectively connected with the first register and the first grating ruler, and the output ends of the first comparator are respectively connected with a latch trigger input end of the first latch, a first operation input end of the logic arithmetic unit, a starting timing input end of the first timer and a starting timing input end of the second timer;

two comparison input ends of a second comparator are respectively connected with the second register and the first grating ruler, and the output ends of the second comparator are respectively connected with a latch trigger input end of a second latch, a second operation input end of a logic arithmetic unit and a timing stopping input end of a second timer;

the output of the second grating scale is connected with the data input ends of the first latch and the second latch; the input of the third register is connected with the data output end of the first latch, and the input of the fourth register is connected with the data output end of the second latch; the output end of the logic arithmetic unit is connected with the camera photographing starting end; the shooting completion signal output end of the camera is connected with the timing stopping input end of the first timer;

the first register is preset as information of one direction in the workpiece plane coordinate information when the camera shoots for the first time, and the second register is preset as information of one direction in the workpiece plane coordinate information when the camera shoots for the second time;

the first timer is used for recording shooting time;

the second timer is used for recording the interval time between two times of shooting.

3. The apparatus for determining the position of a moving object in machine vision according to claim 2, further comprising a data processing unit for calculating actual position information of the shot of the moving object.

4. A system for determining the position of a moving object in machine vision, comprising the apparatus for determining the position of a moving object in machine vision of claim 2, further comprising:

the data processing unit is used for calculating the actual position information of the shooting of the moving object;

the object moving device realizes plane or linear motion of the object.