TWI806780B - System and method for detecting and compensating depth - Google Patents

Abstract

A system includes an object to be tested, multiple rotation modules, a depth sensing module and a computing module. The rotation modules rotate the object around an axis. The depth sensing module is arranged on one side of the object to obtain a rectangular depth image having a sampling direction and a rotation direction. The computing module applies a sliding window on the rectangular depth image, and the sliding window moves along the sampling direction to obtain multiple depth values. The computing module determines whether there is a deviation in the rectangular depth image according to the depth values, and compensates the deviation if so.

Description

Depth detection and compensation system and method

The present disclosure relates to a detection system and method for rotating a test object and scanning the depth of the side surface of the test object, especially to compensate for deviation caused by rotation.

In the steel mill, the thickness of the steel strip after cold rolling is relatively thin, which is prone to edge cracks or jagged edge defects. In order to avoid problems caused by the steel strip with edge defects entering the subsequent process, the steel strip must be inspected according to the inspection system. Whether the edge is flawed. One method is to use a camera to shoot the side of the steel coil after the steel strip is coiled into a steel coil, but because the radius of the steel coil is quite large, it is difficult for the camera to cover the entire range of the steel coil.

Embodiments of the present disclosure provide a depth detection and compensation system, including an object to be measured, a plurality of rotating modules, a depth sensing module, and a computing module. The rotating module is used to rotate the object to be tested around the axis, and the outline of the object to be tested is circular when viewed from the axis. The depth sensing module is arranged on one side of the object to be measured to obtain a rectangular depth image, wherein the rectangular depth image has a sampling direction and a rotation direction, and the scanning range of the depth sensing module is smaller than the diameter of a circle. The calculation module is connected to the rotation module and the depth sensing module in communication, and is used to apply the sliding window on the rectangular depth image, and the sliding window moves along the sampling direction to obtain multiple depth values. The calculation module is used to judge whether there is a deviation in the rectangular depth image according to the depth value, and to compensate the deviation if there is.

In some embodiments, the length of the sliding window in the sampling direction is greater than 1, and the length of the sliding window in the rotation direction is equal to the length of the rectangular depth image in the rotation direction. The computing module is used to execute a moving average algorithm according to the sliding window, and each position of the sliding window corresponds to a depth value.

In some embodiments, the calculation module is also used to calculate the slope between every two adjacent depth values, and judge whether the rectangular depth image has a linear deviation or a nonlinear deviation according to whether the slope is out of range.

In some embodiments, the calculation module is also used to execute a linear regression algorithm according to the depth value, and determine whether the rectangular depth image has a linear deviation or a nonlinear deviation according to the result of the linear regression algorithm.

In some embodiments, the object to be tested is a cylindrical steel coil.

From another point of view, the embodiments of the present disclosure provide a depth detection and compensation method suitable for computer systems. The depth detection and compensation method includes: rotating the object to be measured around the axis through a plurality of rotation modules, wherein the outline of the object to be measured viewed from the axis is a circle; obtaining a rectangular depth image through the depth sensing module, wherein the depth The sensing module is set on one side of the object to be tested. The rectangular depth image has a sampling direction and a rotation direction. The scanning range of the depth sensing module is smaller than the diameter of a circle; apply a sliding window to the rectangular depth image, and slide the window along the moving along the sampling direction to obtain multiple depth values; and judging whether there is a deviation in the rectangular depth image according to the depth value, and compensating for the deviation if there is.

In some embodiments, the length of the sliding window in the sampling direction is greater than 1, and the length of the sliding window in the rotation direction is equal to the length of the rectangular depth image in the rotation direction. The depth detection and compensation method further includes: performing a moving average algorithm according to the sliding window, and each position of the sliding window corresponds to one of the depth values.

In some embodiments, the depth detection and compensation method further includes: calculating a slope between two adjacent depth values, and judging whether the rectangular depth image has a linear deviation or a nonlinear deviation according to whether the slope is out of range.

In some embodiments, the depth detection and compensation method further includes: performing a linear regression algorithm according to the depth value, and judging whether the rectangular depth image has a linear deviation or a nonlinear deviation according to the result of the linear regression algorithm.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail together with the accompanying drawings.

The terms "first", "second" and the like used herein do not specifically refer to a sequence or sequence, but are only used to distinguish elements or operations described with the same technical terms.

FIG. 1 is a schematic diagram illustrating a detection system according to an embodiment. Referring to FIG. 1 , the depth detection and compensation system 100 includes an object under test 110 , a plurality of rotating modules 121 - 122 , a depth sensing module 130 and a computing module 140 . The object to be tested 110 is, for example, a cylindrical steel coil, but the present disclosure is not limited thereto. In other embodiments, the object to be tested 110 may also be cylindrical or disc-shaped, and the material of the object to be tested 110 may include any metal , organic matter, metal compounds and other materials. The rotation modules 121 - 122 may include rollers and motors for rotating the object under test 110 around the axis 150 .

The depth sensing module 130 may include a structured light module, an infrared light module, a dual camera module, a laser distance sensor, or any module capable of sensing scene depth. The object to be tested 110 has two sides on the axis 150. Viewed from any side, the outline (the outermost shape) of the object to be tested 110 is a circle, and the depth sensing module 130 is arranged on one side for photographing The side surface of the object under test 110 . In some embodiments, the depth sensing module 130 is a line scanner. When the object 110 rotates, the depth sensing module 130 continuously scans, and the captured image is called a rectangular depth image, especially a depth sensing The scanning range of the module 130 is smaller than the diameter of the above-mentioned circle, for example, it only needs to be greater than or equal to the radius of the circle.

The computing module 140 is communicatively connected to the depth sensing module 130 and the rotating modules 121 - 122 , and this communication connection can be achieved by any wired or wireless communication means. The computing module 140 is, for example, a central controller, a processor, a computer system, a server, or any electronic device with computing capabilities. The calculation module 140 controls the rotation modules 121 - 122 to determine the rotational angular velocity of the object under test 110 , and the calculation module 140 also receives a rectangular depth image from the depth sensing module 130 .

The object under test 110 may have some displacements when rotating, and these displacements will cause deviations in the sensed rectangular depth image. For example, the axis 160 is perpendicular to the axis 150, and the object under test 110 may be rotated at an angle along the axis 160, so that a part of the object under test 110 will be closer to the depth sensing module 130, and another part will be farther away from the depth sensing module 130. module 130, which makes the sensed rectangular depth image have a linear deviation. In addition, the rotation speed of the object under test 110 may be inconsistent, such that the rotation at an irregular speed will cause a non-linear deviation in the sensed rectangular depth image. The calculation module 140 will implement a depth detection and compensation method to detect whether there is a deviation in the rectangular depth image, and compensate the deviation if any. The method will be described below.

FIG. 2 is a flowchart illustrating a depth detection and compensation method according to an embodiment. FIG. 3 is a schematic diagram illustrating a detection deviation according to an embodiment. Referring to FIG. 2 and FIG. 3 , in step 201 , a rectangular depth image 300 is obtained through the depth sensing module. The rectangular depth image 300 has a sampling direction 301 and a rotation direction 302 . The sampling direction 301 is also the scanning direction of the line scanner, and the rotation direction 302 is the rotation direction of the object under test 110. The line scanner can generate a row of pixels per scan, and the gray scale of each pixel represents the depth. As the object under test 110 rotates, the line scanner will obtain the next row of pixels at the next sampling time.

In step 202 , it is determined whether the rectangular depth image 300 has deviation, and if there is deviation, then in step 203 it is further determined the type of deviation. Specifically, a sliding window (sliding window) 310 can be applied on the rectangular depth image 300 first, the length of the sliding window 310 in the sampling direction 301 is greater than 1 (for example, 3), and the length of the sliding window 310 in the rotation direction 302 The length is, for example, equal to the length of the rectangular depth image 300 in the rotation direction 302 . The sliding window 310 is used to implement a moving average algorithm (moving average), that is, to calculate the average of all pixels within the sliding window 310 . The sliding window 310 moves along the sampling direction 301 , for example, one or more pixels each time, and an average value can be calculated for each position during the moving process. In some embodiments, the ranges of the sliding window 310 at different positions may overlap with each other.

For the sake of illustration, the average calculated by using the moving average algorithm is referred to as the depth value hereinafter, and then it is determined whether the rectangular depth image 300 has a deviation according to these depth values. In an ideal situation without deviation, the distribution of depth values is similar to that shown in chart 320, wherein the horizontal axis is the sampling direction, and the vertical axis is the size of the depth value. Each point 321 in the chart 320 represents a depth value. For the sake of simplicity Not all depth values 321 are indicated here. In addition, the straight line 322 represents the trend of these depth values 321 , and ideally the slope of the straight line 322 should approach zero.

As mentioned above, when the test object 110 is rotated along the vertical axis 160, there will be a linear deviation. At this time, the distribution of depth values will be similar to the graph 330. These depth values 331 will gradually decrease, so the straight line 332 representing the trend The slope of will be less than 0; or the depth value 331 will rise gradually, which means the slope of the straight line of the trend is greater than 0. When there is a non-linear deviation, a chart 340 will be formed, and the depth value 341 will also gradually decrease, but the speed of decline in different segments 341~344 is different, that is to say, the slope of segment 342 will be greater than that of segment 343, and the slope of segment 343 The slope will be greater than the slope of segment 344 . Alternatively, the depth value 341 may rise gradually, but with different slopes in different segments. Here, the slope or linear regression algorithm can be used to determine which kind of deviation it belongs to.

表示第i個斜率，i為正整數。接下來設定一個範圍，表示為-R~+R，其中R為正整數。此外也設定一個基準值T，其初始值設定為0。對於每一個深度值

，如果

成立，則產生一個新的片段，並且設定新的基準值

，否則處理下一個深度值。在處理所有的深度值以後，判斷片段的個數，如果片段的個數為0，則表示沒有偏差。如果片段的個數為1，則有可能是有個轉折點或是線性偏差，因此可以進一步判斷所有斜率

的標準差，如果標準差在某個臨界值以下，則判斷為線性偏差。如果片段的個數大於1則表示非線性偏差。在圖表340的實施例中會產生片段342~344，這是因為片段交界處的斜率超出了上述的範圍。 Firstly, the method of slope is described. Firstly, the slope between every two adjacent depth values is calculated from left to right (along the sampling direction 301),

Indicates the ith slope, where i is a positive integer. Next, set a range, expressed as -R~+R, where R is a positive integer. In addition, a reference value T is also set, and its initial value is set to 0. For each depth value

,if

is established, a new segment is generated and a new benchmark value is set

, otherwise process the next depth value. After processing all the depth values, judge the number of fragments, if the number of fragments is 0, it means there is no deviation. If the number of fragments is 1, there may be a turning point or a linear deviation, so all slopes can be further judged

The standard deviation of , if the standard deviation is below a certain critical value, it is judged as a linear deviation. A non-linear deviation is indicated if the number of fragments is greater than 1. In the embodiment of graph 340, segments 342-344 are generated because the slopes at the segment boundaries exceed the above-mentioned range.

表示第i個深度值。線性迴歸演算法是要執行以下數學式1的最佳化演算法。 [數學式1]

Next, the linear regression algorithm is described. A linear function can be used to approximate all depth values. The following uses

Indicates the i-th depth value. The linear regression algorithm is an optimization algorithm that executes the following Mathematical Expression 1. [mathematical formula 1]

為深度值

所對應在取樣方向上的位置，

代表一個線性方程式，

為變數，數學式1的目的是要找到一條直線來逼近這些深度值，變數

代表直線的斜率。如果數學式1所計算出的誤差小於一臨界值且變數

在範圍-R~+R之內，則表示矩形深度影像300沒有偏差。如果數學式1計算出的誤差小於臨界值但變數

在範圍-R~+R之外，則表示為線性偏差。如果數學式1計算出的誤差大於等於臨界值，則表示為非線性偏差。當判斷為非線性偏差以後，可以再進一步判斷出片段，例如執行以下的最佳化演算法。 [數學式2]

in

is the depth value

corresponding to the position in the sampling direction,

represents a linear equation,

is a variable, the purpose of Mathematical Formula 1 is to find a straight line to approximate these depth values, the variable

represents the slope of the line. If the error calculated by Mathematical Formula 1 is less than a critical value and the variable

Within the range −R˜+R, it means that the rectangular depth image 300 has no deviation. If the error calculated by Mathematical Formula 1 is less than the critical value but the variable

Outside the range -R~+R, it is expressed as a linear deviation. If the error calculated by Mathematical Formula 1 is greater than or equal to the critical value, it is expressed as a nonlinear deviation. After the non-linear deviation is judged, the segment can be further judged, for example, the following optimization algorithm is executed. [mathematical formula 2]

是用以逼近第一個片段內的深度值，而線性方程式

是用以逼近第二個片段內的深度值。數學式2是要找到正整數k以及兩個線性方程式中的變數

，使得數學式2的目標函數有最小的誤差。如果數學式2所計算出的誤差大於一第二臨界值，則表示有多於2個片段，可以新增一個片段重複類似的最佳化演算法，直到誤差小於第二臨界值為止。換言之，根據線性迴歸演算法的結果可判斷矩形深度影像300是否具有線性偏差或是非線性偏差。 Where k is a positive integer, representing the junction of two segments. linear equation

is used to approximate the depth value in the first fragment, while the linear equation

is used to approximate the depth value in the second fragment. Mathematical formula 2 is to find the positive integer k and the variables in the two linear equations

, so that the objective function of Mathematical Formula 2 has the minimum error. If the error calculated by the mathematical formula 2 is greater than a second critical value, it means that there are more than 2 segments, and a similar optimization algorithm can be repeated until the error is smaller than the second critical value. In other words, according to the results of the linear regression algorithm, it can be determined whether the rectangular depth image 300 has a linear deviation or a nonlinear deviation.

In addition to the above slope and linear regression methods, those skilled in the art can use any algorithm to determine whether the depth value can be approximated by a horizontal line (no bias), a sloped line (linear deviation) or multiple segments (non-linear Deviation), the present disclosure is not limited to the above-mentioned practice.

，對深度值

的補償可以寫為

。如果是採用斜率方法，則可以計算所有斜率

的平均當作是斜率

。如此一來，補償後的深度值會接近圖表320所示的水平直線。由於深度值

代表矩形深度影像300多個像素的平均，因此可對所有對應像素都執行上述的補償。如果是非線性偏差，則可以在步驟205進行非線性補償。具體來說，在第一個片段中的補償類似於線性補償，寫為

；在第二個片段中的補償寫為

，如果有更多片段則以此類推。補償以後的深度值會接近圖表320所示的水平直線。類似的，由於深度值

代表矩形深度影像300多個像素的平均，因此可對所有對應像素都執行上述的補償。 After the deviation is detected, compensation can be performed according to whether there is a deviation, and if there is no deviation, no compensation is performed in step 350 . If it is a linear deviation, linear compensation can be performed in step 204 . Specifically, the variable of the linear equation can be found in the above practice

, for the depth value

The compensation can be written as

. In the case of the slope method, all slopes can be calculated

The average of is taken as the slope

. In this way, the compensated depth value will be close to the horizontal straight line shown in the graph 320 . Due to the depth value

Represents the average of more than 300 pixels in the rectangular depth image, so the compensation described above can be performed on all corresponding pixels. If it is a nonlinear deviation, nonlinear compensation can be performed in step 205 . Specifically, the compensation in the first fragment is similar to the linear compensation, written as

; the compensation in the second snippet is written as

, and so on if there are more fragments. The compensated depth value will be close to the horizontal line shown in the graph 320 . Similarly, since the depth value

Represents the average of more than 300 pixels in the rectangular depth image, so the compensation described above can be performed on all corresponding pixels.

Please return to FIG. 2 , if the result of step 202 is negative or after compensation in steps 204 and 205 , in step 206 an ideal data type (similar to the diagram 320 ) can be obtained. In step 207, post-processing may be performed on the compensated rectangular depth image 300, for example, converting the rectangular coordinates of the rectangular image into polar coordinates to become a circular image conforming to the shape of the steel coil.

In the above-mentioned depth detection and compensation system, by rotating the object to be measured, the scanning range of the depth sensing module 130 does not need to cover the entire object to be measured, which can reduce the hardware requirements of the depth sensing module 130 or Improve scanning accuracy. In addition, since the object under test may rotate along the vertical axis or the speed is not fixed, the sensed depth may deviate, and a method for detecting and compensating the deviation is also proposed here.

Although the present invention has been disclosed above with the embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field may make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention should be defined by the scope of the appended patent application.

100: Depth detection and compensation system 110: The object to be tested 121,122:Rotary module 130: Depth sensing module 140: Calculation module 150,160: axis 201~207: Steps 300: rectangular depth image 301: sampling direction 302: Rotation direction 310:Sliding window 320, 330, 340: Charts 321,331,341: Depth value 322,332: straight line 342~344: Fragment 350: step

FIG. 1 is a schematic diagram illustrating a detection system according to an embodiment. FIG. 2 is a flowchart illustrating a depth detection and compensation method according to an embodiment. FIG. 3 is a schematic diagram illustrating a detection deviation according to an embodiment.

201~207: Steps

Claims (8)

A depth detection and compensation system, comprising: an object to be measured; a plurality of rotating modules for rotating the object to be measured around an axis, wherein the outline of the object to be measured viewed from the axis is circular; The depth sensing module is arranged on one side of the object to be measured to obtain a rectangular depth image, wherein the rectangular depth image has a sampling direction and a rotation direction, and the scanning range of the depth sensing module is smaller than the circle the diameter of the shape; and a calculation module, connected to the rotation modules and the depth sensing module in communication, for applying a sliding window on the rectangular depth image, wherein the length of the sliding window in the sampling direction greater than 1, the length of the sliding window in the rotation direction is equal to the length of the rectangular depth image in the rotation direction, wherein the calculation module is used to perform a moving average algorithm according to the sliding window, and the sliding window is along the The sampling direction is moved to obtain a plurality of depth values, and each position of the sliding window corresponds to one of the depth values, wherein the calculation module is used to judge whether the rectangular depth image has deviation according to the depth values, if Compensate for this deviation, if any. The depth detection and compensation system as described in Claim 1, wherein the calculation module is also used to calculate the slope between every two adjacent depth values in the depth values, and according to whether the slope exceeds a range to determine whether the rectangular depth image has a linear deviation or a nonlinear deviation. The depth detection and compensation system as described in Claim 1, wherein the calculation module is also used to execute a linear regression algorithm according to the depth values, and judge whether the rectangular depth image has Linear deviation or non-linear deviation. The depth detection and compensation system according to Claim 1, wherein the object to be measured is a cylindrical steel coil. A depth detection and compensation method suitable for a computer system, the depth detection and compensation method includes: rotating an object to be measured around an axis through a plurality of rotating modules, wherein the contour of the object to be measured viewed from the axis It is circular; a rectangular depth image is obtained through a depth sensing module, wherein the depth sensing module is arranged on one side of the object to be measured, the rectangular depth image has a sampling direction and a rotation direction, the depth sensing The scanning range of the measurement module is smaller than the diameter of the circle; a sliding window is applied on the rectangular depth image, wherein the length of the sliding window in the sampling direction is greater than 1, and the length of the sliding window in the rotation direction is equal to The length of the rectangular depth image in the rotation direction; perform a moving average algorithm according to the sliding window, the sliding window moves along the sampling direction to obtain a plurality of depth values, each position of the sliding window corresponds to these One of the depth values; and judging whether the rectangular depth image has deviation according to these depth values, if so Compensate for this deviation. The depth detection and compensation method as described in claim 5, further comprising: calculating the slope between every two adjacent depth values among the depth values, and judging the depth of the rectangle according to whether the slope exceeds a range Whether the image has a linear bias or a non-linear bias. The depth detection and compensation method as described in Claim 5, further comprising: performing a linear regression algorithm according to the depth values, and judging whether the rectangular depth image has a linear deviation or a nonlinearity according to the result of the linear regression algorithm deviation. The depth detection and compensation method as described in Claim 5, wherein the object to be measured is a cylindrical steel coil.

Images