KR102066862B1 - Apparatus and method for measuring high-speed box size for conveyors using RGB-D camera - Google Patents

Abstract

Disclosed are an apparatus and a method for high-speed measurement of a box size for a conveyor using an RGB-D camera. In a situation where a box shaped object moves in parallel on an industrial conveyor, the present invention is easily installable and can measure the size of the box shaped object at high speed. According to the present invention, as an initial state is measured using the box shaped object as a direct target and a difference from an interest part acquired later is calculated, a calculation range is limited to the interest part rather than all of depth information. Therefore, a calculation amount is extremely small, and real time continuous measurement is possible.

Description

Apparatus and method for measuring high-speed box size for conveyors using RGB-D camera}

The present invention relates to a high-speed box size measuring apparatus and method for a conveyor using an RGB-D camera, and more particularly, in a situation where the box-like object moves in parallel on an industrial conveyor, the size of the box-type object can be easily installed and rapidly. The present invention relates to an apparatus and a method capable of measuring.

The RGB-D camera is a camera that can obtain depth, which is distance information from the focus of the sensor, in addition to the red, greed, and blue color information obtained from the existing camera. Depth information is usually calculated based on the infrared pattern using Time of Flight, Structured Light and Active Stereo. Either way, you can get depth information in the form of a matrix containing distance information from an RGB-D camera.

Many studies have been conducted to measure the size of objects using the depth information obtained here. However, the conventional research has a problem that it can be used in industrial sites where the conveyor is the center, such as measuring only a fixed object or a calculation time delay in mixing several cameras.

Korea Patent Registration No. 1886295 (Kyongsang National University Industry-Academic Cooperation Foundation) 2018. 8. 1. Patent Document 1 is a workpiece measuring device and a method for measuring a workpiece. Patent Document 1 includes a plurality of imaging units for photographing a workpiece to generate an image. A sensor unit for measuring a distance between the plurality of imaging units and a workpiece, a first moving unit for linearly moving the first imaging unit in a first direction, and a plurality of imaging units 2 The movement of the 1st and 2nd moving parts is controlled based on the 2nd moving part which linearly moves an imaging part to the 2nd direction perpendicular | vertical to a 1st direction, and the distance between a some imaging part and a workpiece | work, The content about the workpiece | work measurement apparatus containing the control part which measures the magnitude | size of a workpiece | work using the distance between the imaging part and the workpiece | work of the said is disclosed.

The technical problem to be achieved by the present invention is a high-speed box size measuring apparatus for a conveyor using an RGB-D camera that can easily install and measure the size of the box-shaped object at a high speed in a situation where the box-like object moves in parallel on the industrial conveyor. And a method.

High speed box size measuring apparatus for a conveyor using an RGB-D camera according to the present invention for achieving the above technical problem, the storage; An image acquisition unit storing color information and depth information provided from a photographing unit photographing an upper end of a conveyor on which box-shaped objects are moved in parallel using an RGB-D camera; And a measuring unit which checks whether a box-shaped object exists on the conveyor based on the color information and depth information provided from the image acquisition unit, and calculates the size of the box-shaped object if the box-shaped object exists.

The measuring unit obtains initial ROI information from depth information when a box-shaped object does not exist on the conveyor, stores the initial ROI information obtained in the storage unit, and obtains depth information obtained through the image acquisition unit. Obtains the change interest part information at, checks whether a box-shaped object exists based on the obtained change interest part information and the initial interest part information stored in the storage unit, and if the box-shaped object exists, the change interest part A state adjusting unit which stores information in the storage unit; A ROI measuring unit which measures the size of the box-shaped object based on the initial ROI information and the change interest area information, and provides the measured state of the box-shaped object to the state controller; And a size inference unit configured to determine a final size of the box-shaped object based on the size of the box-shaped object measured by the ROI measuring unit.

The point of interest may include a portion of a point corresponding to a horizontal line intersecting at a center point of depth information and a portion of a point belonging to a preset ROI and a point corresponding to a vertical line intersecting at a center point of depth information. It may consist of some points belonging to the area.

The state adjusting unit is operated in one of an initialization state, a standby state, and a measurement state, wherein the initialization state obtains the initial interest portion information from depth information when a box-shaped object does not exist on the conveyor, And storing the initial interested portion information obtained in the storage unit, wherein the standby state acquires change interested portion information from the depth information acquired through the image obtaining unit, and obtains the changed interested portion. When the difference between the information and the initial interest-information information is less than or equal to a preset reference value, the measurement waits, and the measurement state is obtained by obtaining the change-interest-portion information from the depth information obtained through the image acquisition unit. The difference between the acquired change interest part information and the initial interest part information is greater than a preset reference value. At the same time, the existence of the box-shaped object is checked, and if the box-shaped object is present, the change interest part information is stored in the storage unit, the ROI measuring unit is requested to measure the size of the box-shaped object, and the box-type received from the ROI measuring unit. It may be in a state of performing an operation of storing the size of an object in the storage unit.

The ROI measuring unit measures a height of a box-shaped object by calculating an average distance between an excited point set and a stable point set based on the initial interested part information and the changed interested part information. The maximum distance between the excitation points can be calculated to determine the width and depth of the box-like object.

The ROI measuring unit measures a width and a height of a box-shaped object based on points corresponding to a vertical line among the initial interest portion information and the change interest portion information, and corresponds to a horizontal line among the initial interest portion information and the change interest portion information. Based on the points, we can measure the length and height of the box-shaped object.

The excitation point set is a set consisting of points at which a difference between the initial interest portion information and the change interest portion information is larger than a preset reference value, and the set of stable points is a difference between the initial interest portion information and the change interest portion information. May be a set consisting of points smaller than or equal to the preset reference value.

The high speed box size measuring method for a conveyor using an RGB-D camera according to the present invention for achieving the above technical problem is a high speed box size measuring method of a high speed box size measuring apparatus for a conveyor using an RGB-D camera, a box-shaped object Receiving color information and depth information from a photographing unit which photographs an upper end of a conveyor to which parallel movement is performed using an RGB-D camera; And checking whether a box-shaped object exists on the conveyor based on the provided color information and depth information, and calculating the size of the box-shaped object if the box-shaped object exists.

The step of calculating the size of the box-shaped object may include obtaining initial interest portion information from depth information when a box-shaped object does not exist on the conveyor; Acquiring change interest part information from depth information provided from the photographing unit; Confirming the existence of a box-shaped object based on the obtained change interest part information and the initial interest part information; Measuring the size of the box-shaped object based on the initial ROI information and the changed ROI information when the box-shaped object exists; And determining a final size of the box-shaped object based on the measured size of the box-shaped object.

The point of interest may include a portion of a point corresponding to a horizontal line intersecting at a center point of depth information and a portion of a point belonging to a preset ROI and a point corresponding to a vertical line intersecting at a center point of depth information. It may consist of some points belonging to the area.

The high speed box size measuring apparatus for the conveyor is operated in one of an initialization state, a standby state, and a measurement state, wherein the initialization state is the initial interest portion in depth information when no box-shaped object exists on the conveyor. Acquiring information and storing the obtained initial interest portion information in the high speed box size measuring apparatus for the conveyor, wherein the standby state includes change interest portion information in depth information provided from the photographing unit. And when the difference between the acquired changed interest portion information and the initial interest portion information obtained is less than or equal to a preset reference value, the measurement wait state, and the measurement state is based on the depth information provided from the photographing unit. Obtaining change interest part information, and obtaining the changed interest part information and the initial interest When the difference of the partial information is larger than the preset reference value, the existence of the box-shaped object is checked, and if the box-shaped object exists, the change interest part information is stored in the high speed box size measuring device for the conveyor, and the size of the box-shaped object is determined. The measurement may be performed to store the size of the measured box-like object in the high speed box size measuring apparatus for the conveyor.

The box-shaped object size measuring step may calculate a height of a box-shaped object by calculating an average distance between an excited point set and a stable point set based on the initial interested portion information and the changed interested portion information. And measuring the width and the depth of the box-shaped object by calculating the maximum distance between the excitation points.

The measuring of the box-shaped object size may include measuring a width and a height of the box-shaped object based on points corresponding to a vertical line among the initial interest portion information and the change interest portion information, and among the initial interest portion information and the change interest portion information. The length and height of the box-shaped object may be measured based on the points corresponding to the horizontal lines.

The excitation point set is a set consisting of points at which a difference between the initial interest portion information and the change interest portion information is larger than a preset reference value, and the set of stable points is a difference between the initial interest portion information and the change interest portion information. May be a set consisting of points smaller than or equal to the preset reference value.

The computer program according to the present invention for achieving the above technical problem is stored in a computer-readable recording medium to execute any one of the above methods in a computer.

According to the high speed box size measuring apparatus and method for a conveyor using an RGB-D camera according to the present invention, it is possible to easily install and measure the size of a box-shaped object with high speed and accuracy.

In particular, the present invention measures the initial state of the box-shaped object directly, and calculates the difference with the obtained interest portion thereafter, so that the calculation range is limited to the interest portion instead of all of the depth information. Continuous measurement is possible. In addition, by using a conveyor conveying scenario and a state transition according to the box-shaped object, it is possible to smoothly measure and operate the logistics system without stopping the conveyor when applied to the logistics system.

1 is a block diagram illustrating a high speed box size measuring apparatus for a conveyor using an RGB-D camera according to a preferred embodiment of the present invention.
FIG. 2 is a view showing an example of the box-shaped object shown in FIG. 1.
3 is a view for explaining an operating state of the state adjustment unit shown in FIG. 1.
4 is a view for explaining an example of a scenario for measuring the size of a box-shaped object according to an embodiment of the present invention.
5 is a front view and a side view for explaining a measurement process according to the size measurement scenario of the box-shaped object shown in FIG.
FIG. 6 is a diagram illustrating color information and depth information according to the size measurement scenario of the box-shaped object shown in FIG. 4.
7 is a flowchart illustrating a high speed box size measuring method for a conveyor using an RGB-D camera according to a preferred embodiment of the present invention.
FIG. 8 is a flowchart illustrating the box-shaped object size calculating step shown in FIG. 7 in more detail.

Hereinafter, exemplary embodiments of a high speed box size measuring apparatus and method for a conveyor using an RGB-D camera according to the present invention will be described in detail with reference to the accompanying drawings.

First, a high-speed box size measuring apparatus for a conveyor using an RGB-D camera according to a preferred embodiment of the present invention will be described with reference to FIGS. 1 to 6.

1 is a block diagram for explaining a high speed box size measuring apparatus for a conveyor using an RGB-D camera according to a preferred embodiment of the present invention, Figure 2 is a view showing an example of the box-shaped object shown in FIG. 3 is a view for explaining the operating state of the state adjustment unit shown in Figure 1, Figure 4 is a view for explaining an example of the size measurement scenario of the box-shaped object according to a preferred embodiment of the present invention, Figure 5 is Figure 4 FIG. 6 is a front view and a side view illustrating a measurement process according to the size measurement scenario of the box-shaped object shown in FIG. 6, and FIG. 6 is a view showing color information and depth information according to the size measurement scenario of the box-shaped object shown in FIG. 4.

Referring to Figure 1, a high-speed box size measuring device (hereinafter referred to as "size measuring device") 100 for a conveyor using an RGB-D camera according to a preferred embodiment of the present invention is a box-type that is moved parallel to the conveyor 400 Check whether the object 500 exists, and calculate the size of the box-shaped object 500 (that is, the height, length, and width of the box-shaped object as shown in FIG. 2) if the box-shaped object 500 exists. .

The size measuring apparatus 100 is connected to the photographing unit 200 to receive color information (ie, red cyan information) and depth information (see FIG. 6) from the photographing unit 200. Here, each value of the depth information represents a distance from a corresponding point in the photographing portion. Smaller distances mean smaller values, and larger distances mean larger values. This is why, as shown in FIG. 6, the portion where the box-shaped object 500 is dark in the depth information DI is dark, and the conveyor 400 without the box-shaped object 500 is bright.

The photographing unit 200 includes an RGB-D camera (not shown), is fixedly installed around the conveyor 400 through the fixing unit 300, and is fixed at a predetermined time interval under the control of the size measuring device 100. By using the RGB-D camera to photograph the upper end of the conveyor 400, the box-shaped object 500 is moved in parallel.

Meanwhile, although the size measuring apparatus 100 and the photographing unit 200 are separate devices that are independent of each other, as illustrated in FIG. 1, the size measuring apparatus 100 and the photographing unit 200 are not limited thereto. It can be integrated into one device and implemented. In addition, although the size measuring device 100 is a separate device for the high-speed box size measuring operation according to the present invention is shown in FIG. 1, the size measuring device 100 according to the embodiment is not limited thereto. It may be implemented in a user terminal, such as a desktop PC, a notebook.

To this end, the size measuring apparatus 100 may include a storage 110, an image acquisition unit 130, and a measurement unit 150.

The storage unit 110 stores a program and data necessary for the operation of the size measuring device 100 and may be divided into a program area and a data area.

The program area may include a program for controlling the overall operation of the size measuring device 100, an operating system (OS) for booting the size measuring device 100, checking for the existence of the box-shaped object 500, and checking the existence of the box-shaped object 500. An application program necessary for the operation of the size measuring device 100, such as size calculation, may be stored.

The data area is an area in which data generated according to the use of the size measuring device 100 is stored, and may store color information and depth information, initial interest portion information, change interest portion information, and size of the box-shaped object 500. .

The image acquisition unit 130 stores the color information and the depth information provided from the photographing unit 200 which photographs the upper end of the conveyor 400 in which the box-shaped object 500 is moved in parallel using an RGB-D camera. ).

The measuring unit 150 checks whether the box-shaped object 500 exists on the conveyor 400 based on the color information and the depth information provided from the image obtaining unit 130, and if the box-shaped object 500 exists, it is box-shaped. The size of the object 500 is calculated.

To this end, the measuring unit 150 may include a state adjusting unit 151, a ROI measuring unit 153, and a size inference unit 155.

The state adjusting unit 151 obtains initial interesting portion information from the depth information when the box-shaped object 500 does not exist on the conveyor 400, and stores the obtained initial interested portion information in the storage unit 110. That is, as shown in FIGS. 4 and 5, the state adjusting unit 151 may have the initial interest portion vertical line WR-1 in the depth information obtained when the box-shaped object 500 does not exist on the conveyor 400. An initial interesting portion horizontal line DR-1 may be obtained and stored in the storage unit 110.

In addition, the state adjusting unit 151 obtains the change interested part information from the depth information acquired through the image obtaining unit 130, and obtains the changed interested part information and the initial interested part information stored in the storage 110. Check the existence of the box-shaped object 500 based on. That is, as shown in FIG. 4, when the box-shaped object 500 passes the region of interest, the change-interest vertical line WR-2 becomes different from the initial region of interest vertical line WR-1, and the state adjusting unit 151 The presence of the box-shaped object 500 may be determined based on the difference between the vertical line of interest WR-2 and the initial vertical line of interest WR-1.

In addition, the state adjusting unit 151 stores the change interest part information in the storage unit 110 when the box-shaped object 500 exists.

Here, the interesting part is a part of the points corresponding to the horizontal lines intersecting at the center point of the depth information, and a part of the points of interest preset among the points corresponding to the vertical lines intersecting at the center point of the depth information. It may consist of some points belonging to. The interesting portion may utilize two vertical segments intersecting at the center of the screen, as shown in the color information CI of FIG. 6. However, this is only one embodiment, and not limited thereto. In some embodiments, two vertical line segments intersecting at points other than the center point may be used.

At this time, some of the points corresponding to the horizontal lines intersecting at the center point of the depth information belong to a preset ROI, which are called "interesting horizontal lines", and points corresponding to the vertical lines intersecting at the center point of the depth information. Some of the points belonging to the preset ROI are referred to as "interesting vertical lines".

In more detail, as shown in FIG. 3, the state adjusting unit 151 may be operated in one of an initialization state IS, a standby state SS, and a measurement state MS.

The initialization state IS may be configured to obtain initial ROI information from the depth information when the box-shaped object 500 does not exist on the conveyor 400 and store the obtained ROI information in the storage 110. The state to run.

The standby state SS obtains the change interest part information from the depth information acquired through the image acquisition unit 130, and when the difference between the acquired change interest part information and the initial interest part information is less than or equal to a preset reference value. , Waiting for measurement.

The measurement state MS obtains the change interest part information from the depth information acquired through the image acquisition unit 130, and when the difference between the acquired change interest part information and the initial interest part information is larger than a preset reference value, the box type The existence of the object 500 is checked, and if the box-shaped object 500 exists, the change interest part information is stored in the storage 110, and the ROI measuring unit 153 requests the size measurement of the box-shaped object 500. In this state, the size of the box-shaped object provided from the ROI measurer 153 is stored in the storage 110. That is, due to the passage of the box-shaped object 500, the interesting part is changed from the stable state to the excited state, and thus the changed interesting part information is different from the initial interesting part information, thereby performing the size measurement operation of the box-shaped object 500. Will be performed.

The ROI measurer 153 measures the size of the box-shaped object 500 based on the initial ROI information and the changed ROI information, and provides the state adjuster 151 with the measured size of the box-shaped object 500.

That is, the ROI measuring unit 153 calculates an average distance between the excited point set and the stable point set based on the initial interest part information and the change interest part information, thereby increasing the height of the box-shaped object 500. ) Can be measured. In addition, the ROI measuring unit 153 may measure the width and the depth of the box-shaped object 500 by calculating the maximum distance between the excited points.

Here, the excited point set is a set of points in which the difference between the initial point of interest information and the change point of interest information is larger than a predetermined reference value. The set of stable points is a set of points in which the difference between the initial ROI information and the change ROI information is smaller than or equal to a preset reference value.

In more detail, the ROI measurer 153 is based on the points corresponding to the vertical lines of the initial interest portion information and the change interest portion information, that is, the initial interest portion vertical line WR-1 and the change interest portion vertical line WR-2. ), The width and height of the box-shaped object 500 can be measured. In addition, the ROI measurer 153 may generate the initial ROI DR-1 and the change ROI DR-2 based on the points corresponding to the horizontal lines among the initial ROI information and the change ROI information. Based on this, the length and height of the box-shaped object 500 may be measured.

The size inference unit 155 determines the final size of the box-shaped object 500 based on the size of the box-shaped object 500 measured by the ROI measurer 153.

Then, a high-speed box size measurement process for a conveyor using an RGB-D camera according to a preferred embodiment of the present invention will be described in more detail.

번째 열과

번째 행에 있는 한 점

엔 초점부터 점에 해당하는 부분 사이의 거리 정보를 포함한다.When the state adjusting unit 151 imports the depth information image consisting of V rows and U columns from the image obtaining unit 130 at a point in time t,

Column and

One point in the first row

It includes distance information between the portion corresponding to the point from the yen focus.

로 표기하고, 이에 속한 점을

와 같은 수식으로 정의한다. RGB-D 카메라와 기타 문제로 생기는 거리 오차의 최대값을

라 할 때, 박스형 물체(500)가 없는 빈 컨베이어(400)의 경우 시점 t에서의 임의의 점

와 시점 t+1에서의 동일한 점

사이의 거리는

보다는 작다. 반대로, 만약 박스형 물체(500)가 흥미부분을 통과하여 두 점 사이의 거리가

보다 크다면, 이 변화를 이용해 박스형 물체(500)의 유무를 식별 가능하다. 본 발명에서는 점의 상태를 [수학식 1]과 같이 정의한다.In the present invention, a set of points belonging to an interesting part

, And the points that belong to

Defined by an expression such as Maximum distance error caused by RGB-D cameras and other problems

In the case of the empty conveyor 400 without the box-shaped object 500, any point at time t

And the same point at time t + 1

The distance between

Is smaller than. Conversely, if box-like object 500 passes through the region of interest, the distance between the two points

If greater, this change can be used to identify the presence or absence of the box-shaped object 500. In the present invention, the state of the point is defined as shown in [Equation 1].

In [Equation 1], excited denotes a separately excited state and stable as a stable state. Similarly, the number of excited points in the ROI is expressed by Equation 2.

과 안정(Stable)점 집합

을 [수학식 3]과 [수학식 4]로 정의한다.In addition, based on Equation 1, an excited point set at time t

And set of stable points

Is defined by [Equation 3] and [Equation 4].

계산할 수 있다. 마지막으로 t=0인 초기 상태는

와 같이 모든 ROI안의 점들이 안정 상태라 정의한다. 상기 정의한 [수학식 1] 내지 [수학식 4]의 원리를 이용해 아래와 같이 알고리즘 1를 제시한다.In addition, through [Equation 1], [Equation 3], [Equation 4]

Can be calculated Finally, the initial state of t = 0

We define that all points in the ROI are stable. Using the principles of [Equation 1] to [Equation 4] defined above, Algorithm 1 is presented as follows.

와 초기 시점에서의 초기 안점점 집합

를 입력으로 받아 높이와 너비를 계산하는 알고리즘이다. 내부 함수(Procedure)중 하나인 Distance는 한 쌍의 2차원들을 받아 3차원 위치로 변환 후 두 점 사이의 거리를 반환한다. 상기 알고리즘 1 안에서 줄 5-9는 높이를 측정하는 방법을 나타낸다. 이때 높이 측정값은 각 집합에서 u와 v가 동일한 점들의 거리 평균으로 계산한다. 상기 알고리즘 1의 줄 11-16은 시점 t에서의 들뜬점 집합

을 선분으로 가정할 때 이상값(outlier)을 RANSAC(RANdom SAmple Consensus) 알고리즘을 이용해 제거하고 점들 사이의 최대 거리를 계산하는 것을 포함한다.Algorithm 1 is a set of excited points at the defined time point t

Set of initial eye points at time and initial time

It is an algorithm that takes as input and calculates height and width. One of the internal functions, Distance, takes a pair of two dimensions, converts them to a three-dimensional position, and returns the distance between the two points. Lines 5-9 in Algorithm 1 indicate how to measure height. The height measurement is then calculated as the average of the distances of the points where u and v are the same in each set. Lines 11-16 of Algorithm 1 set the excited points at time t.

Assuming that as a line segment, the outlier is removed by using a random SAmple consensus (RANSAC) algorithm and the maximum distance between the points is calculated.

In order to make the algorithm 1 operate smoothly, the present invention further defines a state transition algorithm (algorithm 2), which is a higher level of the algorithm 1. The state transition algorithm may be used as an embodiment of the operating state of the state adjusting unit 151 of FIG. 3.

,

)가 입력되었을 때 ROI지점의 박스형 물체(500)의 유무에 따라 상태를 전환하는 알고리즘이다. 상기 알고리즘 2의 줄 6은 영상 획득부(130)로부터 깊이 정보를 추출하는 과정을 의미한다. 상기 알고리즘 2의 줄 32 ~ 37은 깊이 정보와 추출할 행 열 범위가 주어졌을 때, 흥미부분 가로선 집합과 흥미부분 세로선 집합을 만들어 반환하는 함수(Procedure)를 의미한다. 본 발명에서 알고리즘 2의 줄 4와 같이 기준 최솟값

과

를 포함한다. 다만 알고리즘에 기재된 값은 본 발명의 실시예 중의 하나로서 해당 값으로 그 보호범위가 제한되지 않는다.Algorithm 2 is the range of interest

,

) Is an algorithm for switching the state depending on the presence or absence of the box-shaped object 500 at the ROI point. Line 6 of the algorithm 2 refers to a process of extracting depth information from the image acquisition unit 130. Lines 32 to 37 of Algorithm 2 represent a function of generating and returning a set of interesting horizontal lines and a set of interesting vertical lines when depth information and a range of row columns to be extracted are given. In the present invention, the reference minimum value as in line 4 of algorithm 2

and

It includes. However, the value described in the algorithm is one of the embodiments of the present invention, and the protection range is not limited to the corresponding value.

가

보다 작으면 측정을 대기하고 그 반대의 경우 state값을 2로 바꾸어 대기 상태(SS)에서 측정 상태(MS)로 상태전이를 한다.Lines 9 to 12 of Algorithm 2 correspond to the initialization state IS of FIG. 3, and write the initial line of interest vertical line WR-1 to the storage medium. Lines 13 to 18 of the algorithm 2 correspond to the standby state SS of FIG. 3,

end

If it is smaller, it waits for the measurement and vice versa to change the state value to 2 to transfer the state from the standby state (SS) to the measurement state (MS).

Lines 19 to 30 of Algorithm 2 correspond to the measurement state MS of FIG. 3, and repeatedly call the ROI-measurement function of Algorithm 1 while the box-shaped object 500 passes through the region of interest. Then, the width, height, and length of the returned box-shaped object 500 are recorded in the storage medium.

이

보다 작아지게 된다. 이 경우 함수 SaveOutput을 호출하여 저장매체에 기록된 값을 크기 추론부(155)에 전달한다.Algorithm 2 is performed when the box-shaped object 500 has completely passed the region of interest.

this

It becomes smaller. In this case, the function SaveOutput is called to transfer the value recorded in the storage medium to the size inference unit 155.

Next, a high-speed box size measuring method for a conveyor using an RGB-D camera according to a preferred embodiment of the present invention will be described with reference to FIGS. 7 and 8.

7 is a flowchart illustrating a high speed box size measuring method for a conveyor using an RGB-D camera according to a preferred embodiment of the present invention.

Referring to FIG. 7, the size measuring apparatus 100 receives color information and depth information from the photographing unit 200 (S110).

Then, the size measuring apparatus 100 checks whether the box-shaped object 500 exists on the conveyor 400 based on the color information and the depth information provided from the photographing unit 200, and the box-shaped object 500 is If present, the size (height, length, and width) of the box-shaped object 500 is calculated (S130).

FIG. 8 is a flowchart illustrating the box-shaped object size calculating step shown in FIG. 7 in more detail.

Referring to FIG. 8, the size measuring apparatus 100 may obtain initial interested portion information from depth information when the box-shaped object 500 does not exist on the conveyor 400 (S131). Here, the interesting part is a part of the points corresponding to the horizontal lines intersecting at the center point of the depth information, and a part of the points of interest preset among the points corresponding to the vertical lines intersecting at the center point of the depth information. It may consist of some points belonging to. The interesting portion may utilize two vertical segments intersecting at the center of the screen, as shown in the color information CI of FIG. 6. However, this is only one embodiment, and not limited thereto. In some embodiments, two vertical line segments intersecting at points other than the center point may be used. In this case, some of the points corresponding to the horizontal lines intersecting at the center point of the depth information belong to a preset ROI, which are called "interesting horizontal lines", and points corresponding to the vertical lines intersecting at the center point of the depth information. Some of the points belonging to the preset ROI are referred to as "interesting vertical lines".

That is, as shown in FIGS. 4 and 5, the size measuring device 100 may have the initial interest portion vertical line WR-1 in the depth information obtained when the box-shaped object 500 does not exist on the conveyor 400. And the initial interest portion horizontal line DR-1 may be obtained and stored.

Thereafter, the size measuring apparatus 100 may obtain the change interest part information from the depth information provided from the photographing unit 200 (S132).

Then, the size measuring apparatus 100 may check the presence or absence of the box-shaped object 500 based on the obtained change interest part information and initial interest part information (S133).

That is, as shown in FIG. 4, when the box-shaped object 500 passes the region of interest, the changed region of interest vertical line WR-2 is different from the initial region of interest vertical line WR-1, and the size measuring device 100 ) May determine whether the box-shaped object 500 exists based on the difference between the vertical line of interest WR-2 and the initial vertical line of interest WR-1.

If the box-shaped object 500 exists (S134-Y), the size measuring device 100 may measure the size of the box-shaped object 500 based on the initial ROI information and the change ROI information (S135).

That is, the size measuring apparatus 100 calculates an average distance between the excited point set and the stable point set based on the initial interest part information and the change interest part information, thereby increasing the height of the box-shaped object 500. ) And the maximum distance between the excitation points can be used to measure the width and depth of the box-shaped object 500. Here, the excitation point set is a set in which the difference between the initial point of interest information and the change point of interest information is larger than a predetermined reference value, and the set of stable points is smaller than the predetermined reference value. Or a set of points.

In more detail, the size measuring apparatus 100 is based on the points corresponding to the vertical lines of the initial interest portion information and the change interest portion information, that is, the initial interest portion vertical line WR-1 and the change interest portion vertical line WR-2. The width and height of the box-shaped object 500 are measured, and based on the points corresponding to the horizontal lines of the initial interest portion information and the change interest portion information, that is, the initial interest portion horizontal line DR-1 and the change. The length and height of the box-shaped object 500 may be measured based on the horizontal line DR-2 of interest.

Then, the size measuring apparatus 100 may obtain the change interest part information from the depth information provided from the photographing unit 200 (S136).

If the acquired change interest part information is different from the initial interest part information (S1237-N), the size measuring apparatus 100 may perform steps S135 and S136 again.

On the other hand, if the acquired change interest part information is the same as the initial interest part information (S137-Y), the size measuring device 100 determines the final size of the box-shaped object 500 based on the measured size of the box-shaped object 500. Can be determined.

In other words, as illustrated in FIG. 3, the size measuring apparatus 100 may be operated in one of an initialization state IS, a standby state SS, and a measurement state MS.

The initialization state IS is a state in which the initial ROI information is acquired from the depth information when the box-shaped object 500 does not exist on the conveyor 400, and the obtained initial ROI information is stored.

The standby state SS obtains the change interest part information from the depth information provided from the photographing unit 200 and measures the difference when the difference between the acquired change interest part information and the initial interest part information is less than or equal to a preset reference value. Waiting.

The measurement state MS obtains the change interest part information from the depth information provided from the photographing unit 200, and when the difference between the acquired change interest part information and the initial interest part information is larger than a preset reference value, the box-shaped object ( Check whether the box 500 exists, and store the change interest part information when the box type object 500 exists, measure the size of the box type object 500, and store the size of the measured box type object. to be.

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like.

Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific preferred embodiments described above, and the technical field to which the present invention pertains without departing from the gist of the present invention claimed in the following claims. Anyone of ordinary skill in the art that various modifications can be made, as well as such changes are within the scope of the claims.

100: high speed box size measuring device for conveyor,
110: storage unit, 130: image acquisition unit,
150: measuring unit, 151: state adjusting unit,
153: ROI measuring unit, 155: size inference unit,
200: photographing part, 300: fixing part,
400: conveyor, 500: box-shaped object

Claims (15)

Storage unit;
An image acquisition unit storing color information and depth information provided from a photographing unit photographing an upper end of a conveyor on which box-shaped objects are moved in parallel using an RGB-D camera; And
A measuring unit which checks whether a box-shaped object exists on the conveyor based on color information and depth information provided from the image acquisition unit, and calculates a size of the box-shaped object if the box-shaped object exists;
Including;
The measuring unit obtains initial ROI information from depth information when a box-shaped object does not exist on the conveyor, stores the initial ROI information obtained in the storage unit, and obtains depth information obtained through the image acquisition unit. Obtains the change interest part information at, checks whether a box-shaped object exists based on the obtained change interest part information and the initial interest part information stored in the storage unit, and if the box-shaped object exists, the change interest part A state adjusting unit which stores information in the storage unit; A ROI measuring unit which measures the size of the box-shaped object based on the initial ROI information and the change interest area information, and provides the measured state of the box-shaped object to the state controller; And a size inference unit configured to determine a final size of the box-shaped object based on the size of the box-shaped object measured by the ROI measuring unit.
The point of interest may include a portion of a point corresponding to a horizontal line intersecting at a center point of depth information and a portion of a point belonging to a preset ROI and a point corresponding to a vertical line intersecting at a center point of depth information. High speed box size measuring device for conveyor using RGB-D camera which consists of some points belonging to the area. delete delete In claim 1,
The state adjusting unit is operated in one of an initialization state, a standby state, and a measurement state,
The initialization state is a state in which the initial interest portion information is obtained from depth information when a box-shaped object does not exist on the conveyor, and the obtained initial interest portion information is stored in the storage unit.
The waiting state is obtained when the change interest part information is acquired from depth information acquired through the image acquisition unit, and the difference between the acquired change interest part information and the initial interest part information is less than or equal to a preset reference value. Is waiting for
The measurement state is obtained by obtaining the change interest part information from the depth information acquired through the image acquisition unit, and when the difference between the acquired change interest part information and the initial interest part information is greater than a preset reference value, Check whether there is a box-type object, and if the box-shaped object exists, store the change interest part information in the storage unit, request the ROI measuring unit to measure the size of the box-type object, and measure the size of the box-type object provided from the ROI measuring unit. In the state of performing the operation to save to the storage unit,
High speed box size measuring device for conveyor using RGB-D camera. In claim 1,
The ROI measuring unit measures a height of a box-shaped object by calculating an average distance between an excited point set and a stable point set based on the initial interested part information and the changed interested part information. By calculating the maximum distance between excitation points, we measure the width and depth of a box-like object,
High speed box size measuring device for conveyor using RGB-D camera. In claim 5,
The ROI measuring unit measures a width and a height of a box-shaped object based on points corresponding to a vertical line among the initial interest portion information and the change interest portion information, and corresponds to a horizontal line among the initial interest portion information and the change interest portion information. To measure the length and height of a box-shaped object
High speed box size measuring device for conveyor using RGB-D camera. In claim 5,
The excited point set is a set consisting of points at which the difference between the initial interest portion information and the change interest portion information is larger than a preset reference value.
The set of stable points is a set consisting of points at which the difference between the initial point of interest information and the change point of interest information is less than or equal to the preset reference value.
High speed box size measuring device for conveyor using RGB-D camera. As a high speed box size measuring method of a high speed box size measuring device for a conveyor using an RGB-D camera,
Receiving color information and depth information from a photographing unit which photographs an upper end of the conveyor on which the box-shaped object is moved in parallel using an RGB-D camera; And
Checking whether a box-shaped object exists on the conveyor based on the provided color information and depth information, and calculating the size of the box-shaped object if the box-shaped object exists;
Including;
The step of calculating the size of the box-shaped object may include obtaining initial interest portion information from depth information when a box-shaped object does not exist on the conveyor; Acquiring change interest part information from depth information provided from the photographing unit; Confirming the existence of a box-shaped object based on the obtained change interest part information and the initial interest part information; Measuring the size of the box-shaped object based on the initial ROI information and the changed ROI information when the box-shaped object exists; And determining a final size of the box-shaped object based on the measured size of the box-shaped object.
The point of interest may include a portion of a point corresponding to a horizontal line intersecting at a center point of depth information and a portion of a point belonging to a preset ROI and a point corresponding to a vertical line intersecting at a center point of depth information. High speed box size measurement method for a conveyor using an RGB-D camera consisting of some points belonging to an area. delete delete In claim 8,
The high speed box size measuring device for the conveyor is operated in one of an initialization state, a standby state, and a measurement state,
In the initialization state, the initial interest portion information is obtained from depth information when a box-shaped object does not exist on the conveyor, and the obtained initial interest portion information is stored in the high speed box size measuring apparatus for the conveyor. State
The waiting state obtains the change interest part information from the depth information provided from the photographing unit, and measures the measurement when the difference between the acquired change interest part information and the initial interest part information is less than or equal to a preset reference value. Is waiting.
The measurement state may be obtained when the change interest part information is acquired from the depth information provided from the photographing unit, and when the difference between the acquired change interest part information and the initial interest part information is greater than a preset reference value, the presence of a box-shaped object Check whether the box-shaped object exists, and store the change-interest part information in the high speed box size measuring apparatus for the conveyor, measure the size of the box type object, and measure the size of the measured box type object in the high speed box size for the conveyor. Performing a save operation on the device,
High speed box size measurement method for conveyor using RGB-D camera. In claim 8,
The box-shaped object size measuring step may calculate a height of a box-shaped object by calculating an average distance between an excited point set and a stable point set based on the initial interested portion information and the changed interested portion information. And measuring the width and length of the box-shaped object by calculating the maximum distance between the excitation points,
High speed box size measurement method for conveyor using RGB-D camera. In claim 12,
The measuring of the box-shaped object size may include measuring a width and a height of the box-shaped object based on points corresponding to a vertical line among the initial interest portion information and the change interest portion information, and among the initial interest portion information and the change interest portion information. Consisting of measuring the length and height of a box-shaped object based on the points corresponding to the horizontal lines,
High speed box size measurement method for conveyor using RGB-D camera. In claim 12,
The excited point set is a set consisting of points at which the difference between the initial interest portion information and the change interest portion information is larger than a preset reference value.
The set of stable points is a set consisting of points at which the difference between the initial point of interest information and the change point of interest information is less than or equal to the preset reference value.
High speed box size measurement method for conveyor using RGB-D camera. A computer program stored in a computer-readable recording medium for executing a high speed box size measuring method for a conveyor using an RGB-D camera according to any one of claims 8 and 11 to 14.

Images