KR20220160314A - Method and apparatus for setting data matrix coordianate - Google Patents

Abstract

According to the present invention, disclosed are a data matrix recognition method and apparatus which can identify a plurality of samples stored in a cryobox. According to the present invention, the apparatus includes a processor and a memory connected to the processor, wherein the memory stores program instructions executed by the processor to perform: (a) generating a binarized image by applying a first binarization threshold to an RGB image obtained by photographing a plurality of data matrices corresponding to a plurality of columns stored in a cryobox; (b) extracting a set of points included in the binarized image; (c) extracting a first shape set corresponding to the shape of the data matrix by removing adjacent points and concave surfaces from the extracted point set; (d) estimating one or more data matrices by applying a preset rule to the extracted first shape set; (e) decoding the estimated one or more data matrices; and (f) generating a binarized image by applying a binarization threshold having a value sequentially larger than the first binarization threshold to the RGB image, and repeating steps (b) to (e).

Description

Method and apparatus for setting data matrix coordinates {Method and apparatus for setting data matrix coordianate}

The present invention relates to a method and apparatus for setting data matrix coordinates, and more particularly, to a method and apparatus capable of rapidly obtaining information on a plurality of samples stored in a cryobox.

In general, samples such as microorganisms, antibodies, and biosynthetic materials are stored in elongated columns, and a plurality of columns are accommodated and stored in a cryo box having a plurality of grids.

In the past, in order to identify a sample, it was necessary to open the column and perform sample analysis, but recently, a data matrix (two-dimensional code) for sample identification is attached to each column cover.

However, in the prior art, there is a cumbersome problem because the data matrix must be individually recognized by taking out individual columns from a plurality of grids.

On the other hand, a method of recognizing a plurality of objects stored in a specific space using a data matrix has been proposed, but since it is based on coordinates, there is a limit to applying it to a cryobox for storing small-sized specimens.

In addition, even if the data matrix of each column stored in a plurality of grids is recognized, if the position of the corresponding data matrix is not accurately known, there is a problem in that it is inevitable to take out and recognize the data matrix individually.

Republic of Korea Patent Publication KR 10-1713134

In order to solve the above problems of the prior art, the present invention proposes a data matrix coordinate setting method and apparatus capable of identifying a plurality of samples stored in a cryobox at once.

In order to achieve the above object, according to an embodiment of the present invention, an apparatus for setting data matrix coordinates in a cryobox, comprising: a processor; and a memory connected to the processor, wherein the memory includes a number of lattices, a lattice height, a lattice width of a plurality of lattices formed in the cryobox, and a distance from the top of the cryobox to an upper surface of the plurality of lattices. A depth is stored, an RGB image obtained by photographing a plurality of data matrices corresponding to a plurality of columns stored in the plurality of grids is loaded, and information related to an outermost point corresponding to the top of the cryo box is obtained from the RGB image. obtaining a lattice surface corresponding to an upper surface of the plurality of lattices using the RGB image and the depth, and equally dividing the lattice surface using the number of lattices to divide the lattice surface into the plurality of lattices in the RGB image Each center point is set, and a data matrix surface corresponding to the plurality of data matrices obtained from the RGB image is projected and converted to the grid surface by reflecting a preset parallax correction value, and each of the plurality of data matrices obtained from the projection and conversion is A program executed by the processor to compare distances to the center point of each of the plurality of lattices, and to set coordinates of a center point closest to each of the plurality of data matrices subjected to projection transformation as coordinates of each of the plurality of data matrices. An apparatus for setting data matrix coordinates in a cryobox storing instructions is provided.

The parallax correction value may be determined by a vertical distance (H) of the camera lens and the plurality of gratings and the grating height (h).

When H is located within a preset range, the parallax correction value may be determined as a constant value.

When H is in the range of 180 to 210 mm, the parallax correction value may be determined as a constant value.

The program commands generate a binarized image by applying a first binarization threshold to the RGB image, extract a point set included in the binarized image, and remove adjacent points and concave surfaces from the extracted point set to obtain the data. A first shape set corresponding to the shape of the matrix may be extracted, one or more data matrices may be estimated by applying a preset rule to the extracted first shape set, and the estimated one or more data matrices may be decoded.

The program instructions generate a binarized image by applying a binarization threshold having a value sequentially greater than the first binarization threshold to the RGB image, and repeatedly perform point set extraction, first shape set extraction, data matrix estimation, and decoding. can do.

In the program instructions, the size of the column vector may be set to less than half of the grid width in order to prevent a coordinate setting error caused by the column vector according to the storage direction of the column stored in one of the plurality of grids.

In order to prevent a coordinate setting error due to the column vector, the grid width (w) and the grid height (h) have a relation of the following equation.

[mathematical expression]

where d is the radius of the column and L is the height of the column

According to another aspect of the present invention, a method for setting coordinates of a data matrix in a cryobox in a device including a processor and a memory, comprising the number of lattices, lattice height, lattice width and the storing depths from the top of the cryobox to upper surfaces of the plurality of gratings; loading an RGB image of a plurality of data matrices corresponding to a plurality of columns stored in the plurality of grids; obtaining information related to an outermost point corresponding to the top of the cryobox from the RGB image; obtaining information about a grid surface corresponding to an upper surface of the plurality of grids using the RGB image and the depth; setting a center point of each of the plurality of lattices in the RGB image by equally dividing the lattice surface using the number of lattices; projecting and transforming a data matrix surface corresponding to the plurality of data matrices obtained from the RGB image into the grid surface by reflecting a preset parallax correction value; comparing a distance between each of the plurality of projection-transformed data matrices and a center point of each of the plurality of lattices; and setting coordinates of a central point closest to each of the plurality of projection-transformed data matrices as coordinates of each of the plurality of data matrices.

According to another aspect of the present invention, a computer readable program for performing the method described above is provided.

According to the present invention, since a plurality of data matrices and a plurality of grids are matched, a plurality of data matrices can be quickly and accurately recognized.

1 is a view showing a state in which cryo boxes and columns having a plurality of grids are accommodated.
2 is a diagram showing the configuration of a data matrix coordinate setting device according to an embodiment of the present invention.
3 is a diagram illustrating a coordinate setting process of the device according to the present embodiment.
4 to 5 are diagrams illustrating a cryobox configuration and various surfaces according to the present embodiment.
6 is a diagram for explaining a parallax correction value according to an embodiment of the present invention.
7 is a diagram for explaining a coordinate setting misrecognition process according to a direction in which a column is inserted into a grid.
8 is a view showing column vectors according to the insertion direction of the columns accommodated in the lattice.
9 is a flowchart illustrating a data matrix recognition process according to an embodiment of the present invention.
10 shows a binarized image obtained by sequentially changing the threshold value from 55 to 155 in units of 5.
11 is a diagram sequentially illustrating a process of estimating a data matrix from an RGB image according to an exemplary embodiment.

Since the present invention can make various changes and have various embodiments, specific embodiments are illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The present invention relates to a method and apparatus for matching and storing a data matrix attached to a plurality of columns to each of a plurality of grids.

1 is a view showing a state in which cryo boxes and columns having a plurality of grids are accommodated.

As shown in FIG. 1, a plurality of grids having predetermined heights and widths are formed in the cryobox, and at least some of the grids contain columns for storing samples, and data for sample identification are placed at the top of the columns. Matrix is attached.

When a plurality of columns are accommodated in the cryobox, it is not easy to recognize the data matrix attached to each cartridge at once due to the influence of tilt, roughness, gradient, and the like of the columns. In addition, even if a plurality of data matrices are recognized at the same time, a grid position corresponding to the corresponding data matrix must be stored in correspondence to a column required by the user in the future to be easily found.

To this end, the present invention matches the lattice corresponding to each data matrix using information about the lattice and an image of a cryobox.

Hereinafter, a process of setting the coordinates of the data matrix corresponding to the lattice will be first described, and decoding of the data matrix will be additionally described.

2 is a diagram showing the configuration of a data matrix coordinate setting device according to an embodiment of the present invention, and FIG. 3 is a diagram showing a coordinate setting process of the device according to this embodiment.

Referring to FIG. 2 , the apparatus for recognizing data matrix according to the present embodiment may include a processor 200 and a memory 202 .

Here, the processor 200 may include a central processing unit (CPU) capable of executing a computer program or other virtual machines.

Memory 202 may include a non-volatile storage device such as a non-removable hard drive or a removable storage device. The removable storage device may include a compact flash unit, a USB memory stick, and the like. Memory 202 may also include volatile memory, such as various random access memories.

Program instructions executable by the processor 200 are stored in such a memory 202 .

Program instructions according to this embodiment perform a process as shown in FIG. 3 .

Referring to FIG. 3 , the apparatus according to the present embodiment has the same number of lattices, lattice height, lattice width, and depth from the top of the cryobox to the upper surface of the plurality of lattices of the plurality of lattices formed in the cryobox. Grid-related information is stored (step 300).

As shown in FIG. 4 , here, the number of lattices is the number of lattices capable of accommodating columns, and the lattice height (h) and lattice width (w) are the height and horizontal/vertical size of individual lattices. In general, the grid is formed of a square, and horizontal and vertical widths are the same, but it is not necessarily limited thereto.

In addition, in a general cryobox, the upper surface of the grid is located at a position spaced apart from the top by a predetermined distance, and in the apparatus according to the present embodiment, the depth from the top of the cryobox to the upper surface of the plurality of grids is related to the grid. stored as information.

Next, the apparatus according to the present embodiment loads an RGB image obtained by photographing a plurality of data matrices corresponding to a plurality of columns stored in a plurality of grids (step 302).

As shown in FIG. 5 , the user may designate the outermost point corresponding to the top of the cryobox on the RGB image. Here, the outermost point is the NW point and the SE point that can define the boundary of the cryobox, and the device according to the present embodiment obtains information related to the outermost point according to the user's selection. (step 304).

Then, information on the lattice surfaces corresponding to the upper surfaces of the plurality of lattices is obtained using the RGB image and the depth from the top of the cryobox to the upper surfaces of the plurality of lattices (step 306).

As shown in FIGS. 4 and 5 , the lattice surface obtained in step 306 may be defined by coordinates corresponding to a plane corresponding to an upper surface of the lattice.

After the lattice surface is obtained, the lattice surface is evenly divided using the previously stored number of lattices to set the center point of each of the plurality of lattices on the RGB image (step 308).

Step 308 is a process of setting the center point coordinates of each grid on the RGB image.

The apparatus according to the present embodiment projects and transforms a data matrix surface (DM surface) corresponding to the plurality of data matrices obtained from an RGB image into the grid surface by reflecting a preset parallax correction value (step 310).

As shown in FIG. 4, the length of the column to which the data matrix is attached is generally greater than the height of the lattice, and accordingly, the plane (data matrix surface) on which the data matrix is located is higher than the lattice surface.

Considering this point, in step 310, the data matrix surface is projected and converted into a grid surface in consideration of the parallax caused by the camera image.

6 is a diagram for explaining a parallax correction value according to an embodiment of the present invention.

)에 따라 다음과 같이 표현된다. Referring to FIG. 6, the parallax (L) by camera shooting is the distance between the camera lens and the grating (H), the grating height (h), the grating width (w), the actual and apparent visual angles (

) is expressed as:

로 표현하면 다음과 같다. Equation 1

Expressed as:

From Equations 1 and 2, the time difference can be expressed as follows.

Equation 3 means that when the object moves from the projection center by w, the object center moves by k from the apparent center.

Here, the parallax correction value is defined as k, and k is determined by the vertical distance (H) of the camera lens and the plurality of gratings and the grating height (h).

In addition, generally, when the user shoots at a distance of 180 to 210 mm from the grating, that is, when H is 180 to 210 mm, the apparent center moves by 0.1262 to 0.1056 mm, and accordingly, the parallax correction value can be assumed to be a constant value. have.

In this way, the distance between each of the plurality of data matrices projected and transformed through the parallax correction value and the center point of each of the plurality of grids is compared (step 312), and the coordinates of the center point closest to each of the plurality of projection and transformed data matrices are described above. Coordinates of each of a plurality of data matrices are set (step 314).

According to a preferred embodiment of the present invention, the optimal height and width of the lattice can be set to prevent a coordinate setting error due to a column vector according to a direction in which a column stored in one of the plurality of lattices is inserted.

Here, the column vector may be defined as a vector connecting the center of the column bottom and the center point of the data matrix attached to the column in the actual grid.

7 is a diagram for explaining a coordinate setting misrecognition process according to a direction in which a column is inserted into a grid.

Referring to FIG. 7, when the column vector according to the direction in which the column is plugged and the parallax vector according to the shooting direction of the camera are identical, there is a possibility that an error may occur in which the data matrix is recognized as being close to the center point of another adjacent grid rather than the actual grid. It rises.

In FIG. 7, the distortion vector is a vector corresponding to a column on the RGB image, and is defined as the sum of the disparity vector and the column vector, and the correction vector is a vector obtained by removing the disparity vector through the parallax correction value.

8 is a view showing column vectors according to the insertion direction of the columns accommodated in the lattice.

라 할때, 컬럼이 해당 격자 내에 수납된 것으로 인식되기 위해서는 컬럼의 바닥 중심으로부터 데이터 매트릭스 중심점까지의 컬럼 벡터(

)가 격자 너비의 절반 이하여야 한다. Referring to FIG. 8, the lattice height and width are h and w, respectively, the diameter of the cylindrical column is d, the height is L, and the inclination angle with respect to the lattice is d.

, in order for the column to be recognized as being stored in the grid, the column vector from the center of the bottom of the column to the center point of the data matrix (

) should be less than half the grid width.

Expressing this as a formula is:

Here, since L is always greater than h, the following always holds.

는 컬럼의 직경, h, w에 종속되므로 다음 식을 만족할때 컬럼이 꽂힌 방향에 따라 좌표 설정 과정에서 오차가 발생하지 않게 된다. and,

is dependent on the diameter of the column, h, and w, so when the following equation is satisfied, no error occurs in the process of setting the coordinates according to the direction in which the column is inserted.

가 최대일 때,

를 다음과 같이 표현할 수 있다. In a relationship like Equation 6

When is maximum,

can be expressed as:

Substituting this into Equation 5 gives:

For example, if the height of a cylinder, which is a type of cryobox, is 41 mm, h is 30 mm, the radius of the column is 10 mm, and the grid width is 13, it can be numerically approximated within the range as follows. have.

When considering the difference between correction vectors for correcting errors due to parallax vectors and column vectors and other errors such as shooting angle errors of 10% (when the right term of Equation 8 is multiplied by 0.9), Equation 9 is as follows: is expressed

That is, when the grid height and width satisfy Equation 9 or 10, an error in which the data matrix attached to the corresponding column is recognized as close to the center point of another adjacent grid regardless of the column vector does not occur.

According to an embodiment of the present invention, prior to comparing the distance between each of the plurality of data matrices and the center point of each of the plurality of grids, the data matrix may be searched through generation of a binarized image, and the center point thereof may be determined.

According to the present embodiment, a binarization threshold is sequentially changed for an image captured by a cryobox to generate a binarized image, a shape corresponding to a data matrix is extracted for the binarized image generated at each threshold, and the extracted shape A data matrix recognition process, such as decoding for , may be performed.

9 is a flowchart illustrating a data matrix recognition process according to an embodiment of the present invention.

The apparatus according to the present embodiment loads an image of the cryobox taken from an upward direction (step 900).

The image in step 900 is an RGB image including a data matrix stored in a cryo box and attached to a plurality of columns as shown in FIG. 5 .

To improve the data matrix recognition speed, the loaded RGB image is resized (step 902).

For example, if the RGB image has a size of 4320×5760, in step 902, the width may be fixed to 1000, and the height may be resized to 1000×1333 by adjusting the same ratio.

A first binarization threshold is applied to the resized RGB image to generate a binarized image (step 904).

Here, the first binarization threshold is a value for converting each pixel of the resized RGB image to 0 or 1, and may be equally applied to R, G, and B, but is not necessarily limited thereto.

The device according to the present embodiment decodes at least some of a plurality of data matrices by sequentially changing the threshold for binarization. Preferably, the binarization threshold according to the present embodiment may be sequentially changed in predetermined units from a lower limit value to an upper limit value.

Preferably, the lower limit value may be set to 55 and the upper limit value may be set to 155, and the binarization threshold may be sequentially increased by 5 units.

Here, the reason why the upper limit value is set to 155 is that if the data matrix is within RGB (155, 155, 155) to RGB (255, 255, 255), the difference cannot be recognized and binarization is difficult.

The reason why the threshold for binarization is sequentially changed is that a data matrix of a specific area may not be recognized according to the threshold.

10 shows a binarized image obtained by sequentially changing the threshold value from 55 to 155 in units of 5.

Referring to FIG. 10, as the threshold increases sequentially, the recognized data matrix changes. For example, at a low threshold, a data matrix located below the cryo box is recognized, and as the threshold increases, the data matrix located above the cryo box is recognized. can be recognized.

Considering this point, the device according to the present embodiment performs data matrix recognition and decoding processes from a low threshold, and when the next threshold is applied, decoding is performed only on newly estimated data matrices excluding previously decoded data matrices. will perform

Referring back to FIG. 9 , a shape set corresponding to the shape of the data matrix is extracted from the binarized image obtained by applying the first binarization threshold (step 906).

Step 906 includes a process of extracting a set of points corresponding to the shape of the data matrix by extracting a set of points through contouring, removing adjacent points through approxPolyDP, and removing concave surfaces through convexHull (concave contour smoothing). .

As shown in FIGS. 4 and 5 , the data matrix may be a rectangle, and step 906 may be defined as a process of searching for a set of rectangles through a square approximation in a binarized image obtained by applying a predetermined binarization threshold.

After the shape corresponding to the data matrix is searched in step 906, the device according to the present embodiment estimates the data matrix using a preset rule (step 908) and decodes the estimated data matrix (step 910).

If the data matrix is a rectangle, step 906 is a process of searching all the rectangle sets recognized in the binarized image, and step 908 is a process of estimating an actual data matrix from the searched rectangle set.

According to this embodiment, the size and width/height ratio of the minimum area (minArea) and the maximum area (maxArea) for data matrix estimation may be set in advance.

The minimum area and the maximum area may be set by dividing the size of the RGB image resized in step 902 by a predetermined value in consideration of the size of a typical data matrix.

When the size of the resized RGB image is A, the minimum area may be set to A/B and the maximum area may be set to A/C.

Here, B=5 and C=2000, and in the above example, when the resized RGB image is 1000×1333, the minimum area may be 666 and the maximum area may be 266,000.

Also, the horizontal/vertical ratio may be set to 1.3.

The apparatus according to the present embodiment estimates a data matrix by estimating rectangles belonging to preset minimum and maximum regions among the rectangles extracted in step 906 and having a preset horizontal/vertical ratio.

After decoding of the estimated data matrix is completed, the data matrix recognition apparatus returns to step 904 to binarize the RGB image by applying the next threshold, and repeats steps 906 to 910.

In this case, the process of estimating and decoding the data matrix decoded in the previous process is omitted, so that the calculation speed can be increased.

More specifically, the apparatus for recognizing data matrices according to the present embodiment pre-stores a rectangular area (shape coordinates) corresponding to a data matrix decoded at a first binarization threshold, and stores a pre-stored rectangular area (decoded data matrix) at the next binarization threshold. ) to avoid duplicate recognition.

In this case, the shape coordinates of the decoded data matrix may be coordinates including a margin (eg, an area expanded by 10% of a rectangular area).

11 sequentially shows a process of estimating a data matrix from an RGB image according to the present embodiment, including resizing the RGB image, extracting a set of points from each binarized image, removing adjacent points and concave surfaces, and extracting a set of shapes corresponding to the data matrix. And data matrix can be recognized at once through data matrix estimation through minArea/maxArea.

The embodiments of the present invention described above have been disclosed for illustrative purposes, and those skilled in the art having ordinary knowledge of the present invention will be able to make various modifications, changes, and additions within the spirit and scope of the present invention, and such modifications, changes, and additions will be considered to fall within the scope of the following claims.

Claims (10)

As a data matrix coordinate setting device in a cryobox,
processor; and
Including a memory coupled to the processor,
the memory,
Storing the number of lattices, the lattice height, the lattice width, and the depth from the top of the cryobox to the upper surface of the plurality of lattices of the plurality of lattices formed in the cryobox;
Loading an RGB image of a plurality of data matrices corresponding to a plurality of columns stored in the plurality of grids;
Obtaining information related to an outermost point corresponding to the top of the cryobox from the RGB image;
Obtaining grating surfaces corresponding to upper surfaces of the plurality of gratings using the RGB image and the depth;
Setting the center point of each of the plurality of grids in the RGB image by equally dividing the grid surface using the number of grids;
Projecting and transforming a data matrix surface corresponding to the plurality of data matrices obtained from the RGB image into the grid surface by reflecting a preset parallax correction value;
comparing a distance between each of the projection-transformed data matrices and a center point of each of the plurality of lattices;
To set the coordinates of the central point closest to each of the plurality of projection-transformed data matrices to the coordinates of each of the plurality of data matrices,
An apparatus for setting data matrix coordinates in a cryobox storing program instructions executed by the processor. According to claim 1,
The parallax correction value is determined by the vertical distance (H) of the camera lens and the plurality of grids and the grid height (h) in the data matrix coordinate setting device in the cryobox. According to claim 2,
When H is located within a preset range, the parallax correction value is determined as a constant value. According to claim 3,
When H is in the range of 180 to 210 mm, the parallax correction value is determined as a constant value. According to claim 1,
The program instructions are
Generating a binarized image by applying a first binarization threshold to the RGB image;
Extracting a set of points included in the binarized image;
Extracting a first shape set corresponding to the shape of the data matrix by removing adjacent points and concave surfaces from the extracted point set;
Estimating one or more data matrices by applying a preset rule to the extracted first shape set;
An apparatus for setting data matrix coordinates in a cryobox for decoding the estimated one or more data matrices. According to claim 5,
The program instructions are
A binarization threshold having a value sequentially greater than the first binarization threshold is applied to the RGB image to generate a binarized image, and the point set extraction, the first shape set extraction, data matrix estimation, and decoding are repeatedly performed in a cryobox. Data Matrix coordinate setting device. According to claim 1,
The program instructions are
In order to prevent a coordinate setting error due to a column vector according to a storage direction of a column stored in one of the plurality of grids, the size of the column vector is set to less than half of the grid width in a data matrix coordinate setting device in a cryobox. . According to claim 7,
In order to prevent a coordinate setting error due to the column vector, the grid width (w) and the grid height (h) have a relation of the following equation.
[mathematical expression]

where d is the radius of the column and L is the height of the column A method for setting the coordinates of a data matrix in a cryobox in a device including a processor and a memory, comprising:
storing the number of lattices, the lattice height, the lattice width, and the depth from the top of the cryobox to the upper surface of the plurality of lattices of the plurality of lattices formed in the cryobox;
loading an RGB image of a plurality of data matrices corresponding to a plurality of columns stored in the plurality of grids;
obtaining information related to an outermost point corresponding to the top of the cryobox from the RGB image;
obtaining information about a grid surface corresponding to an upper surface of the plurality of grids using the RGB image and the depth;
setting a center point of each of the plurality of lattices in the RGB image by equally dividing the lattice surface using the number of lattices;
projecting and transforming a data matrix surface corresponding to the plurality of data matrices obtained from the RGB image into the grid surface by reflecting a preset parallax correction value;
comparing a distance between each of the plurality of projection-transformed data matrices and a center point of each of the plurality of lattices; and
and setting coordinates of a central point closest to each of the plurality of projection-transformed data matrices as coordinates of each of the plurality of data matrices. A computer readable program for performing the method according to claim 9 .

Images