KR102481756B1 - Apparatus and method for transforming contour point - Google Patents

Abstract

The present specification provides an apparatus for transforming a contour point and a method for transforming the contour point. At this time, the apparatus for transforming the contour point includes: a memory for storing a target image, which is an image obtained by capturing a plate material through a camera; and a processor for determining a plurality of contour points in the target image, determining a plate material plane in the target image, projecting the plurality of contour points to the plate material plane to determine projection points for one or more of the plurality of contour points, and, and calculating coordinates on the plate material plane for one or more of the plurality of contour points based on the projection points.

Description

Contour point conversion device and method thereof {APPARATUS AND METHOD FOR TRANSFORMING CONTOUR POINT}

The present invention relates to a contour point conversion device and method.

In various fields such as the shipbuilding industry or the woodworking industry, production and sorting of sheet metal parts requiring assembly are performed. At this time, in order to automate the task of classifying the plate member, an operation of identifying the plate member must first be performed.

In general, a method of leaving an identification mark on a plate material in advance may be used to identify the plate material. However, it may be impossible to leave an identification mark on the board in advance due to the surface properties of the board or problems in the manufacturing process.

Except for this case, the shape of the plate can be an important factor in the process of identifying the plate. For example, when identifying a plate material in a video image captured by a camera or the like, if the plate material is mixed with each other or the camera does not accurately point in a direction parallel to the normal vector of the plane of the plate material, projection distortion for the plate material seen on the video image this can happen

Embodiments provide a contour point conversion device and method capable of obtaining contour points from which projection distortion is removed from an image captured by a camera.

An embodiment includes obtaining a target image, which is an image of a plate material captured through a camera, determining a plurality of contour points in the target image, determining a plate material plane in the target image, and converting the plurality of contour points to a plate material plane. determining projected points for one or more of the plurality of contour points by projecting onto the contour points, and calculating coordinates on a plate material plane for one or more of the plurality of contour points based on the projected points. Provides a point conversion method.

Another embodiment is a memory for storing a target image, which is an image of a plate material captured through a camera; and determining projected points for one or more of the plurality of contour points by determining a plurality of contour points in the target image, determining a plate material plane in the target image, and projecting the plurality of contour points to the plate material plane, Provided is a contour point conversion device including a processor for calculating coordinates on a plate material plane for one or more of a plurality of contour points based on .

According to the contour point conversion apparatus and method according to the embodiments, it is possible to obtain contour points from which projection distortion is removed from an image captured by a camera.

1 is a configuration diagram of a contour point conversion device according to an exemplary embodiment.
FIG. 2 is a flowchart illustrating an example of an operation of the contour point conversion device of FIG. 1 .
3 is a diagram illustrating a method of determining a plate material plane by the contour point conversion apparatus according to an exemplary embodiment.
4 is a diagram illustrating a method of representing a plane of a plate material on a camera coordinate system according to an exemplary embodiment.
5 is a diagram illustrating a sample on a camera coordinate system for a plurality of contour points according to an exemplary embodiment.
6 is a diagram illustrating projection points for one or more of a plurality of contour points according to an exemplary embodiment.
7 is a diagram illustrating coordinates on a plate material plane for projected points according to an exemplary embodiment.
8 is a flowchart of a contour point conversion method according to an exemplary embodiment.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to components of each drawing, it should be noted that the same components have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

In addition, in describing the components of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, order, or order of the corresponding component is not limited by the term. When an element is described as being “connected,” “coupled to,” or “connected” to another element, that element is directly connected or connectable to the other element, but there is another element between the elements. It will be understood that elements may be “connected”, “coupled” or “connected”.

1 is a configuration diagram of a contour point conversion device 100 according to an exemplary embodiment.

Referring to FIG. 1 , the contour point conversion device 100 may include a memory 110 and a processor 120 .

The memory 110 may store a target image, which is an image obtained by capturing a plate material through the camera 200 .

The camera 200 may be a general digital camera or a special camera (e.g. ToF camera) capable of acquiring additional information about an image while capturing an image.

The target image may be one or more still images or moving images. The target video can have various formats (e.g. JPG, PNG, AVI, MP4).

The memory 110 may be a volatile memory (e.g. SRAM, DRAM) or a non-volatile memory (e.g. NAND flash, NOR flash).

The processor 120 determines a plurality of contour points in the target image based on the target image stored in the memory 110, and determines projection points generated by projecting one or more of the plurality of contour points onto a plate material plane, , the coordinates on the plate plane for the projected points can be calculated.

The processor 120 may drive firmware stored in the above-described memory 110 or other storage devices (e.g. HDD, SSD, SD Card) in order to execute the above-described operation. The processor 120 may execute the above-described operation by executing logical operations defined in firmware.

The processor 120 may be implemented as a microprocessor, CPU, system on chip (SoC), or the like.

Hereinafter, specific operations of the contour point conversion device 100 will be described.

FIG. 2 is a flowchart illustrating an example of an operation of the contour point conversion device 100 of FIG. 1 .

Referring to FIG. 2 , the processor 120 of the contour point conversion apparatus 100 may determine a plurality of contour points from a target image captured by the camera 200 (S210).

The processor 120 may use various algorithms to determine the plurality of contour points in the target image. For example, the processor 120 may determine a plurality of contour points using Canny Edge Detection and findContour of OpenCV or VNContourObservation of AppleVisionKit.

Also, the processor 120 may determine a plate material plane in the target image (S220). The plate material plane represents a two-dimensional plane corresponding to the plate material in the target image.

For example, the plate material plane may be determined as a virtual plane determined to be most suitable for the position of the plate material. The actual plate material may not be positioned on a complete two-dimensional plane, and in this case, the plate material plane may be a plane determined to most closely match the plate material.

Processor 120 may use various algorithms to determine the planar plane. For example, the processor 120 may determine the plate material plane using FindSurface, Hough Transform, RANSAC, or the least squares method.

Further, the processor 120 may determine one or more projected points by projecting one or more of the plurality of contour points determined in step S210 onto the plate material plane determined in step S220 (S230).

Further, the processor 120 may calculate coordinates on the plate material plane for the projected points based on the projected points determined in step S230 (S240).

Hereinafter, detailed embodiments of each step of the operation of the above-described contour point conversion device 100 will be described.

3 is a diagram illustrating a method of determining a plate material plane by the contour point conversion apparatus 100 according to an exemplary embodiment.

Referring to FIG. 3 , the processor 120 of the contour point conversion apparatus 100 obtains a plate material plane from the camera 200 or a plurality of consecutive images extracted from a target image captured by the camera 200. A plate plane can be determined based on depth information.

In this case, the depth information may include a depth value of one or more of pixels corresponding to the target image. For example, the depth information may include a depth value of 10 for the pixel (x1, y1), a depth value of 0 for the pixel (x2, y2), and a depth value of 5 for the pixel (x3, y3).

For example, the camera 200 may be a depth camera (e.g. Time of Flight (ToF) camera) capable of directly providing depth information on a target image. In this case, the processor 120 may directly obtain depth information from the camera 200 .

As another example, the processor 120 may obtain depth information from a plurality of consecutive images extracted from a target image photographed by the camera 200 according to a predefined depth information extraction method.

In this case, the plurality of consecutive images may be images continuously captured by the camera 200 at regular time intervals (eg, 1/1000 second).

Also, the method for extracting depth information may be, for example, Depth API (https://developers.google.com/ar/develop/java/depth/introduction#depth_images) provided by Android.

4 is a diagram illustrating a method of representing a plane of a plate material on a camera coordinate system according to an exemplary embodiment.

Referring to FIG. 4 , the camera coordinate system has the position of the camera 200 as the origin, the horizontal axis of the target image, the x-axis, the vertical axis of the target image, the y-axis, and the camera 200's lens viewing axis, z. axis can be determined. At this time, the x-axis, y-axis, and z-axis are perpendicular to each other.

For example, the position of the camera 200 may be determined as the center of the pinhole lens.

로 결정될 수 있다.The plate material plane satisfies the following Equation 1 in the camera coordinate system

can be determined by

은 판재 평면의 법선 벡터를 의미한다. 그리고

은 판재 평면 위의 임의의 한 점을 의미한다.At this time,

means the normal vector of the plate plane. And

means an arbitrary point on the plate plane.

5 is a diagram illustrating a sample on a camera coordinate system for a plurality of contour points according to an exemplary embodiment.

는 다음과 같이 표현될 수 있다.Referring to FIG. 5 , image coordinates of pixels constituting a target image

can be expressed as

는 타깃 영상의 가로 방향 해상도이고

는 타깃 영상의 세로 방향 해상도이다. 일 예로, 640 * 480의 해상도로 촬영된 타깃 영상에서

일 수 있다.At this time,

is the horizontal resolution of the target image and

is the vertical resolution of the target image. For example, in a target image captured at a resolution of 640 * 480

can be

는 다음과 같이 정의될 수 있다.A plurality of contour points in the target image

can be defined as:

는 복수의 윤곽점들 중 하나에 해당하는 이미지 좌표를 나타낸다.here,

represents an image coordinate corresponding to one of a plurality of contour points.

에 대하여 이를 카메라 좌표계 상의 좌표로 변환하기 위해 샘플

을 다음의 수학식 2로 계산할 수 있다.Processor 120 is expressed in image coordinates of pixels

For a sample to convert it to coordinates on the camera coordinate system

can be calculated by Equation 2 below.

는 카메라(200)에 대한 카메라 고유 행렬(camera intrinsic matrix)로서,

로 정의될 수 있다. 카메라 고유 행렬

은 카메라(200)의 상태를 기초로 결정될 수 있다.At this time,

Is the camera intrinsic matrix for the camera 200,

can be defined as camera eigen matrix

may be determined based on the state of the camera 200 .

는 카메라(200)의 수평 초점거리이고,

는 카메라(200)의 수직 초점거리이다.

is the horizontal focal length of the camera 200,

is the vertical focal length of the camera 200.

는 미리 설정된 비대칭 계수(skew coefficient)이다.

는 타깃 영상에서 카메라(200)의 투영 중심점(CoP, center of projection)의 x축 상의 좌표값이고,

는 타깃 영상에서 투영 중심점의 y축 상의 좌표값이다.

Is a preset skew coefficient.

Is a coordinate value on the x-axis of the center of projection (CoP) of the camera 200 in the target image,

is a coordinate value on the y-axis of the projection center point in the target image.

6 is a diagram illustrating projection points for one or more of a plurality of contour points according to an exemplary embodiment.

Referring to FIG. 6 , one or more of a plurality of contour points may be projected onto a plane of the plate material.

At this time, in order to determine a projected point where one or more of the plurality of contour points is projected onto the plate material plane, it is necessary to determine an intersection point formed by a straight line extending from the camera to the plate material plane and the plate material plane.

Meanwhile, in embodiments of the present invention, the processor 120 may determine the aforementioned intersection point differently according to how many pixels the depth information includes among pixels included in the target image.

First, a case in which the depth information includes the depth of pixels less than a threshold among pixels included in the target image will be described.

에서 전술한 샘플

를 향하는 방향 벡터

를 다음과 같이 결정할 수 있다. 카메라 좌표계에서 카메라(200)의 위치가 원점으로 정해지므로 위치 벡터

는 영(zero) 벡터일 수 있다.At this time, the processor 120 generates a position vector representing the position of the camera 200 on the camera coordinate system.

the sample described above in

direction vector towards

can be determined as follows. Since the position of the camera 200 is determined as the origin in the camera coordinate system, the position vector

may be a zero vector.

에서 방향 벡터

방향으로 뻗은 직선이 판재 평면과 이루는 교점

을 다음과 같이 결정할 수 있다.At this time, the processor 120 generates a position vector of the camera.

direction vector from

The point of intersection of a straight line extending in the direction

can be determined as follows.

에서 방향 벡터

방향으로 뻗은 직선 위의 임의의 점은 다음과 같이 결정될 수 있다.First, the position vector of the camera

direction vector from

Any point on a straight line extending in the direction can be determined as

에 대입하면

는 다음과 같이 도출된다.Equation of the plate plane described above

Substituting into

is derived as follows.

를

에 대입하면, 카메라의 위치 벡터

에서 방향 벡터

방향으로 뻗은 직선이 판재 평면과 이루는 교점

는 다음과 같이 수학식 3으로 결정될 수 있다.calculated

cast

Substituting into , the position vector of the camera

direction vector from

The point of intersection of a straight line extending in the direction

Can be determined by Equation 3 as follows.

Next, a case in which the depth information includes depths of pixels greater than or equal to a threshold value among pixels included in the target image will be described.

에 대한 카메라 좌표계 상의 좌표인

를 다음과 같이 결정할 수 있다.At this time, the processor 120 is

is a coordinate on the camera coordinate system for

can be determined as follows.

는 타깃 영상에서

에 대응되는 위치의 좌표를 의미한다.here

in the target image

It means the coordinates of the position corresponding to .

는

에서 카메라(200)의 투영 방향과 수직인 평면까지의 수직 거리를 의미한다.And

Is

It means the vertical distance from to the plane perpendicular to the projection direction of the camera 200.

에서, 전술한

를 향하는 방향 벡터

를 다음과 같이 결정할 수 있다.At this time, the processor 120 generates a position vector of the camera.

in the above

direction vector towards

can be determined as follows.

에서 방향 벡터

방향인 직선이 판재 평면과 이루는 교점

을 다음과 같이 수학식 4로 결정할 수 있다.At this time, the processor 120 generates a position vector of the camera.

direction vector from

The point of intersection of the straight line with the plane of the sheet

can be determined by Equation 4 as follows.

와

의 거리가 설정된 임계 거리를 초과하는 경우에는, 프로세서(120)는

에 대응하는 윤곽점이 판재 평면에 대응하지 않는다고 판단하고,

를 판재 평면에 존재하는 투영점에서 제외할 수 있다.However, at this time, the aforementioned

Wow

When the distance of exceeds the set threshold distance, the processor 120

It is determined that the contour point corresponding to does not correspond to the plate material plane,

can be excluded from projection points existing on the plate plane.

7 is a diagram illustrating coordinates on a plate material plane for projected points according to an exemplary embodiment.

Referring to FIG. 7 , projected points are located on the plate material plane. Points located on the plate material plane may be expressed as two-dimensional coordinates having two mutually perpendicular straight lines A and B located on the plate material plane as axes.

및 판재 평면에 평행한 단위 벡터

에 수직인 벡터

를 다음과 같이 구할 수 있다.In embodiments of the present invention, the processor 120 first calculates the normal vector of the plane of the plank.

and a unit vector parallel to the plate plane.

vector perpendicular to

can be obtained as:

및, 판재 평면 상의 한 점

을 기초로 하여, 카메라 좌표계 상의 점을 판재 평면 상의 좌표로 변환하는 행렬

을 다음과 같이 구할 수 있다.The processor 120 is described above

and, a point on the plate plane

Matrix for converting points on the camera coordinate system to coordinates on the plate plane based on

can be obtained as:

및

를 기초로 하여, 투영점들에 대한 판재 평면 상에서의 좌표

를 다음과 같이 결정할 수 있다.The processor 120 is described above

and

Coordinates on the plate plane for projection points, based on

can be determined as follows.

8 is a flowchart of a contour point conversion method according to an exemplary embodiment.

Referring to FIG. 8 , the contour point conversion method may first include acquiring a target image, which is an image obtained by capturing a plate material through a camera 200 (S810).

And, the contour point conversion method may include determining a plurality of contour points in the target image acquired in step S810 (S820).

And, the contour point conversion method may include determining a plate material plane in the target image acquired in step S810 (S830).

In operation S830 , for example, a plate material plane may be determined based on depth information extracted from i) a camera or ii) a plurality of consecutive images extracted from a target image.

로 결정될 수 있다.The plate plane is the camera coordinate system, which is a coordinate system determined by the x-axis, which is the horizontal axis of the target image, the y-axis, which is the vertical axis of the target image, and the z-axis, which is the direction axis of the camera lens, with the camera position as the origin. satisfied

can be determined by

은 판재 평면의 법선 벡터이고,

은 판재 평면 위의 임의의 한 점이다.At this time,

is the normal vector of the plate plane,

is an arbitrary point on the plate plane.

The contour point conversion method may include a step of determining projected points for one or more of the plurality of contour points by projecting the plurality of contour points determined in step S820 onto the plane of the plate (S840).

의 카메라 좌표계 상의 좌표를 구하기 위한 샘플

을 다음과 같이 계산할 수 있다.In step S840, image coordinates corresponding to one of the plurality of contour points

Sample for obtaining coordinates on the camera coordinate system of

can be calculated as:

이고,here

ego,

는 카메라(200)의 수평 초점거리이고,

is the horizontal focal length of the camera 200,

는 카메라(200)의 수직 초점거리이고,

is the vertical focal length of the camera 200,

는 카메라(200)의 비대칭 계수(skew coefficient)이고,

is the skew coefficient of the camera 200,

는 카메라(200)의 투영 중심점의 x축 상의 좌표값이고,

Is the coordinate value on the x-axis of the projection center point of the camera 200,

는 투영 중심점의 y축 상의 좌표값이다.

is the coordinate value on the y-axis of the projection center point.

에서 샘플

를 향하는 방향 벡터

를 다음과 같이 결정할 수 있다.For example, in step S840, when the depth information includes depth information on a pixel less than a set threshold among pixels included in the target image, a position vector indicating a position of the camera on the camera coordinate system.

sample from

direction vector towards

can be determined as follows.

에서 방향 벡터

방향으로 뻗은 직선이 판재 평면과 이루는 교점

을 다음과 같이 결정할 수 있다.And step S840, position vector

direction vector from

The point of intersection of a straight line extending in the direction

can be determined as follows.

에 대한 카메라 좌표계 상의 좌표인

를 다음과 같이 결정할 수 있다.As another example, in step S840, when the depth information includes depth information about a pixel equal to or greater than a set threshold among pixels included in the target image,

is a coordinate on the camera coordinate system for

can be determined as follows.

는 타깃 영상에서

에 대응되는 위치의 좌표이고,

는

에서 z축과 수직인 평면까지의 수직 거리이다.At this time,

in the target image

is the coordinates of the position corresponding to

Is

is the vertical distance from to the plane perpendicular to the z-axis.

에서,

를 향하는 방향 벡터

를 다음과 같이 결정할 수 있다.And step S840, the position vector indicating the position of the camera on the camera coordinate system

at,

direction vector towards

can be determined as follows.

에서 방향 벡터

방향인 직선이 판재 평면과 이루는 교점

을 다음과 같이 결정할 수 있다.And step S840, position vector

direction vector from

The point of intersection of the straight line with the plane of the sheet

can be determined as follows.

와

의 거리가 설정된 임계 거리를 초과하는 경우,

는 투영점들에서 제외될 수 있다.Meanwhile,

Wow

If the distance of exceeds the set critical distance,

can be excluded from projection points.

Further, the contour point conversion method may include calculating coordinates on the plate material plane for one or more of the plurality of contour points based on the projected points determined in step S840 (S850).

및 판재 평면에 평행한 단위 벡터

에 수직인 벡터

를 다음과 같이 구할 수 있다.Step S850 is the normal vector of the plate plane.

and a unit vector parallel to the plate plane.

vector perpendicular to

can be obtained as:

및 판재 평면 상의 한 점

을 기초로 카메라 좌표계 상의 점을 판재 평면 상에서의 좌표로 변환하는 행렬

을 다음과 같이 구할 수 있다.And step S850,

and a point on the plane of the plate

A matrix that converts points on the camera coordinate system to coordinates on the plate plane based on

can be obtained as:

를 다음과 같이 결정할 수 있다.And step S850, the coordinates on the plate material plane for the projected points

can be determined as follows.

The present embodiments described above may be implemented through various means. For example, the present embodiments may be implemented by hardware, firmware, software, or a combination thereof.

In the case of hardware implementation, the contour point conversion device and method according to the present embodiments include one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), PLDs ( It can be implemented by Programmable Logic Devices), Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers or microprocessors.

In the case of implementation by firmware or software, the contour point conversion method according to the present embodiments may be implemented in the form of a device, procedure, or function that performs the functions or operations described above. The software codes may be stored in a memory unit and driven by a processor. The memory unit may be located inside or outside the processor and exchange data with the processor by various means known in the art.

Also, the terms "system", "processor", "controller", "component", "module", "interface", "model", or "unit" as described above generally refer to computer-related entities hardware, hardware and software. can mean a combination of, software, or running software. For example, but is not limited to, a process driven by a processor, a processor, a controller, a control processor, an object, a thread of execution, a program, and/or a computer. For example, a component can be both an application running on a controller or processor and a controller or processor. One or more components may reside within a process and/or thread of execution, and components may reside on one device (eg, system, computing device, etc.) or may be distributed across two or more devices.

Meanwhile, another embodiment provides a computer program stored in a computer recording medium that performs the above-described contour point conversion method. In addition, another embodiment provides a computer-readable recording medium on which a program for realizing the above-described contour point conversion method is recorded.

A program recorded on a recording medium may be read, installed, and executed in a computer to execute the above-described steps.

In this way, in order for the computer to read the program recorded on the recording medium and execute the functions implemented by the program, the above-described program is C, C++ that can be read by the computer's processor (CPU) through the computer's device interface. , JAVA, may include a code coded in a computer language such as machine language.

These codes may include functional codes related to functions defining the above-described functions, and may include control codes related to execution procedures necessary for a processor of a computer to execute the above-described functions according to a predetermined procedure.

In addition, these codes may further include memory reference related codes for which location (address address) of the computer's internal or external memory should be referenced for additional information or media necessary for the computer's processor to execute the above-mentioned functions. .

In addition, when the computer processor needs to communicate with any other remote computer or server in order to execute the above-mentioned functions, the code allows the computer processor to use the computer's communication module to communicate with any other remote computer or server. Communication-related codes for how to communicate with other computers or servers, what information or media to transmit/receive during communication, and the like may be further included.

Recording media that can be read by a computer on which the program as described above is recorded are, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical media storage device, etc., and also carrier wave (e.g. , Transmission through the Internet) may also include what is implemented in the form of.

In addition, the computer-readable recording medium is distributed in computer systems connected through a network, so that computer-readable codes can be stored and executed in a distributed manner.

In addition, a functional program for implementing the present invention, codes and code segments related thereto, in consideration of the system environment of a computer that reads a recording medium and executes a program, etc., help programmers in the art to which the present invention belongs It may be easily inferred or changed by

In the above, even though all the components constituting the embodiment of the present invention have been described as being combined or operated as one, the present invention is not necessarily limited to these embodiments. That is, within the scope of the object of the present invention, all of the components may be selectively combined with one or more to operate.

Terms such as "comprise", "comprise" or "having" described above mean that the corresponding component may be inherent unless otherwise stated, and therefore do not exclude other components. It should be construed that it may further include other components. All terms, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless defined otherwise. Commonly used terms, such as terms defined in a dictionary, should be interpreted as consistent with the meaning in the context of the related art, and unless explicitly defined in the present invention, they are not interpreted in an ideal or excessively formal meaning.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

100: contour point converter
110: memory
120: processor

Claims (16)

Obtaining a target image, which is an image obtained by capturing a plate material through a camera;
determining a plurality of contour points in the target image;
determining a plate material plane that is a two-dimensional plane corresponding to the plate material in the target image;
determining one or more projected points generated by projecting one or more of the plurality of contour points onto the plate material plane; and
Calculating coordinates on the plate material plane for the projected points;
Determining the plate material plane in the target image,
i) determining the plate material plane based on depth information extracted from the camera or ii) a plurality of consecutive images extracted from the target image;
The plate plane is
On the camera coordinate system, which is a coordinate system determined by the x-axis, which is the horizontal axis of the target image, the y-axis, which is the vertical axis of the target image, and the z-axis, which is the axis in the direction in which the lens of the camera is viewed, with the position of the camera as the origin, the following satisfied

is determined by

remind

Is the normal vector of the plate material plane,
remind

Is an arbitrary point on the plate material plane,
The step of determining the projected points is,
Image coordinates corresponding to one of the plurality of contour points

Sample for obtaining coordinates on the camera coordinate system of

is calculated as follows,

here

ego,

is the horizontal focal length of the camera,

is the vertical focal length of the camera,

is the skew coefficient of the camera,

Is the coordinate value on the x-axis of the projection center point of the camera,

is a coordinate value on the y-axis of the projection center point, the contour point conversion method.
delete delete delete According to claim 1,
The step of determining the projected points is,
When the depth information includes depth information on pixels less than a set threshold among pixels included in the target image,
A position vector representing the position of the camera on the camera coordinate system

from the above sample

direction vector towards

is determined as follows,

the position vector

the direction vector from

The point of intersection of the straight line extending in the direction and the plane of the plate

Contour point conversion method, which determines as follows.

According to claim 1,
The step of determining the projected points is,
When the depth information includes depth information about a pixel equal to or greater than a set threshold among pixels included in the target image,
remind

Coordinates on the camera coordinate system for

is determined as follows,

In the target image,

is the coordinates of the position corresponding to

said

is the vertical distance from to the plane perpendicular to the z-axis,
A position vector representing the position of the camera on the camera coordinate system

in the above

direction vector towards

is determined as follows,

the position vector

the direction vector from

The point of intersection of the straight line in the direction and the plane of the plate

Contour point conversion method, which determines as follows.

According to claim 6,
remind

and above

If the distance of exceeds the set threshold distance, the

Is excluded from the projection points, the contour point conversion method.
According to claim 1,
Calculating coordinates on the plate material plane for one or more of the plurality of contour points,
Normal vector of the plate plane

and a unit vector parallel to the plate plane.

vector perpendicular to

is obtained as follows,

remind

and a point on the plate plane.

A matrix for converting points on the camera coordinate system to coordinates on the plate material plane based on

is obtained as follows,

Coordinates on the plate plane for the projected points

Contour point conversion method for determining as follows.

a memory for storing a target image, which is an image obtained by capturing a plate material through a camera; and
determining a plurality of contour points in the target image;
Determining a plate material plane, which is a two-dimensional plane corresponding to the plate material, in the target image;
determining one or more projection points generated by projecting one or more of the plurality of contour points onto the plate material plane;
A processor for calculating coordinates of the projected points on the plate material plane;
the processor,
i) determining the plate material plane based on depth information obtained from the camera or ii) a plurality of consecutive images extracted from the target image;
The plate plane is
On the camera coordinate system, which is a coordinate system determined by the x-axis, which is the horizontal axis of the target image, the y-axis, which is the vertical axis of the target image, and the z-axis, which is the axis in the direction in which the lens of the camera is viewed, with the position of the camera as the origin, the following satisfied

is determined by

remind

Is the normal vector of the plate material plane,
remind

Is an arbitrary point on the plate material plane,
the processor,
Image coordinates corresponding to one of the plurality of contour points

Sample for obtaining coordinates on the camera coordinate system of

is calculated as follows,

here

ego,

is the horizontal focal length of the camera,

is the vertical focal length of the camera,

is the skew coefficient of the camera,

Is the coordinate value on the x-axis of the projection center point of the camera,

is a coordinate value of the projection center point on the y-axis.
delete delete delete According to claim 9,
the processor,
When the depth information includes a depth of a pixel less than a set threshold among pixels included in the target image,
A position vector representing the position of the camera on the camera coordinate system

from the above sample

direction vector towards

is determined as follows,

the position vector

the direction vector from

The point of intersection of the straight line extending in the direction and the plane of the plate

A contour point conversion device that determines as follows.

According to claim 9,
the processor,
When the depth information includes a depth of a pixel greater than or equal to a set threshold among pixels included in the target image,
remind

Coordinates on the camera coordinate system for

is determined as follows,

In the target image,

is the coordinates of the position corresponding to

said

is the vertical distance from to the plane perpendicular to the z-axis,
A position vector representing the position of the camera on the camera coordinate system

in the above

direction vector towards

is determined as follows,

the position vector

the direction vector from

The point of intersection of the straight line in the direction and the plane of the plate

A contour point conversion device that determines as follows.

According to claim 14,
remind

and above

If the distance of exceeds the set threshold distance, the

Is excluded from the projected points, the contour point conversion device.
According to claim 9,
the processor,
Normal vector of the plate plane

and a unit vector parallel to the plate plane.

vector perpendicular to

is obtained as follows,

remind

and a point on the plate plane.

A matrix for converting points on the camera coordinate system to coordinates on the plate material plane based on

is obtained as follows,

Coordinates on the plate plane for the projected points

A contour point conversion device that determines as follows.

Images