WO2023106472A1

WO2023106472A1 - Contour point transformation apparatus and method therefor

Info

Publication number: WO2023106472A1
Application number: PCT/KR2021/018782
Authority: WO
Inventors: 김성구
Original assignee: 주식회사 커브서프
Priority date: 2021-12-09
Filing date: 2021-12-10
Publication date: 2023-06-15
Also published as: KR102481756B1

Abstract

The present specification provides a contour point transformation apparatus and a contour point transformation method. Herein, the contour point transformation apparatus comprises: a memory storing a target image, which is an image obtained by capturing a board through a camera; and a processor, which determines a plurality of contour points in the target image, determines a board plane in the target image, projects the plurality of contour points onto the board plane so as to determine projection points for one or more of the plurality of contour points, and calculates the coordinates on the board plane of the one or more of the plurality of contour points on the basis of the projection points.

Description

Contour point conversion device and method

The present invention relates to a contour point conversion device and method.

In various fields such as the shipbuilding industry or the woodworking industry, production and sorting of sheet metal parts requiring assembly are performed. At this time, in order to automate the task of classifying the plate member, an operation of identifying the plate member must first be performed.

In general, a method of leaving an identification mark on a plate material in advance may be used to identify the plate material. However, it may be impossible to leave an identification mark on the board in advance due to the surface properties of the board or problems in the manufacturing process.

Except for these cases, the shape of a plate can be an important factor in the process of identifying a plate. For example, when identifying a plate material in a video image captured by a camera or the like, if the plate material is mixed with each other or the camera does not point in a direction exactly parallel to the normal vector of the plane of the plate material, projection distortion for the plate material seen on the video image this can happen

Embodiments provide a contour point conversion device and method capable of obtaining contour points from which projection distortion is removed from an image captured by a camera.

An embodiment includes obtaining a target image, which is an image of a plate material captured through a camera, determining a plurality of contour points in the target image, determining a plate material plane in the target image, and converting the plurality of contour points to a plate material plane. determining projected points for one or more of the plurality of contour points by projecting onto the contour points, and calculating coordinates on a plate material plane for one or more of the plurality of contour points based on the projected points. Provides a point conversion method.

Another embodiment is a memory for storing a target image, which is an image of a plate material captured through a camera; and determining projected points for one or more of the plurality of contour points by determining a plurality of contour points in the target image, determining a plate material plane in the target image, and projecting the plurality of contour points to the plate material plane, Provided is a contour point conversion device including a processor for calculating coordinates on a plate material plane for one or more of a plurality of contour points based on .

According to the contour point conversion apparatus and method according to the embodiments, it is possible to obtain contour points from which projection distortion is removed from an image captured by a camera.

1 is a configuration diagram of a contour point conversion device according to an exemplary embodiment.

FIG. 2 is a flowchart illustrating an example of an operation of the contour point conversion device of FIG. 1 .

3 is a diagram illustrating a method of determining a plate material plane by the contour point conversion apparatus according to an exemplary embodiment.

4 is a diagram illustrating a method of representing a plane of a plate material on a camera coordinate system according to an exemplary embodiment.

5 is a diagram illustrating a sample on a camera coordinate system for a plurality of contour points according to an exemplary embodiment.

6 is a diagram illustrating projection points for one or more of a plurality of contour points according to an exemplary embodiment.

7 is a diagram illustrating coordinates on a plate material plane for projected points according to an exemplary embodiment.

8 is a flowchart of a contour point conversion method according to an exemplary embodiment.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to components of each drawing, it should be noted that the same components have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

In addition, in describing the components of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, order, or order of the corresponding component is not limited by the term. When an element is described as being “connected,” “coupled to,” or “connected” to another element, that element is directly connected or connectable to the other element, but there is another element between the elements. It will be understood that elements may be “connected”, “coupled” or “connected”.

1 is a configuration diagram of a contour point conversion device 100 according to an exemplary embodiment.

Referring to FIG. 1 , the contour point conversion device 100 may include a memory 110 and a processor 120 .

The memory 110 may store a target image, which is an image obtained by capturing a plate material through the camera 200 .

The camera 200 may be a general digital camera or a special camera (e.g. ToF camera) capable of acquiring additional information about an image while capturing an image.

The target image may be one or more still images or moving images. The target video can have various formats (e.g. JPG, PNG, AVI, MP4).

The memory 110 may be a volatile memory (e.g. SRAM, DRAM) or a non-volatile memory (e.g. NAND flash, NOR flash).

The processor 120 determines a plurality of contour points in the target image based on the target image stored in the memory 110, and determines projection points generated by projecting one or more of the plurality of contour points onto a plate material plane, , the coordinates on the plate plane for the projected points can be calculated.

The processor 120 may drive firmware stored in the above-described memory 110 or other storage devices (e.g. HDD, SSD, SD Card) in order to execute the above-described operation. The processor 120 may execute the above-described operation by executing logical operations defined in firmware.

The processor 120 may be implemented as a microprocessor, CPU, system on chip (SoC), or the like.

Hereinafter, specific operations of the contour point conversion device 100 will be described.

FIG. 2 is a flowchart illustrating an example of an operation of the contour point conversion device 100 of FIG. 1 .

Referring to FIG. 2 , the processor 120 of the contour point conversion apparatus 100 may determine a plurality of contour points from a target image captured by the camera 200 (S210).

The processor 120 may use various algorithms to determine the plurality of contour points in the target image. For example, the processor 120 may determine a plurality of contour points using Canny Edge Detection and findContour of OpenCV or VNContourObservation of AppleVisionKit.

Also, the processor 120 may determine a plate material plane in the target image (S220). The plate material plane represents a two-dimensional plane corresponding to the plate material in the target image.

For example, the plate material plane may be determined as a virtual plane determined to be most suitable for the position of the plate material. The actual plate material may not be positioned on a complete two-dimensional plane, and in this case, the plate material plane may be a plane determined to most closely match the plate material.

Processor 120 may use various algorithms to determine the planar plane. For example, the processor 120 may determine the plate material plane using FindSurface, Hough Transform, RANSAC, or the least squares method.

Further, the processor 120 may determine one or more projected points by projecting one or more of the plurality of contour points determined in step S210 onto the plate material plane determined in step S220 (S230).

Further, the processor 120 may calculate coordinates on the plate material plane for the projected points based on the projected points determined in step S230 (S240).

Hereinafter, detailed embodiments of each step of the operation of the above-described contour point conversion device 100 will be described.

3 is a diagram illustrating a method of determining a plate material plane by the contour point conversion apparatus 100 according to an exemplary embodiment.

Referring to FIG. 3 , the processor 120 of the contour point conversion apparatus 100 obtains a plate material plane from the camera 200 or a plurality of consecutive images extracted from a target image captured by the camera 200. A plate plane can be determined based on depth information.

In this case, the depth information may include a depth value of one or more of pixels corresponding to the target image. For example, the depth information may include a depth value of 10 for the pixel (x1, y1), a depth value of 0 for the pixel (x2, y2), and a depth value of 5 for the pixel (x3, y3).

For example, the camera 200 may be a depth camera (e.g. Time of Flight (ToF) camera) capable of directly providing depth information on a target image. In this case, the processor 120 may directly obtain depth information from the camera 200 .

As another example, the processor 120 may obtain depth information from a plurality of consecutive images extracted from a target image photographed by the camera 200 according to a predefined depth information extraction method.

In this case, the plurality of consecutive images may be images continuously captured by the camera 200 at regular time intervals (eg, 1/1000 second).

Also, the method for extracting depth information may be, for example, Depth API (https://developers.google.com/ar/develop/java/depth/introduction#depth_images) provided by Android.

Referring to FIG. 4 , the camera coordinate system has the position of the camera 200 as the origin, the horizontal axis of the target image, the x-axis, the vertical axis of the target image, the y-axis, and the camera 200's lens viewing axis, z. axis can be determined. At this time, the x-axis, y-axis, and z-axis are perpendicular to each other.

For example, the position of the camera 200 may be determined as the center of the pinhole lens.

The plate material plane satisfies the following Equation 1 in the camera coordinate system

can be determined by

[Equation 1]

At this time,

means the normal vector of the plate plane. and

means an arbitrary point on the plate plane.

Referring to FIG. 5 , image coordinates of pixels constituting a target image

can be expressed as

At this time,

is the horizontal resolution of the target image and

is the vertical resolution of the target image. For example, in a target image captured at a resolution of 640 * 480

can be

A plurality of contour points in the target image

can be defined as:

here,

represents an image coordinate corresponding to one of a plurality of contour points.

Processor 120 is expressed in image coordinates of pixels

For a sample to convert it to coordinates on the camera coordinate system

can be calculated by Equation 2 below.

[Equation 2]

At this time,

Is the camera intrinsic matrix for the camera 200,

can be defined as camera eigen matrix

may be determined based on the state of the camera 200 .

is the horizontal focal length of the camera 200,

is the vertical focal length of the camera 200.

Is a preset skew coefficient.

Is a coordinate value on the x-axis of the center of projection (CoP) of the camera 200 in the target image,

is a coordinate value on the y-axis of the projection center point in the target image.

Referring to FIG. 6 , one or more of a plurality of contour points may be projected onto a plane of the plate material.

At this time, in order to determine a projected point where one or more of the plurality of contour points is projected onto the plate material plane, it is necessary to determine an intersection point formed by a straight line extending from the camera to the plate material plane and the plate material plane.

Meanwhile, in embodiments of the present invention, the processor 120 may determine the aforementioned intersection point differently according to how many pixels the depth information includes among pixels included in the target image.

First, a case in which the depth information includes the depth of pixels less than a threshold among pixels included in the target image will be described.

At this time, the processor 120 generates a position vector representing the position of the camera 200 on the camera coordinate system.

the sample described above in

direction vector towards

can be determined as follows. Since the position of the camera 200 is determined as the origin in the camera coordinate system, the position vector

may be a zero vector.

At this time, the processor 120 generates a position vector of the camera.

direction vector from

The point of intersection of a straight line extending in the direction

can be determined as follows.

First, the position vector of the camera

direction vector from

Any point on a straight line extending in the direction can be determined as

Equation of the plate plane described above

Substituting into

is derived as follows.

calculated

cast

Substituting into , the position vector of the camera

direction vector from

The point of intersection of a straight line extending in the direction

Can be determined by Equation 3 as follows.

[Equation 3]

Next, a case in which the depth information includes depths of pixels greater than or equal to a threshold value among pixels included in the target image will be described.

At this time, the processor 120 is

is a coordinate on the camera coordinate system for

can be determined as follows.

here

in the target image

It means the coordinates of the position corresponding to .

and

Is

It means the vertical distance from to the plane perpendicular to the projection direction of the camera 200.

At this time, the processor 120 generates a position vector of the camera.

in the above

direction vector towards

can be determined as follows.

At this time, the processor 120 generates a position vector of the camera.

direction vector from

The point of intersection of the straight line with the plane of the sheet

can be determined by Equation 4 as follows.

[Equation 4]

However, at this time, the aforementioned

and

When the distance of exceeds the set threshold distance, the processor 120

It is determined that the contour point corresponding to does not correspond to the plate material plane,

can be excluded from projection points existing on the plate plane.

Referring to FIG. 7 , projected points are located on the plate material plane. Points located on the plate material plane may be expressed as two-dimensional coordinates having two mutually perpendicular straight lines A and B located on the plate material plane as axes.

In embodiments of the present invention, the processor 120 first calculates the normal vector of the plane of the plank.

and a unit vector parallel to the plate plane.

vector perpendicular to

can be obtained as:

The processor 120 is described above

and, a point on the plate plane

Matrix for converting points on the camera coordinate system to coordinates on the plate plane based on

can be obtained as:

The processor 120 is described above

and

Coordinates on the plate plane for projection points, based on

can be determined as follows.

Referring to FIG. 8 , the contour point conversion method may first include acquiring a target image, which is an image obtained by capturing a plate material through a camera 200 (S810).

And, the contour point conversion method may include determining a plurality of contour points in the target image acquired in step S810 (S820).

And, the contour point conversion method may include determining a plate material plane in the target image acquired in step S810 (S830).

In operation S830 , for example, a plate material plane may be determined based on depth information extracted from i) a camera or ii) a plurality of consecutive images extracted from a target image.

The plate plane is the camera coordinate system, which is a coordinate system determined by the x-axis, which is the horizontal axis of the target image, the y-axis, which is the vertical axis of the target image, and the z-axis, which is the direction axis of the camera lens, with the camera position as the origin. satisfied

can be determined by

At this time,

is the normal vector of the plate plane,

is an arbitrary point on the plate plane.

The contour point conversion method may include a step of determining projected points for one or more of the plurality of contour points by projecting the plurality of contour points determined in step S820 onto the plate material plane (S840).

In step S840, image coordinates corresponding to one of the plurality of contour points

Sample for obtaining coordinates on the camera coordinate system of

can be calculated as:

here

ego,

is the horizontal focal length of the camera 200,

is the vertical focal length of the camera 200,

is the skew coefficient of the camera 200,

Is the coordinate value on the x-axis of the projection center point of the camera 200,

is the coordinate value on the y-axis of the projection center point.

For example, in step S840 , when the depth information includes depth information on a pixel less than a set threshold value among pixels included in the target image, a position vector indicating a position of the camera on the camera coordinate system.

sample from

direction vector towards

can be determined as follows.

And step S840, position vector

direction vector from

The point of intersection of a straight line extending in the direction

can be determined as follows.

As another example, in step S840, when the depth information includes depth information about a pixel equal to or greater than a set threshold among pixels included in the target image,

is a coordinate on the camera coordinate system for

can be determined as follows. At this time,

in the target image

is the coordinates of the position corresponding to

Is

is the vertical distance from to the plane perpendicular to the z-axis.

And step S840, the position vector indicating the position of the camera on the camera coordinate system

at,

direction vector towards

can be determined as follows.

And step S840, position vector

direction vector from

The point of intersection of the straight line with the plane of the sheet

can be determined as follows.

Meanwhile,

and

If the distance of exceeds the set critical distance,

can be excluded from projection points.

Further, the contour point conversion method may include calculating coordinates on the plate material plane for one or more of the plurality of contour points based on the projected points determined in step S840 (S850).

Step S850 is the normal vector of the plate plane.

and a unit vector parallel to the plate plane.

vector perpendicular to

can be obtained as:

And step S850,

and a point on the plane of the plate

A matrix that converts points on the camera coordinate system to coordinates on the plate plane based on

can be obtained as:

And step S850, the coordinates on the plate material plane for the projected points

can be determined as follows.

The present embodiments described above may be implemented through various means. For example, the present embodiments may be implemented by hardware, firmware, software, or a combination thereof.

In the case of hardware implementation, the contour point conversion device and method according to the present embodiments include one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), PLDs ( It can be implemented by Programmable Logic Devices), Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers or microprocessors.

In the case of implementation by firmware or software, the contour point conversion method according to the present embodiments may be implemented in the form of a device, procedure, or function that performs the functions or operations described above. The software codes may be stored in a memory unit and driven by a processor. The memory unit may be located inside or outside the processor and exchange data with the processor by various means known in the art.

Also, the terms "system", "processor", "controller", "component", "module", "interface", "model", or "unit" as described above generally refer to computer-related entities hardware, hardware and software. can mean a combination of, software or running software. For example, but is not limited to, a process driven by a processor, a processor, a controller, a control processor, an object, a thread of execution, a program, and/or a computer. For example, a component can be both an application running on a controller or processor and a controller or processor. One or more components may reside within a process and/or thread of execution, and components may reside on one device (eg, system, computing device, etc.) or may be distributed across two or more devices.

Meanwhile, another embodiment provides a computer program stored in a computer recording medium that performs the above-described contour point conversion method. In addition, another embodiment provides a computer-readable recording medium on which a program for realizing the above-described contour point conversion method is recorded.

A program recorded on a recording medium may be read, installed, and executed in a computer to execute the above-described steps.

In this way, in order for the computer to read the program recorded on the recording medium and execute the functions implemented by the program, the above-described program is C, C++ that can be read by the computer's processor (CPU) through the computer's device interface. , JAVA, may include a code coded in a computer language such as machine language.

These codes may include functional codes related to functions defining the above-described functions, and may include control codes related to execution procedures necessary for a processor of a computer to execute the above-described functions according to a predetermined procedure.

In addition, these codes may further include memory reference related codes for which location (address address) of the computer's internal or external memory should be referenced for additional information or media necessary for the computer's processor to execute the above-mentioned functions. .

In addition, when the computer processor needs to communicate with any other remote computer or server in order to execute the above-mentioned functions, the code allows the computer processor to use the computer's communication module to communicate with any other remote computer or server. Communication-related codes for how to communicate with other computers or servers, what information or media to transmit/receive during communication, and the like may be further included.

Recording media that can be read by a computer on which the program as described above is recorded are, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical media storage device, etc., and also carrier wave (e.g. , Transmission through the Internet) may also include what is implemented in the form of.

In addition, the computer-readable recording medium is distributed in computer systems connected through a network, so that computer-readable codes can be stored and executed in a distributed manner.

In addition, a functional program for implementing the present invention, codes and code segments related thereto, in consideration of the system environment of a computer that reads a recording medium and executes a program, etc., help programmers in the art to which the present invention belongs It may be easily inferred or changed by

In the above, even though all the components constituting the embodiment of the present invention have been described as being combined or operated as one, the present invention is not necessarily limited to these embodiments. That is, within the scope of the object of the present invention, all of the components may be selectively combined with one or more to operate.

Terms such as "comprise", "comprise" or "having" described above mean that the corresponding component may be inherent unless otherwise stated, and therefore do not exclude other components. It should be construed that it may further include other components. All terms, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless defined otherwise. Commonly used terms, such as terms defined in a dictionary, should be interpreted as consistent with the meaning in the context of the related art, and unless explicitly defined in the present invention, they are not interpreted in an ideal or excessively formal meaning.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

CROSS-REFERENCE TO RELATED APPLICATIONCROSS-REFERENCE TO RELATED APPLICATION

This patent application claims priority in accordance with U.S. Patent Act Article 119 (a) (35 U.S.C § 119 (a)) for Patent Application No. 10-2021-0175602 filed in Korea on December 09, 2021, and All contents are incorporated into this patent application by reference. In addition, if this patent application claims priority for the same reason as above for countries other than the United States, all the contents are incorporated into this patent application as references.

Claims

Obtaining a target image, which is an image obtained by capturing a plate material through a camera;

determining a plurality of contour points in the target image;

determining a plate material plane that is a two-dimensional plane corresponding to the plate material in the target image;

determining one or more projected points generated by projecting one or more of the plurality of contour points onto the plate material plane; and

and calculating coordinates on the plate material plane for the projected points.
According to claim 1,

Determining the plate material plane in the target image,

The contour point conversion method of determining the plate material plane based on depth information extracted from i) the camera or ii) a plurality of consecutive images extracted from the target image.
According to claim 2,

The plate plane is

On the camera coordinate system, which is a coordinate system determined by the x-axis, which is the horizontal axis of the target image, the y-axis, which is the vertical axis of the target image, and the z-axis, which is the direction axis in which the lens of the camera is viewed, with the position of the camera as the origin, the following satisfied
is determined by

remind
Is the normal vector of the plate material plane,

remind
Is an arbitrary point on the plate material plane, the contour point conversion method.
According to claim 3,

The step of determining the projected points is,

Image coordinates corresponding to one of the plurality of contour points
Sample for obtaining coordinates on the camera coordinate system of
is calculated as follows,

here
ego,

is the horizontal focal length of the camera,

is the vertical focal length of the camera,

is the skew coefficient of the camera,

Is the coordinate value on the x-axis of the projection center point of the camera,

is a coordinate value on the y-axis of the projection center point, the contour point conversion method.
According to claim 4,

The step of determining the projected points is,

When the depth information includes depth information on pixels less than a set threshold among pixels included in the target image,

A position vector representing the position of the camera on the camera coordinate system
from the above sample
direction vector towards
is determined as follows,

the position vector
the direction vector from
The point of intersection of the straight line extending in the direction and the plane of the plate
Contour point conversion method, which determines as follows.
According to claim 4,

The step of determining the projected points is,

When the depth information includes depth information about a pixel equal to or greater than a set threshold among pixels included in the target image,

remind
Coordinates on the camera coordinate system for
is determined as follows,

In the target image,
is the coordinates of the position corresponding to

said
is the vertical distance from to the plane perpendicular to the z-axis,

A position vector representing the position of the camera on the camera coordinate system
in the above
direction vector towards
is determined as follows,

the position vector
the direction vector from
The point of intersection of the straight line in the direction and the plane of the plate
Contour point conversion method, which determines as follows.
According to claim 6,

remind
and above
If the distance of exceeds the set threshold distance, the
Is excluded from the projection points, the contour point conversion method.
According to claim 5 or 6,

Calculating coordinates on the plate material plane for one or more of the plurality of contour points,

Normal vector of the plate plane
and a unit vector parallel to the plate plane.
vector perpendicular to
is obtained as follows,

remind
and a point on the plate plane.
A matrix for converting points on the camera coordinate system to coordinates on the plate material plane based on
is obtained as follows,

Coordinates on the plate plane for the projected points
Contour point conversion method for determining as follows.
a memory for storing a target image, which is an image obtained by capturing a plate material through a camera; and

determining a plurality of contour points in the target image;

Determining a plate material plane, which is a two-dimensional plane corresponding to the plate material, in the target image;

determining one or more projection points generated by projecting one or more of the plurality of contour points onto the plate material plane;

Contour point conversion device comprising a; processor for calculating coordinates on the plate material plane for the projected points.
According to claim 9,

the processor,

The contour point conversion device for determining the plate material plane based on depth information obtained from i) the camera or ii) a plurality of consecutive images extracted from the target image.
According to claim 10,

The plate plane is

On the camera coordinate system, which is a coordinate system determined by the x-axis, which is the horizontal axis of the target image, the y-axis, which is the vertical axis of the target image, and the z-axis, which is the axis of the direction in which the lens of the camera is viewed, with the position of the camera as the origin, the following satisfied
is determined by

remind
Is the normal vector of the plate material plane,

remind
Is an arbitrary point on the plate material plane, the contour point conversion device.
According to claim 11,

the processor,

Image coordinates corresponding to one of the plurality of contour points
Sample for obtaining coordinates on the camera coordinate system of
is calculated as follows,

here
ego,

is the horizontal focal length of the camera,

is the vertical focal length of the camera,

is the skew coefficient of the camera,

Is the coordinate value on the x-axis of the projection center point of the camera,

is a coordinate value of the projection center point on the y-axis.
According to claim 12,

the processor,

When the depth information includes a depth of a pixel less than a set threshold among pixels included in the target image,

A position vector representing the position of the camera on the camera coordinate system
from the above sample
direction vector towards
is determined as follows,

the position vector
the direction vector from
The point of intersection of the straight line extending in the direction and the plane of the plate
A contour point conversion device that determines as follows.
According to claim 12,

the processor,

When the depth information includes a depth of a pixel greater than or equal to a set threshold among pixels included in the target image,

remind
Coordinates on the camera coordinate system for
is determined as follows,

In the target image,
is the coordinates of the position corresponding to

said
is the vertical distance from to the plane perpendicular to the z-axis,

A position vector representing the position of the camera on the camera coordinate system
in the above
direction vector towards
is determined as follows,

the position vector
the direction vector from
The point of intersection of the straight line in the direction and the plane of the plate
A contour point conversion device that determines as follows.
According to claim 14,

remind
and above
If the distance of exceeds the set threshold distance, the
Is excluded from the projected points, the contour point conversion device.
According to claims 13 to 14,

the processor,

Normal vector of the plate plane
and a unit vector parallel to the plate plane.
vector perpendicular to
is obtained as follows,

remind
and a point on the plate plane.
A matrix for converting points on the camera coordinate system to coordinates on the plate material plane based on
is obtained as follows,

Coordinates on the plate plane for the projected points
A contour point conversion device that determines as follows.