KR101882092B1 - Camera calibration system and operating method thereof - Google Patents

Abstract

The camera calibration system includes a camera that is mounted on a satellite and captures an image, a plurality of rod-shaped booms, each of the rod-shaped boom including one or more markers spaced a predetermined distance apart from the camera, And estimates at least one of an external parameter, an internal parameter, and a lens distortion coefficient of the camera.

Description

[0001] CAMERA CALIBRATION SYSTEM AND OPERATING METHOD THEREOF [0002]

The present invention relates to a camera calibration system and an operation method thereof, and more particularly to a system and an operation method thereof capable of automatically performing camera calibration using additional equipment minimized in an environment requiring miniaturization and light weight such as a satellite.

Many researches have been made to miniaturize satellites while diversifying satellite functions such as satellites and exploration rovers. In particular, the use of a camera is essential to perform close-up operation such as autonomous navigation, automatic docking and capture in a satellite.

In order to improve the precision and stability of the close-up operation, a calibration process may be required to estimate the external parameters and internal parameters of the camera mounted on the satellites. In general, it is possible to implement the proximity operation algorithm by estimating the camera's external parameters and internal parameters in advance on the ground, but there is a possibility that the camera's attitude or the physical properties of the lens are changed by an unexpected physical collision or environmental change.

In the ground, a method of performing camera calibration by photographing a wide lattice plate in the form of a checkerboard at various angles and attitudes is used. However, in an environment with a large space constraint such as a very small satellite, the same method as in the ground is difficult to use.

According to one aspect, a camera calibration system includes a camera that is mounted to a satellite and captures an image, a plurality of rod-shaped booms each including one or more markers spaced a predetermined distance apart from the camera, And an operation unit for estimating at least one of an external parameter, an internal parameter and a lens distortion coefficient of the camera based on the position of the camera.

In one embodiment, the one or more markers include one or more LEDs, and the camera calibration system further includes an LED control unit for controlling lighting of the one or more LEDs.

In one embodiment, the LED control unit groups the one or more LEDs according to an interval spaced from the camera so that each group of LEDs is sequentially turned on by one group, To take an image.

In one embodiment, the operation unit obtains the position of the one or more markers by applying Mean Filter to the result of repeatedly photographing the image for each LED group for k times.

In one embodiment, while photographing the image, the plurality of rod-shaped booms are arranged to extend in a direction away from the camera on the outer wall of the satellite, and while the rod-shaped boom is not photographing the plurality of rod- And is arranged to contract on the outer wall of the satellite or inside the satellite. In one embodiment, the plurality of rod-shaped booms include a multi-stage foldable structure.

In one embodiment, the external parameter comprises at least one of rotation and translation of the camera. In one embodiment, the internal parameter includes at least one of a focal length, a lens center and an asymmetry coefficient of the camera.

According to another aspect, a method of performing a camera calibration using a computing device on a satellites includes controlling a plurality of rod-shaped booms comprising one or more LEDs to position the one or more LEDs so as to be spaced from the camera by a predetermined distance Estimating at least one of an external parameter, an internal parameter and a lens distortion coefficient of the camera based on the position of the one or more LEDs in the image, using the camera, .

In one embodiment, the step of photographing the one or more LEDs comprises grouping the one or more LEDs according to an interval spaced from the camera, and controlling each of the LED groups to be turned on by one group at a time And taking an image of each of the LED groups using the camera.

In one embodiment, the camera calibration method further comprises obtaining a position of the at least one LED by applying a mean filter to the result of repeatedly imaging the image for each LED group for k times.

According to another aspect, a camera calibration system includes a camera that is mounted to a satellite and captures an image, each of the at least one marker being spaced a predetermined distance apart from the camera, each of the at least one marker having a different color An internal parameter and a lens distortion coefficient of the camera based on the position of the at least one marker in the image.

In one embodiment, the one or more markers include one or more LEDs, and the camera calibration system further includes an LED control unit for controlling lighting of the one or more LEDs.

In one embodiment, the operation unit obtains the position of the at least one marker by applying a mean filter to the result of repeatedly photographing the image for k times.

According to another aspect, a method of performing a camera calibration using a computing device on a satellites includes controlling a plurality of bar booms comprising one or more LEDs such that the one or more LEDs are spaced from the camera by a predetermined interval, Wherein the at least one LED has a different color according to the spaced distance from the camera, using the camera to take an image for the one or more LEDs, and based on the position of the one or more LEDs in the image Estimating at least one of an external parameter, an internal parameter, and a lens distortion coefficient of the camera.

In one embodiment, the calibration method further comprises obtaining a position of the one or more LEDs by applying Mean Filter to the result of repeatedly photographing the image for k times.

1 is a block diagram illustrating an exemplary configuration of a camera calibration system in accordance with one embodiment.
FIG. 2 is a view illustrating a part of a camera calibration system according to an exemplary embodiment. Referring to FIG.
3 is a flowchart illustrating a camera calibration method according to an exemplary embodiment.
4 is an exemplary diagram illustrating data stored for calibration in a calibration system according to one embodiment.
5 illustrates an exemplary location of a projected marker on a coordinate in an image in a calibration system according to one embodiment.
6 illustrates an exemplary location of a marker displayed on three-dimensional spatial coordinates in a calibration system according to one embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the rights is not limited or limited by these embodiments. Like reference symbols in the drawings denote like elements.

The terms used in the following description are chosen to be generic and universal in the art to which they are related, but other terms may exist depending on the development and / or change in technology, customs, preferences of the technician, and the like. Accordingly, the terminology used in the following description should not be construed as limiting the technical thought, but should be understood in the exemplary language used to describe the embodiments.

Also, in certain cases, there may be a term chosen arbitrarily by the applicant, in which case the meaning of the detailed description in the corresponding description section. Therefore, the term used in the following description should be understood based on the meaning of the term, not the name of a simple term, and the contents throughout the specification.

1 is a block diagram illustrating an exemplary configuration of a camera calibration system 100 in accordance with one embodiment.

In the following, the camera calibration system 100 according to one embodiment is described on the assumption that it operates on a satellite or an exploration rover in a vacuum or an under vacuum environment, but is not necessarily limited thereto, and is not limited to the technical idea of the illustrated embodiment And can also be used in a terrestrial environment or the like.

In one embodiment, the camera calibration system 100 may include a rod boom 110, an LED control unit 120, a camera 130, and a computing unit 140.

In one embodiment, the bar boom 110 may provide a fiducial marker for camera calibration. For example, the bar boom 110 may include one or more markers positioned at a predetermined spaced distance from the camera 130. The rod boom 110 may include a plurality of rod-like booms for accuracy of calibration, and may preferably include four rod-like booms.

In one embodiment, one or more of the markers of the bar boom 110 may be implemented using LEDs. In this case, the LED controller 120 can be used to supply power to the LEDs and to control the turning on and off of the LEDs. The LEDs may be implemented to have the same color, but may be implemented to have different colors depending on the spacing distance from the camera. For example, LEDs with the same spacing apart from the camera in each of the plurality of bar booms 110 may be designed to have the same color, and LEDs with different spacing distances from the camera may have different colors.

When the rod boom 110 and the LED control unit 120 are used to provide a fiducial marker for camera calibration, a wide grating plate in the form of a checkerboard, which is generally used in a ground environment, The camera calibration can be performed using a lightweight and miniaturized structure as compared with a method of photographing and performing camera calibration.

The bar boom 110 may be of a multi-stage type, for example, having a folding structure. For example, a structure in which a part of the bar boom 110 can be inserted into another part of the bar boom 110, or a structure in which a part of the bar boom 110 is rotated by an angle change using a hinge or the like, It is possible to reduce the volume and facilitate the storage while the camera calibration is not performed. Due to the structure of such a boom 110, it is possible to perform sophisticated camera calibration in a limited environment without equipments such as a wide lattice plate and an additional robotic arm to control it.

In one embodiment, the camera 130 is mounted on a satellite and performs the function of shooting an image. The camera 130 may be any type of camera that requires calibration with spatial coordinates as well as a typical camera that captures 2D color images. The camera 130 may be used to take an image of one or more markers of the bar boom 110 for camera calibration.

In one embodiment, the operation unit 140 may perform various operations such as image processing required for camera calibration. In one embodiment, the computing unit 140 may be a chip, a machine, or a computing device including one or more processors. The operation unit 140 may include an internal memory and / or an external memory as needed. Operations that can be performed in the arithmetic unit 140 will be described in more detail below with reference to FIG.

FIG. 2 is a diagram illustrating a portion of a camera calibration system 200 according to an exemplary embodiment. Referring to FIG. In one embodiment, the camera calibration system 200 may include a camera 210, a plurality of rod booms 221, 222, 223, 224, and one or more LED groups 231, 232.

The plurality of bar booms 221, 222, 223, and 224 may each include one or more LEDs disposed at a predetermined spaced distance from the camera 210. Each LED may be implemented to have a different color depending on the spacing distance from the camera for ease of identification.

The plurality of rod booms 221, 222, 223, and 224 may be arranged to extend in a direction away from the camera on the outer wall of the satellite during image capturing for calibration, It may contract on the outer wall of the satellite or inside the satellite. To this end, the plurality of bar booms 221, 222, 223, 224 may comprise a multi-stage folding structure.

When a plurality of rod booms 221, 222, 223 and 224 are photographed using the state air camera 210 extended as shown in Fig. 2, one or more LED groups 231 and 232 are displayed on an indicator So that camera calibration can be performed. Each LED group 231, 232 may be a collection of LEDs that are expected to be spaced equally from the camera 210.

In one embodiment, LEDs that are expected to be spaced equally spaced from the camera 210 in a manner that causes each LED group 231, 232 to sequentially light up one group at a time, can do. In another embodiment, a method of capturing all the LEDs on one image and then estimating the spacing distance from the camera 210 based on the color of the LEDs may be used.

3 is a flowchart illustrating a camera calibration method according to an exemplary embodiment. The method of FIG. 3 may be used, for example, to operate the calibration system of FIG.

In one embodiment, the camera calibration method sequentially captures n LED groups on each image, repeats this process k times, and performs a camera calibration that is robust to noise such as vibration by applying Mean Filter . When the camera calibration is started, the values of n and k can be initialized to 0 or an appropriate value.

In step 310, the n-th LED group located on the plural-type boom can be lit through the LED control unit. This operation is for simultaneously lighting only the LEDs expected to be spaced apart from the camera by the same interval, and all LEDs not belonging to the n-th LED group can be turned off.

In step 320, a camera may be used to take an image of the LED group. The brightness or color of the LED is designed to be sufficient to distinguish the LEDs illuminated in step 310 from the remaining LEDs on the photographed image.

In step 330, the position of the lit LED on the photographed image may be read and stored in the form of a position table in a memory or the like. Any suitable feature point reading technique commonly used for marker extraction in an image may be used for positional reading of the LED.

In step 340, it may be determined whether an image has been taken for all the LED groups. If no image has been taken for all the LED groups, the process may proceed to step 310 for the next iteration of taking an image for the next LED group. If an image has been taken for all the LED groups, step 350 may be performed.

At step 350, it may be determined whether an image for all the LED groups has been taken by a predetermined number of repetitions. For example, in order to increase the precision of the camera calibration, the image for all the LED groups can be repeatedly photographed a predetermined number of repetitions k times. The value of k may be specified differently depending on the level of noise expected in the environment in which the camera calibration is performed. If the image has not been photographed a predetermined number of times, the process may proceed to step 310 for performing the next iteration. If the image has been photographed a predetermined number of times, the process may proceed to step 360.

In step 360, a mean filter may be applied to the positions of the LEDs read from the k images repeatedly photographed. The coordinates of the LED in the repeatedly photographed image may be distorted due to noise generated during shooting and noise due to vibration of the boom. Therefore, in order to correct the influence of the noise, a mean filter which takes an average of the coordinates of the LED can be applied. This process can offset the effect of noise.

In step 370, a camera calibration operation may be performed that uses the position of the LED to estimate the camera's internal parameters, external parameters, and lens distortion coefficients. The external parameters of the camera may include rotation and translation of the camera, and may be stored in a matrix form indicating the conversion relationship between the camera coordinate system and the spatial coordinate system. The internal parameters of the camera may include the focal length of the camera, the lens center, and the asymmetry coefficient, and likewise may be stored in a matrix form. The lens distortion factor of the camera may also include a radial distortion factor and a tangential distortion factor. Thus, the camera calibration result can be obtained through estimating the internal parameters, external parameters and lens distortion coefficients of the camera.

4 is an exemplary diagram illustrating data stored for calibration in a calibration system according to one embodiment.

In one embodiment, the three-dimensional position and the two-dimensional position of the marker implemented with an LED or the like can be read and the coordinates in the image stored in the position table as shown. By using the method of storing the coordinates other than the image, for example, even when k times of repeated shooting is performed, it is possible to implement a camera calibration system robust against noise with a small memory cost.

5 illustrates an exemplary location of a projected marker on a coordinate in an image in a calibration system according to one embodiment. The position of the marker shown in Fig. 5 may be obtained, for example, by the camera calibration method of Fig.

As described above, it is possible to repeatedly photograph the same marker by a predetermined number of repetition times k times, whereby the presence and degree of noise can be grasped as shown in FIG.

6 illustrates an exemplary location of a marker displayed on three-dimensional spatial coordinates in a calibration system according to one embodiment. The position of the marker shown in Fig. 5 may be obtained, for example, by the camera calibration method of Fig.

As described above, the three-dimensional position of the marker photographed in the image can be confirmed through sequential shooting or color-based distinction. As in FIG. 5, it is possible to repeatedly photograph the same marker by a predetermined number of repetition times k, so that the presence and degree of noise can be grasped as shown in FIG.

The embodiments described above may be implemented in hardware components, software components, and / or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, such as an array, a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be embodyed temporarily. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced. Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (17)

A camera mounted on a satellite and capturing an image;
A plurality of rod-shaped booms each including one or more markers spaced a predetermined distance apart from the camera; And
An internal parameter and a lens distortion coefficient of the camera based on the position of the one or more markers in the image,
Lt; / RTI >
Wherein the at least one marker comprises one or more LEDs,
Wherein the camera calibration system further comprises an LED control for controlling lighting and lighting of the one or more LEDs. delete The method according to claim 1,
Wherein the LED control unit groups the one or more LEDs according to an interval spaced from the camera so that each of the LED groups is sequentially turned on by one group,
The camera capturing an image for each of the LED groups,
Camera calibration system. The method of claim 3,
Wherein the operation unit obtains a position of the one or more markers by applying Mean Filter to a result of repeatedly photographing the image for each LED group for k times,
Camera calibration system. The method according to claim 1,
Wherein the plurality of rod-shaped booms are arranged to extend in a direction away from the camera on an outer wall of the satellite,
Wherein the plurality of rod-shaped booms are arranged to contract on an outer wall of the satellite body or inside the satellite body,
Camera calibration system. The method according to claim 1,
Wherein the plurality of rod-shaped booms comprises a multi-stage foldable structure,
Camera calibration system. The method according to claim 1,
Wherein the external parameter comprises at least one of a rotation and a translation of the camera.
Camera calibration system. The method according to claim 1,
Wherein the internal parameter comprises at least one of a focal length, a lens center and an asymmetry coefficient of the camera,
Camera calibration system. A method of performing camera calibration using a computing device on a satellite,
Controlling a plurality of barber booms comprising one or more LEDs to place the one or more LEDs so that they are spaced from the camera by a predetermined interval;
Capturing an image of the one or more LEDs using the camera; And
Estimating at least one of an external parameter, an internal parameter and a lens distortion coefficient of the camera based on the position of the one or more LEDs in the image
/ RTI > 10. The method of claim 9,
Wherein photographing the image for the one or more LEDs comprises:
Grouping the one or more LEDs according to an interval spaced from the camera, and controlling each of the LED groups to sequentially turn on a group of LEDs; And
Capturing an image for each of the LED groups using the camera
/ RTI > 11. The method of claim 10,
Obtaining a position of the at least one LED by applying Mean Filter to a result of repeatedly photographing the image for each LED group for k times,
Wherein the camera calibration method further comprises: A camera mounted on a satellite and capturing an image;
A plurality of rod-shaped booms spaced apart from each other by a predetermined distance from the camera, each rod comprising a plurality of bar-shaped booms each having one or more markers with different colors according to spaced distances from the camera; And
An internal parameter and a lens distortion coefficient of the camera based on the position of the one or more markers in the image,
And a camera calibration system. 13. The method of claim 12,
Wherein the at least one marker comprises one or more LEDs,
Wherein the camera calibration system further comprises an LED control unit for controlling lighting and lighting of the one or more LEDs,
Camera calibration system. 13. The method of claim 12,
Wherein the operation unit acquires the position of the one or more markers by applying Mean Filter to the result of repeatedly photographing the image for k times,
Camera calibration system. A method of performing camera calibration using a computing device on a satellite,
Controlling a plurality of barber booms comprising one or more LEDs such that the one or more LEDs are spaced from the camera by a predetermined interval, the one or more LEDs having a different color according to a spaced distance from the camera;
Capturing an image of the one or more LEDs using the camera; And
Estimating at least one of an external parameter, an internal parameter and a lens distortion coefficient of the camera based on the position of the one or more LEDs in the image
/ RTI > 16. The method of claim 15,
Obtaining a position of the one or more LEDs by applying Mean Filter to the result of repeatedly photographing the image k times,
Wherein the camera calibration method further comprises: A computer-readable recording medium on which a program for executing the camera calibration method according to claim 9 or 15 is recorded.

Images