KR20150131543A - Apparatus and method for three-dimensional calibration of video image - Google Patents

Abstract

Disclosed are a device to three-dimensionally correct an image, and a method thereof. According to an embodiment of the present invention, the device to three-dimensionally correct an image comprises: a template overlapping module to obtain the image, and overlap a 3D template with the obtained image to generate an overlay image; and a template adjusting module to arrange a preset virtual object in an overlay image, adjusting a feature parameter of the 3D template by comparing features of the arranged virtual object with features of a real object in the image, thus making the 3D template to effectively be overlaid with an image of a monitoring camera or the like.

Description

[0001] APPARATUS AND METHOD FOR THREE-DIMENSIONAL CALIBRATION OF VIDEO IMAGE [0002]

Embodiments of the present invention relate to object recognition technology in an image using a camera.

As surveillance cameras such as CCTV (Closed-Circuit Television) are becoming common, video data is increasingly used to track movement paths of suspects or vehicles. The CCTV-based control is mainly based on event-driven control that detects the size and movement of objects (people, vehicles, etc.) in the video through video analysis of cameras installed in specific places. .

Since an image photographed through a camera or the like is two-dimensional, a subject actually photographed is three-dimensional. Therefore, in order to detect a three-dimensional object on a two-dimensional screen, information such as a viewing angle, height, In order to achieve this, conventionally, the system operator manually inputs the information while viewing the screen. However, since it is a task requiring a long experience and trial and error of the operator, it takes a very long time to work, and the whole process is performed by hand.

Korean Registered Patent No. 10-0863507 (2008. 10. 08)

Embodiments of the present invention are intended to provide a means for effectively overlaying a three-dimensional template on an image of a surveillance camera or the like.

According to an exemplary embodiment of the present invention, there is provided a template superposition module for acquiring an image, superimposing a three-dimensional template on the acquired image to generate an overlay image, And a template adjustment module for arranging a virtual object preset in the overlay image and adjusting characteristic parameters of the three-dimensional template through feature comparison between the virtual object and the real object in the image, A correction device is provided.

Wherein the template overlapping module displays the one or more previously stored candidate templates together with the obtained image on the screen and selects one of the one or more candidate templates displayed, So that the overlay image can be generated.

The template superposition module may extract the feature information of the image from the image and generate the overlay image by selecting a candidate template having the highest degree of similarity with the feature information extracted from the previously stored candidate template.

The template superimposing module may select a candidate template having the highest degree of similarity to the feature information extracted from the previously stored candidate templates based on at least one of the name of the photographing area of the camera that captured the image or the keyword stored in association with the camera have.

The template adjustment module may arrange, in the overlay image, a candidate virtual object having the highest degree of similarity to the real object among the one or more candidate virtual objects previously stored.

The template adjustment module may select a candidate virtual object having the highest degree of similarity with the real object by comparing the size of the real object input from the user with the size of the stored one or more candidate virtual objects.

The template adjustment module may select a candidate virtual object to be placed in the overlay image among previously stored candidate virtual objects based on at least one of the name of the shooting area of the camera that captured the image or the keyword stored in association with the camera.

Wherein the template adjustment module is configured to calculate a size change in the image of the real object measured as the real object moves in the image and a size change in the image of the real object calculated at the same position as the real object Dimensional template, and adjust the characteristic parameters of the three-dimensional template according to the comparison result.

The characteristic parameter may include at least one of a projection height, a projection direction, a slope, an angle of view, and a focus position of the three-dimensional template.

The template adjustment module may adjust the characteristic parameter of the three-dimensional template by reflecting the inputted input value when the characteristic parameter correction value of the three-dimensional template is inputted from the user.

According to another exemplary embodiment of the present invention, there is provided an image processing method comprising the steps of: acquiring an image; superimposing a three-dimensional template on the acquired image to generate an overlay image; And a step of arranging virtual objects preset in the overlay image and adjusting characteristic parameters of the three-dimensional template through feature comparison between the arranged virtual objects and the real objects in the image, / RTI >

The generating of the overlay image may include displaying at least one candidate template previously stored on the screen together with the acquired image; And superimposing the selected candidate template on the image as the candidate template of the displayed one or more candidate templates is selected.

In the step of adjusting the characteristic parameter, a candidate virtual object having the highest degree of similarity to the real object may be arranged in the overlay image among the previously stored one or more candidate virtual objects.

The generating of the overlay image may extract the feature information of the image from the image and generate the overlay image by selecting a candidate template having the highest degree of similarity with the feature information extracted from the stored candidate template.

The template superimposing module may select a candidate template having the highest degree of similarity to the feature information extracted from the previously stored candidate templates based on at least one of the name of the photographing area of the camera that captured the image or the keyword stored in association with the camera have.

The step of adjusting the characteristic parameter may select a candidate virtual object having the highest degree of similarity with the real object by comparing the size of the real object inputted from the user with the size of the stored one or more candidate virtual objects.

Wherein the step of adjusting the characteristic parameter comprises the step of selecting a candidate virtual object to be placed in the overlay image among the previously stored candidate virtual objects based on at least one of the name of the shooting area of the camera and the keyword stored in association with the camera .

Wherein the step of adjusting the characteristic parameter includes a step of changing a size of the real object in the image measured as the real object moves in the image and a change of the size of the real object calculated at the same position as the real object, Dimensional template, and adjust characteristic parameters of the three-dimensional template according to the comparison result.

The characteristic parameter may include at least one of a projection height, a projection direction, a tilt angle, and a focus position of the three-dimensional template.

The step of adjusting the characteristic parameter may adjust the characteristic parameter of the three-dimensional template by reflecting the inputted input value when the characteristic parameter correction value of the three-dimensional template is input from the user.

According to embodiments of the present invention, various types and types of three-dimensional templates can be stored in advance, and then superimposed on actual images using templates determined to be appropriate among templates stored according to characteristics of images to be photographed, A three-dimensional image correction operation of the image can be performed.

According to embodiments of the present invention, a previously stored virtual object is arranged in the three-dimensional template, and parameters of the overlaid three-dimensional template are corrected through feature comparison between the virtual object and the real object in the image, The accuracy of the image correction can be improved.

In addition, according to embodiments of the present invention, there is an advantage that not only the distance of the object but also the size change of the object according to the change of the viewing angle of the camera can be corrected.

1 is a block diagram for explaining an apparatus 100 for correcting three-dimensional images according to an embodiment of the present invention.
2 is an exemplary diagram illustrating a three-dimensional template according to an embodiment of the present invention.
3 is an exemplary diagram showing an example in which the three-dimensional template shown in FIG. 2 is superimposed on an image
4 is an exemplary diagram showing an example in which a virtual object is arranged in the overlay image shown in Fig.
5 is an exemplary diagram for explaining a process of adjusting characteristic parameters of a three-dimensional template in the template adjustment module 106 according to an embodiment of the present invention
6 and 7 are diagrams for explaining a process for calculating a characteristic value of a camera according to an embodiment of the present invention.
8 is a flowchart for explaining a three-dimensional correction method 800 of an image according to an embodiment of the present invention.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to provide a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, this is merely an example and the present invention is not limited thereto.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. The following terms are defined in consideration of the functions of the present invention, and may be changed according to the intention or custom of the user, the operator, and the like. Therefore, the definition should be based on the contents throughout this specification. The terms used in the detailed description are intended only to describe embodiments of the invention and should in no way be limiting. Unless specifically stated otherwise, the singular form of a term includes plural forms of meaning. In this description, the expressions "comprising" or "comprising" are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, Should not be construed to preclude the presence or possibility of other features, numbers, steps, operations, elements, portions or combinations thereof.

FIG. 1 is a block diagram for explaining an apparatus 100 for correcting three-dimensional images according to an embodiment of the present invention. An apparatus for correcting three-dimensional images 100 according to an embodiment of the present invention includes a device 100 for calculating a position on a three-dimensional space of an object (object) in the two-dimensional image by superimposing a three- to be. The three-dimensional image correcting apparatus 100 according to an exemplary embodiment of the present invention may be included as a component in a camera device such as a surveillance camera or may be configured as a separate device from a surveillance camera, . As shown, the apparatus for correcting three-dimensional images 100 according to an exemplary embodiment of the present invention includes a data storage module 102, a template superposition module 104, and a template adjustment module 106.

The data storage module 102 is a space for storing one or more 3D templates and one or more virtual objects to be overlaid on an image photographed from a surveillance camera or the like.

In one embodiment of the present invention, a three-dimensional template is a kind of a guide line for calculating a position on a three-dimensional space of an object located in an image. As shown in FIG. 2, And may include a grid. Although each of the grids constituting the three-dimensional template may appear to be different in size on the two-dimensional screen, it corresponds to a space of the same size in the actual space. If the three-dimensional template superimposed on the image is used, The size and position of the object in the actual space can be calculated. FIG. 3 shows an example in which the three-dimensional template shown in FIG. 2 is superimposed on an image.

The data storage module 102 may store one or more different three-dimensional templates. For example, the data storage module 102 may store three-dimensional templates having different shapes according to characteristics of a space such as a flat, a road, an intersection, an entrance, an office, a corridor, and the like.

The virtual object is a kind of virtual object to be superimposed and displayed together with the three-dimensional template in the image and has a shape such as a person (man, woman, child), a vehicle (vans, buses, passenger cars) . The data storage module 102 may store size information (height, area, etc.) of each object together with the virtual object. The virtual object is used to correct the three-dimensional template through comparison with a real object in an image. FIG. 4 shows an example in which a virtual object (a tree, a person, an automobile, etc.) is arranged in the overlay image shown in FIG.

The template superposition module 104 acquires an image from a camera or the like, and superimposes the three-dimensional template on the acquired image to generate an overlay image.

In one embodiment, the template superposition module 104 stores and manages the information of one or more three-dimensional templates (candidate templates) to be superimposed on the image in the data storage module 102, Dimensional template can be displayed on the screen. Then, the user selects one of the displayed three-dimensional templates, and the template superimposing module 104 may generate an overlay image by superimposing the selected candidate template on the image.

In another embodiment, the template superimposing module 104 derives the characteristics of the acquired image (presence / absence of horizon in the image, position of the main feature line, connection relation, etc.) and automatically selects the optimal three- May be configured to select. For this, the template superposition module 104 may use various algorithms to derive in-image feature information. For this purpose, the template superposition module 104 may include an algorithm for recognizing an object in the screen and its type (automobile, person, etc.) in order to accurately recognize the position and connection relation of the main feature lines in the image.

In addition, the template superposition module 104 may select an optimal three-dimensional template using information of the camera that has captured the image. For example, a name of a shooting area or a shooting area (POSCO INTERSECTION, NAMSAN 1 NO. TUNNEL, etc.) is added to the camera in which the image is captured, or a keyword such as a doorway, A conference room, a corridor, etc.) is added, the template superposition module 104 can use it to select an optimal template among pre-stored templates.

The template adjustment module 106 arranges a virtual object in the overlay image generated by the template overlap module 104 and compares the characteristic parameter of the three-dimensional template with the real object in the image, .

In an embodiment of the present invention, the characteristic parameter is information including at least one of a projection height, a projection direction, a slope, an angle of view, and a focus position of the three-dimensional template, ≪ / RTI > Since the characteristic parameters depend on the installation height of the camera or the like, the lens type (focal length, etc.), the tilt angle, and the like, when the characteristic parameters of the three-dimensional template are calculated, The angle of view, and the like.

Specifically, the template adjustment module 106 may select a candidate virtual object having the highest degree of similarity with the size of a real object in the image among the one or more virtual objects stored in the data storage module 102, and may arrange the candidate virtual object in the overlay image. For this, the template adjustment module 106 may receive the size information (key or area, etc.) of the real object in the image from the user. For example, if there is a woman with a height of 160 centimeters on the screen, the template adjustment module 106 may place a virtual object having the same key as the woman on the overlay image. In addition, the template adjustment module 106 may automatically place an optimal virtual object on the overlay image according to the characteristics of the image. For example, a vehicle may be arranged as a virtual object when the image is an intersection, and a person may be arranged as a virtual object in the case of a hallway or an entrance. Such characteristics of the image can be obtained by using the name or keyword of the camera that captured the image as described above.

The size of the virtual object in the image depends on the position of the virtual object in the image and the information obtained from the overlapped three-dimensional template. For example, even if a virtual object of the same size is located on a three-dimensional template, the virtual object at a position distant from the camera is smaller in size than the virtual object at a position determined to be located at a shorter distance.

When the virtual objects are arranged in the overlay image as described above, the template adjustment module 106 adjusts the characteristic parameters of the three-dimensional template by comparing the real objects in the image with the virtual objects. Specifically, the template adjustment module 106 adjusts the size of the real object in the image, which is measured as the real object moves in the image, and the size of the real object, which is calculated at the same position as the real object, The size variation on the three-dimensional template is compared, and the characteristic parameters of the three-dimensional template are adjusted according to the comparison result. That is, in the present invention, the feature of a real object or a virtual object means a degree of magnitude change when the corresponding object moves in the image. This will be described in more detail as follows.

5 is an exemplary diagram for explaining a process of adjusting characteristic parameters of a three-dimensional template in the template adjustment module 106 according to an embodiment of the present invention. In the illustrated example, 502 represents a real object in an image and 504 represents a virtual object. As shown, the height (key) is changed from h1 to h1 'as a real object (e.g., a person) in the image moves from position A to position B, and at the same position, (Key) is changed from h2 to h2 '. As described above, the size of the virtual object in the image is determined according to the characteristic parameter of the three-dimensional template superimposed on the image. Therefore, if h1 is equal to h2, h1 'is not equal to h2', it means that the characteristic parameter of the overlapped three-dimensional template does not coincide with the actual space. Therefore, The projection angle, the projection angle, the projection angle, the tilt angle, the angle of view, and the focus position. Such an adjustment is repeated until the change due to the movement of the real object in the image becomes equal to the virtual object.

In addition, the template adjustment module 106 may receive a correction value for the characteristic parameter from the user and reflect it in the characteristic parameter adjustment. For this, the template adjustment module 106 may provide an interface for the user to intuitively adjust the template using a keyboard, a mouse, a screen touch, or the like. As described above, when the template adjustment module 106 changes the size of the object in the image and uses the correction value input from the user, the accuracy of the characteristic parameter can be further increased.

When the three-dimensional template is adjusted as described above, the installation characteristic value of the camera can be calculated from the object in the image. 6 and 7 are for explaining this. As shown, there are two objects, object A and object B on the screen, object A is located on a center line (indicated by a dotted line) connected from the camera, object B is located at an angle x .

If the distance between the object A and the object B is y and the distance between the object B and the center line is z, the installation position cl of the camera can be calculated as follows.

cl = z / tan (x) + y

If the size of the object A on the screen is ah and the size of the object B on the screen is bh, the camera installation height (ch) can be calculated as follows.

ch = (bh-ah) * cl / y

From this, the angle of view (cv) and the angle of installation (ct) of the camera can be calculated as follows.

cv = arctan (ch / cl)

ct = 90 - arctan (ch / cl)

FIG. 8 is a flowchart illustrating a three-dimensional correction method 800 of an image according to an exemplary embodiment of the present invention.

In step 802, the template superposition module 104 selects a three-dimensional template to be superimposed with an image from a user or the like. For this, the template superposition module 102 may provide an appropriate interface for allowing a user to visually browse pre-stored templates and select a template suitable for the images. For example, the template superposition module 104 may display one or more previously stored candidate templates together with the acquired images on the screen, and select one of the displayed one or more candidate templates.

In step 804, the template superposition module 104 determines whether an input is required for a set value (characteristic parameter, etc.) for the selected template, and if it is determined to be necessary, the template superposition module 104 receives the input from the user in step 806.

In step 808, the template adjustment module 106 receives a virtual object to be placed on the image from a user or the like. At this time, the virtual object may be a candidate virtual object having the highest degree of similarity to the real object among one or more previously stored candidate virtual objects.

In step 810, the template adjustment module 106 determines whether an input for a set value (height or length of the object, etc.) for the selected virtual object is required. If it is determined to be necessary, Receive.

In step 814, the template adjustment module 106 determines whether or not the characteristics of the virtual object and the real object coincide with each other through a feature comparison between the virtual object disposed in the image and the real object in the image. As described above, the template adjustment module 106 adjusts the size of the real object in the image, which is measured as the real object moves in the image, and the size of the real object, which is calculated at the same position as the real object Dimensional template on the three-dimensional template, and adjust the characteristic parameter of the three-dimensional template according to the comparison result.

If it is determined in step 814 that the characteristics of the real object and the virtual object do not match, the template adjustment module 106 adjusts the characteristic parameters of the template until the feature matches.

In step 818, the template adjustment module 106 determines whether or not a request to add a virtual object is received from the user, and if there is a request to add a virtual object, the template adjustment module 106 repeats the steps below step 808 for the newly selected virtual object.

On the other hand, an embodiment of the present invention may include a computer-readable recording medium including a program for performing the methods described herein on a computer. The computer-readable recording medium may include a program command, a local data file, a local data structure, or the like, alone or in combination. The media may be those specially designed and constructed for the present invention, or may be those that are commonly used in the field of computer software. Examples of computer readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and magnetic media such as ROMs, And hardware devices specifically configured to store and execute program instructions. Examples of program instructions may include machine language code such as those generated by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, . Therefore, the scope of the present invention should not be limited to the above-described embodiments, but should be determined by equivalents to the appended claims, as well as the appended claims.

100: Three-dimensional correction device of image
102: Data storage module
104: Template Overlap Module
106: template adjustment module

Claims (20)

A template superposition module for acquiring an image and superimposing a three-dimensional template on the acquired image to generate an overlay image; And
And a template adjustment module for arranging a virtual object preset in the overlay image and adjusting characteristic parameters of the three-dimensional template through feature comparison between the virtual object and the real object in the image, Device.
The method according to claim 1,
Wherein the template superposition module comprises:
Displaying the one or more previously stored candidate templates together with the acquired image on a screen,
And the overlay image is generated by superimposing the selected candidate template on the image as one of the candidate templates displayed on the displayed one or more candidate templates is selected.
The method according to claim 1,
Wherein the template superposition module extracts feature information of the image from the image and generates the overlay image by selecting a candidate template having the highest degree of similarity to the feature information extracted from the previously stored candidate template, Device.
The method of claim 3,
The template overlap module selects a candidate template having the highest degree of similarity with the feature information extracted from previously stored candidate templates based on at least one of the name of the shooting area of the camera that captured the image or the keyword stored in association with the camera , A three-dimensional correction device for an image.
The method according to claim 1,
Wherein the template adjustment module disposes the candidate virtual object having the highest degree of similarity with the real object among the one or more candidate virtual objects stored in the overlay image.
The method of claim 5,
Wherein the template adjustment module selects a candidate virtual object to be placed in the overlay image among previously stored candidate virtual objects on the basis of at least one of a name of the shooting area of the camera that captured the image or a keyword stored in association with the camera / RTI >
The method of claim 5,
Wherein the template adjustment module selects a candidate virtual object having the highest degree of similarity with the real object by comparing the size of the real object input from the user with the size of the stored one or more candidate virtual objects.
The method of claim 5,
Wherein the template adjustment module is configured to calculate a size change in the image of the real object measured as the real object moves in the image and a size change in the image of the real object calculated at the same position as the real object Dimensional template, and adjusts the characteristic parameter of the three-dimensional template according to the comparison result.
The method of claim 8,
Wherein the characteristic parameter includes at least one of a projection height, a projection direction, a slope, an angle of view, and a focus position of the three-dimensional template.
The method of claim 8,
Wherein the template adjustment module adjusts a characteristic parameter of the three-dimensional template by reflecting the input value when the characteristic parameter correction value of the three-dimensional template is input from a user.
Generating an overlay image by superimposing a three-dimensional template on the acquired image; And
Arranging a virtual object preset in the overlay image, and adjusting characteristic parameters of the three-dimensional template through feature comparison between the arranged virtual object and a real object in the image.
The method of claim 11,
Wherein the generating the overlay image comprises:
Displaying one or more pre-stored candidate templates on the screen together with the acquired images; And
Further comprising superimposing the selected candidate template on the image as the candidate template selected from the displayed one or more candidate templates is selected.
The method of claim 11,
Wherein the generating the overlay image comprises:
Extracting feature information of the image from the image and selecting the candidate template having the highest degree of similarity with the feature information extracted from the stored candidate template to generate the overlay image.
The method of claim 11,
The template overlap module selects a candidate template having the highest degree of similarity with the feature information extracted from previously stored candidate templates based on at least one of the name of the shooting area of the camera that captured the image or the keyword stored in association with the camera , A three-dimensional correction method of an image.
The method of claim 11,
Wherein the step of adjusting the characteristic parameter includes placing a candidate virtual object having the highest degree of similarity with the real object among the one or more candidate virtual objects previously stored in the overlay image.
16. The method of claim 15,
Wherein the step of adjusting the characteristic parameter comprises the step of selecting a candidate virtual object to be placed in the overlay image among previously stored candidate virtual objects based on at least one of a name of the shooting area of the camera and a keyword stored in association with the camera A three-dimensional correction method of an image.
16. The method of claim 15,
Wherein the step of adjusting the characteristic parameter comprises: comparing a size of the real object input from a user with a size of the stored one or more candidate virtual objects, thereby selecting a candidate virtual object having the highest degree of similarity with the real object; Correction method.
16. The method of claim 15,
Wherein adjusting the characteristic parameter comprises:
A size change in the image of the real object measured as the real object moves in the image and a change in size of the real object calculated on the same position as the real object on the three- And adjusting characteristic parameters of the three-dimensional template according to the comparison result.
19. The method of claim 18,
Wherein the characteristic parameter includes at least one of a projection height, a projection direction, a tilt angle, and a focus position of the three-dimensional template.
19. The method of claim 18,
Wherein the step of adjusting the characteristic parameter comprises the steps of adjusting a characteristic parameter of the three-dimensional template by reflecting the inputted input value when a characteristic parameter correction value of the three-dimensional template is input from a user, .

Images