KR102473165B1 - Cctv control system for being enable to measure real size of subject using 3d environment model - Google Patents

Abstract

Embodiments include a 3D modeling unit for generating a 3D environment model in which the screen space is implemented as a three-dimensional virtual space based on actual measurement information related to the screen space captured on a CCTV screen; a mapping unit for mapping the 3D environment model to the CCTV screen; an object detection unit for detecting an object in an image of a CCTV screen captured by the CCTV; and a real size measuring unit measuring a real size of a subject included in the detected object in a 3D real environment using a mapped 3D environment model. and a method for measuring the actual size of a subject using a 3D environment model performed by a CCTV control system including one or more CCTVs and a processor.

Description

CCTV control system capable of measuring the actual size of a subject using a 3D environment model

Embodiments of the present invention relate to a technology for measuring the actual size of a subject captured by CCTV, and more particularly, to a CCTV control system capable of measuring the actual size of a subject using a measurement-based 3D environment model.

In order to measure the actual size of an object, it is common to newly install a 3D-CCTV using a special camera such as a stereoscopic camera.

However, since such 3D CCTV equipment is expensive, it is difficult to implement in real life.

To solve this problem, an attempt to use a two-dimensional image of an existing CCTV screen (title of invention: method for measuring distance on a CCTV screen using satellite photos, Korean Patent Publication No. 10-2017-0021423 (2017.02.28.) ) is there. The attempt measures the width or length of a facility such as a road by matching a satellite image with a CCTV screen.

However, this attempt must necessarily require satellite photos, and can only measure static objects such as facilities, and has limitations in that it cannot measure the length of dynamically moving objects (eg, the height of a pedestrian).

Patent Registration Publication No. 10-1917000 (2018.11.02.)

A CCTV control system capable of measuring the actual size of a subject displayed in a two-dimensional image of a CCTV screen may be provided using a 3D environment model based on actual measurements according to an aspect of the present invention.

In addition, a method for measuring the actual size of a subject using a 3D environment model and a computer readable recording medium recording instructions for performing the method may be provided.

A CCTV control system for measuring the actual size of a subject captured by one or more CCTVs according to an aspect of the present invention: based on actual measurement information related to the screen space captured on the CCTV screen, the screen space is implemented as a three-dimensional virtual space 3D modeling unit for generating a 3D environment model; a mapping unit for mapping the 3D environment model to the CCTV screen; an object detection unit for detecting an object in an image of a CCTV screen captured by the CCTV; and a real size measuring unit that measures a real size of a subject included in the detected object in a 3D real environment using a mapped 3D environment model.

In one embodiment, the 3D modeling unit may be further configured to generate virtual cameras corresponding to the one or more CCTVs on the 3D virtual space. Here, the virtual camera may have the same view as the corresponding CCTV screen.

In one embodiment, the mapping unit, in order to map the CCTV screen and the 3D environment model, based on the ground plane captured on the screen of the virtual camera and the ground plane implemented on the 3D virtual space, the screen of the virtual camera and the view of the 3D environment model may be matched.

In one embodiment, the mapping unit matches the virtual camera screen and the 3D environment model through the following equation,

[mathematical expression]

Here, {Ψi, λi} represents the position of the ith projected point on the 3D model space and the position of the ith point on the 2D screen of the virtual camera image, respectively, and dk is the 3D space considering the camera distortion (k). As a function that projects the point of the camera into the 2D space of the camera and calculates the distance to the point of the other 2D space, based on the specifications of CCTV, R and t may represent the angle and position of the virtual camera in the 3D space, respectively have.

In one embodiment, the object detection unit extracts a feature from an input image, and applies the image of the CCTV screen to a pre-learned machine learning model to detect an object included in the CCTV screen based on the extracted feature, thereby applying the object can also detect

In one embodiment, the actual size measuring unit searches for a real size value mapped to a pixel size value in a 3D virtual location on a mapped 3D environment model corresponding to a location in a CCTV screen of the detected object, Also, the searched actual size value may be determined as the actual size of the object.

A method for measuring the actual size of a subject using a 3D environment model according to another aspect of the present invention is performed by a processor, comprising: obtaining an image of a CCTV screen including the subject by a CCTV; and measuring a real size of a subject on a CCTV screen using a pre-stored 3D environment model. Here, the pre-stored 3D environment model is a model mapped to the CCTV screen.

In one embodiment, the 3D environment model may be generated by 3D rendering of the screen space based on actual measurement information of the screen space captured on the CCTV screen. In addition, the 3D environment model may include a virtual camera corresponding to the CCTV.

In one embodiment, the 3D environment model is based on a (2-dimensional) ground plane captured on the screen of the virtual camera and a (3-dimensional) ground plane implemented on the 3-dimensional virtual space. The view of the 3D environment model may be matched and mapped to the CCTV screen.

In one embodiment, the 3D environment model is matched to the virtual camera screen through the following equation,

[mathematical expression]

Here, {Ψi, λi} represents the position of the ith projected point on the 3D model space and the position of the ith point on the 2D screen of the virtual camera image, respectively, and dk is the 3D space considering the camera distortion (k). As a function that projects the point of the camera into the 2D space of the camera and calculates the distance to the point of the other 2D space, based on the specifications of CCTV, R and t may represent the angle and position of the virtual camera in the 3D space, respectively have.

In one embodiment, the step of measuring the actual size of the subject of the CCTV screen is to search for an actual size value mapped to a pixel size value at a 3D virtual location on a 3D environment model mapped to the CCTV screen in which the object is detected. and determining the searched actual size value as the actual size of the object. Here, the 3D virtual location is a location on a mapped 3D environment model corresponding to the location of the detected object in the CCTV screen.

A computer readable recording medium according to another aspect of the present invention may be readable by a computing device and may store program instructions operable by the computing device. The program command, when the program command is executed by the processor of the computing device, causes the processor to perform a method of measuring the actual size of an object using the 3D environment model according to the above-described embodiments.

The CCTV control system according to one aspect of the present invention may calculate measurement information about a dynamic subject photographed on a CCTV screen as well as measurement information about a static subject using a 3D environment model generated in advance based on actual measurements.

In addition, since the subject is measured using the two-dimensional image of the CCTV screen, additional images other than the two-dimensional image of the CCTV screen such as satellite photos are not required.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

BRIEF DESCRIPTION OF THE DRAWINGS To describe the technical solutions of the embodiments of the present invention or the prior art more clearly, drawings required in the description of the embodiments are briefly introduced below. It should be understood that the drawings below are for the purpose of explaining the embodiments of the present specification and not for limiting purposes. In addition, for clarity of explanation, some elements applied with various modifications, such as exaggeration and omission, may be shown in the drawings below.
1 is a conceptual diagram of a CCTV control system according to an embodiment of the present invention.
2 is a diagram illustrating a result of mapping a CCTV screen and a 3D environment model according to an embodiment of the present invention.
3 is a diagram for explaining a process of calculating a measured length of a subject on a CCTV screen through a mapped 3D environment model according to an embodiment of the present invention.
4 is a flowchart of a method of measuring the actual size of a subject using a 3D environment model according to an embodiment of the present invention.
5 is a flowchart of a process of mapping a virtual camera screen and a 3D environment model according to an embodiment of the present invention.
6 is a flowchart of a process of mapping a CCTV screen and a 3D environment model according to an embodiment of the present invention.

The terminology used herein is only for referring to specific embodiments and is not intended to limit the present invention. As used herein, the singular forms also include the plural forms unless the phrases clearly indicate the opposite. The meaning of "comprising" as used herein specifies particular characteristics, regions, integers, steps, operations, elements and/or components, and the presence or absence of other characteristics, regions, integers, steps, operations, elements and/or components. Additions are not excluded.

Although not defined differently, all terms including technical terms and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention belongs. Terms defined in commonly used dictionaries are additionally interpreted as having meanings consistent with related technical literature and currently disclosed content, and are not interpreted in ideal or very formal meanings unless defined.

In this specification, the pixel size of an object means a value expressing the size of an object on a 2D screen in units of pixels.

In this specification, the actual size of an object means a value expressed in a commonly used size unit of the size of an object in an actual 3D space. The commonly used size unit includes a metric unit or a non-metric unit, wherein the non-metric unit includes a yard, a mile unit, and the like.

Hereinafter, with reference to the drawings, look at the embodiments of the present invention in detail.

1 is a conceptual diagram of a CCTV control system according to an embodiment of the present invention.

Referring to Figure 1, CCTV control system 1 is one or more CCTV (100); object detection unit 250; and an actual size measuring unit 270 .

In certain embodiments, the CCTV control system 1 includes a 3D modeling unit 210; And/or may further include a mapping unit 230. In other specific embodiments, the CCTV control system 1 may further include a model storage unit (not shown) for storing a 3D environment model. The model storage unit receives and stores the pre-mapped 3D environment model from the outside of the CCTV control system 1. Hereinafter, for clarity of explanation, the present invention will be described in more detail with embodiments in which an operation of mapping a 3D environment model to a CCTV screen is performed by the CCTV control system 1.

The CCTV control system 1 according to embodiments may have aspects that are entirely hardware, entirely software, or partially hardware and partially software. For example, the system may collectively refer to hardware equipped with data processing capability and operating software for driving the hardware. In this specification, terms such as "unit", "module", "device" or "system" are intended to refer to a combination of hardware and software driven by the hardware. For example, the hardware may be a data processing device including a Central Processing Unit (CPU), a Graphic Processing Unit (GPU), or another processor. Also, software may refer to a running process, an object, an executable file, a thread of execution, a program, and the like.

The CCTV 100 is a device for acquiring images. The CCTV100 may acquire images by photographing an object on the screen. In the CCTV100, the shooting range may be expressed on the screen.

In one embodiment, the CCTV control system 1 may include one or more CCTV100 generating two-dimensional image data.

The 3D modeling unit 210 creates a 3D environment model for a space captured on a CCTV screen (hereinafter, “screen space”). The 3D modeling unit 210 includes actual measurement information of the screen space; and/or a 3D environment model in which the screen space is implemented as a 3D virtual space is generated based on the actual measurement information of the photographing object located on the screen space. The photographing object includes a static object disposed in screen space and/or a positionable dynamic object. Here, the static object is an object that generally does not move for a long time, and may include facilities (eg, roads, buildings, etc.) and structures (eg, furniture). A dynamic object is an object that is not a static object and may include, for example, a person or a car. In some embodiments, dynamic objects that have a history of appearing on a CCTV screen previously recorded in a corresponding space may be reflected in a 3D environment model.

When the space is a building, the 3D modeling unit 210 creates a 3D environment model for the building space based on actual measurement information of the space, such as BIM (Building Information Modeling) information.

When the space is outside the building, the 3D modeling unit 210 creates a 3D environment model for the building space based on external actual measurement information such as Geographic Information System (GIS) information.

The 3D environment model includes a structure obtained by rendering a corresponding space in three dimensions based on actual measurement information. Also, the 3D environment model may further include information about a corresponding space. The information about the space includes, for example, actual measurement information and meta data. The meta data may include background and object identification data.

A 3D modeling process of generating a 3D model based on actually measured information may be performed by various prior arts. For example, through a 3D modeling process disclosed by Korean Patent Application (Patent Publication No. 10-2008-0111991 (2008.12.14.)), a 3D environment model modeling a screen space in a three-dimensional structure may be created. .

Since the 3D environment model created by the 3D modeling unit 210 is based on actual measurement information, the actual size of the object may be calculated from the spatial size of the object located on the 3D environment model. The actual size of the object is calculated by determining the actual size corresponding to the spatial size of the object located on the 3D environment model.

In addition, the 3D modeling unit 210 creates a virtual camera corresponding to one or more CCTV100 included in the CCTV control system 1 on a 3D environment model. A virtual camera corresponding to the CCTV100 installed in the real environment is modeled on a 3D virtual space by the 3D modeling unit 210 .

The virtual camera is disposed on the 3D environment model to have the same shooting range as the CCTV100, that is, the same screen. The CCTV control system 1 may obtain a screen forming a shooting range of a virtual camera. For example, the CCTV control system 1 may acquire image data of a virtual camera screen in which a part of a 3D virtual space is expressed. The camera screen captured by the virtual camera is the same as that of the CCTV 100.

In one embodiment, the CCTV control system 1 may determine a view image of a 3D environment model as a screen of the CCTV100 through a virtual camera.

The mapping unit 230 maps the CCTV screen and the 3D environment model using the CCTV screen used to generate the 3D environment model.

In one embodiment, the mapping unit 230 maps the 2D pixel size of the CCTV screen to the 3D actual size of the 3D environment model by matching the view of the 3D environment model to the CCTV screen. The view of the 3D environment model may be implemented as an image capturing the 3D environment model based on a virtual camera, and thus corresponds to a screen of the virtual camera.

Since the CCTV screen used to create the 3D environment model is used for mapping, the objects appearing on the CCTV screen are already implemented in the 3D environment model. In addition, the CCTV screen matches the virtual camera screen.

The mapping unit 230 aligns the view of the 3D environment model and the virtual camera screen based on the 2D ground plane captured on the virtual camera screen and the 3D ground plane implemented in the 3D virtual space of the 3D environment model. do. Since the virtual camera screen matches the CCTV screen, the view of the 3D environment model and the virtual camera screen are eventually matched.

The mapping unit 230 may detect a 2D or 3D area corresponding to the ground plane of the 3D environment model.

The mapping unit 230 may project an arbitrary point Ψ in the 3D space onto the space of the 2D screen of the virtual camera. If the arbitrary point is the installation position of the virtual camera, projection is performed based on the position (t) and angle (R) of the virtual camera in the 3D space. The projected view may be implemented as a virtual camera screen.

As such, since the virtual camera screen represents a specific view consisting of the shooting range of the virtual camera from the installation point of the virtual camera in the three-dimensional space of the 3D environment model, the mapping unit 230 maps the two-dimensional area of the ground plane of the virtual camera screen to can also be detected.

Then, pixel lengths between 2D points on the CCTV screen and space lengths between 3D points on the virtual space corresponding to the 2D points are mapped. Mapping is based on individual positions of 2D pixels and distances between 2D pixels. This is because, even if the distance between pixels is the same, the 3D spatial distance may be different depending on the distortion of the camera.

Through this mapping operation, the spatial length on the 3D environment model may be obtained based on the information of the pixel length of the CCTV screen and the 2D point forming the pixel length.

As a result, the CCTV control system 1 may obtain information about the actual size of the 3D object represented by the space length using the mapped 3D environment model.

Also, the mapping unit 230 may minimize the distance between the projected point Ψ on the 3D space and the point λ on the 2D space of the virtual camera screen. Then, the virtual camera screen and the view of the 3D environment model are more accurately matched.

In one embodiment, the mapping unit 230 may match the screen of the virtual camera and the view of the 3D environment model through the following equation.

Here, {Ψi, λi} represents the position of the i-th point on the 3D model space and the position of the i-th point on the 2D screen of the virtual camera image, respectively. R and t represent the angle and position of the corresponding camera in 3D space, respectively. dk refers to a function that projects a point in a 3D space into a camera 2D space considering camera distortion (k) and calculates a distance to a point in another 2D space. This function dk is set based on the specifications of the CCTV 100.

The mapping unit 230 may match the view of the virtual camera screen and the 3D environment model using n (n>1 or more natural numbers) correspondence points. The mapping unit 230 may use, for example, four points (Ψi, λi, where i = {1, 2, 3, 4}), but is not limited thereto.

In some embodiments, correspondence point information may be input by a user. In some other embodiments, a corresponding point may be automatically selected.

Also, the mapping unit 230 may determine an object area of the CCTV screen corresponding to the object of the 3D environment model. Due to this, the actual size information of the object of the 3D environment model is mapped with the 2D size information of the object of the CCTV screen.

When such mapping is completed, the 2D pixel size of the object on the CCTV screen may be converted to the 3D actual size of the 3D environment model. For example, when two pixel points in the CCTV screen are selected, a measured distance mapped to the distance between the two pixel points is obtained.

2 is a diagram illustrating a result of mapping a CCTV screen and a 3D environment model according to an embodiment of the present invention.

Referring to FIG. 2 , the CCTV screen and the view of the 3D environment model are matched. In order to create a 3D environment model, ground truth information of an object on a 3D environment model has already been obtained. In addition, the location and size of objects on the CCTV screen were detected to create a 3D environment model. The position and size of the object may be expressed in units of pixels constituting the CCTV screen.

The CCTV control system 1 may determine a three-dimensional virtual location on the mapped 3D environment model of the object, corresponding to the location of the object in the CCTV screen, through the mapped 3D environment model. A three-dimensional virtual position on the rendered 3D environment model of the person corresponding to the position of the person in the CCTV screen of FIG. 2 may be determined. For example, a 3D virtual position of a person located in the center of the screen of FIG. 2 on the 3D environment model or a 3D virtual position of a person located below the screen on the 3D environment model is determined.

Then, the pixel size of the person at the center of the screen in FIG. 2 and the actual size at the 3D virtual location on the 3D environment model are mapped. In addition, the pixel size of the person located under the screen of FIG. 2 and the actual size in the 3D virtual position on the 3D environment model are mapped.

The object detection unit 250 may receive photographing data from the CCTV100. The photographing data may be implemented as an image of a CCTV screen. The image of the CCTV screen used by the object detector 250 for object detection may be the same as or different from the image of the CCTV screen used by the 3D modeling unit 210 for modeling. For example, the image of the CCTV screen used by the 3D modeling unit 210 may be taken as a sample before the image of the CCTV screen captured by the object detector 250.

The object detection unit 250 detects an object included in the CCTV image. As a result of the detection, the area occupied by the object in the CCTV image is obtained. The size and position of the area occupied by the object in the CCTV screen may be used as the size and position of the object.

In one embodiment, the object detection unit 250 may detect an object in the CCTV screen by applying the CCTV screen to the machine learning model.

The machine learning model includes a feature extraction part from an input image; and a part for determining whether a portion of the input image is an object or corresponds to a specific object based on the extracted feature. The machine learning model may be, for example, a CNN network or various models based thereon.

The actual size measurement unit 270 calculates the actual size of the object on the CCTV screen.

In one embodiment, the actual size measuring unit 270 may select some or all of the objects in the CCTV screen as the subject and calculate the actual size of the subject. The detected object includes a subject that is a size measurement target.

The subject may be selected in response to a user input. For example, when a user input designating an area of a specific object expressed in a CCTV screen is obtained, the object itself in the designated area and/or a plurality of objects related thereto may be selected as the subject. Here, the associated object is an object identical to or similar to the specific object. Similar objects are objects that share one or more properties, eg, objects belonging to the same type (or class).

The subject may be selected according to a preset subject rule. For example, when the subject rule includes information on a specific object, the specific object of the subject rule is automatically selected as the subject. Alternatively, when the subject rule includes information of a specific attribute (eg, object group), one or more objects belonging to the object group of the subject rule are automatically selected as the subject.

The actual size measuring unit 270 calculates the pixel size of the subject in the CCTV screen.

In an embodiment, a pixel size of a subject may be measured based on a point plane. The 2D pixel size of a subject in a CCTV screen may be measured as a pixel length from a ground plane in an area occupied by the subject. The pixel size from the ground plane may be obtained as the height of the subject.

However, the pixel size of the subject is not limited thereto. For example, a pixel size corresponding to a width of a subject may be measured.

The actual size measurement unit 270 converts the pixel size of the subject on the CCTV screen into the actual size of the subject using the mapped 3D environment model.

The actual size measurement unit 270 may measure the actual size of the subject on the CCTV screen by calculating a real size value mapped based on the location and pixel size of the subject in the CCTV screen.

Since the CCTV screen photographing the subject, which is the object of measuring the actual size, has already been mapped to the 3D environment model, the actual size measuring unit 270 corresponds to the object on the 3D space of the 3D environment model corresponding to the object position in the CCTV screen. You can also convert an object's pixel size to its actual size.

3 is a diagram illustrating a result of measuring an actual size from a pixel length according to an embodiment of the present invention.

Referring to FIG. 3 , the actual size measuring unit 270 measures a pixel length from the ground plane of the subject as the pixel size of the subject. The actual size measurement unit converts the pixel size of the subject into the actual size using a pre-mapped 3D environment model, that is, a 3D rendering model in which the view is mapped to the CCTV screen of the CCTV100 in which the subject is photographed.

For example, as shown in FIG. 3 , the actual size measurement unit 270 obtains a pixel size of a person by measuring a pixel length from the ground plane of the person. The actual size measurement unit 270 searches for an actual size value mapped to a value of a pixel size measured at a 3D virtual location on a mapped 3D environment model corresponding to a location in a CCTV screen of a person. The actual size measurement unit 270 determines the retrieved actual size value as the actual size value of the person.

Since the actual size of the subject does not simply depend on the pixel size within the screen, but rather considers the position of the object within the screen, even if the pixel size values of the person in the middle of the screen and the person below the screen in FIG. 3 are the same, the actual size values are different from each other. may be different. Conversely, even if the pixel size values of the person in the middle of the screen and the person below the screen in FIG. 3 are different, the actual size values may be the same.

It will be clear to those skilled in the art that the CCTV control system 1 may include other components not described herein. For example, the CCTV control system 1 includes a network interface, an input device for data entry, a memory for storing data, and an output device for display, printing or other presentation of data, including operations described herein. It may also include other hardware elements required for

A method of measuring the actual size of a subject using a 3D environment model according to another aspect of the present invention is performed by a computing device including a processor. The computing device including the processor may be performed by a computer, for example, the CCTV control system 1.

4 is a flowchart of a method of measuring the actual size of a subject using a 3D environment model according to an embodiment of the present invention.

Referring to FIG. 4, the method includes acquiring an image of a CCTV screen including a subject by the CCTV 100 (S430;) and calculating the actual size of the subject using a pre-mapped 3D environment model (S450). ). In certain embodiments, the method may further include step S410 of mapping the CCTV screen and the 3D environment model. The step S410 is performed before the step S450.

4 is a flowchart of a process of mapping a CCTV screen and a 3D environment model according to an embodiment of the present invention.

Referring to FIG. 4 , the step S410 includes: generating a 3D environment model (S411).

In step S411, a 3D environment model for the screen space of the CCTV screen is created based on the actually measured information on the screen space of the CCTV screen. Since it is created based on actual measurements, the actual size corresponding to the length from one point to another point on the 3D environment model may be obtained.

The step (S410) includes a step (S413) of creating a virtual camera. In the step S413, a virtual camera corresponding to the CCTV 100 actually installed is created on the 3D environment model. The screen image photographed by the virtual camera is the same as the screen image photographed by the corresponding CCTV (100). Therefore, the result of matching the screen of the virtual camera with another screen represents the result of matching the screen of the CCTV 100 .

The step (S410) includes a step (S415) of mapping the CCTV screen and the 3D environment model.

In one embodiment, the step S415 includes matching the virtual camera screen of the step 413 and the view of the 3D environment model of the step S411. When the 2D screen of the virtual camera and the 2D view of the 3D environment model are matched, the 2D screen of the CCTV screen and the 3D structure of the 3D environment model are matched.

In one embodiment, the matching of the 2D screen of the virtual camera and the 3D structure of the 3D environment model may be performed through Equation 1 above (S415).

In this way, when the 3D environment model is matched with the virtual camera screen, a 3D point on the 3D environment model corresponding to the 2D point in the CCTV screen is determined. In addition, the 3D size of the object rendered on the 3D environment model corresponding to the object located in the CCTV screen is determined as the actual size of the object on the CCTV screen.

5 is a flowchart of a process of mapping a CCTV screen and a 3D environment model according to an embodiment of the present invention.

Referring to FIG. 5 , the step S450 includes: detecting an object in the CCTV screen of step S430 (S451).

In the above step (S451), an object in the CCTV screen is detected. Due to object detection, a subject whose actual size is to be calculated may be acquired.

In one embodiment, the image of the CCTV screen in step S430 may be applied to a pre-learned machine learning model to detect objects in the CCTV screen.

The step (S450) includes: measuring a pixel length of a subject on a 2D screen (S453). When an object is detected in step S451, an area occupied by the detected object within the CCTV screen is determined, and the length of the area may be measured as the pixel length of the subject.

In one embodiment, a pixel length of an object may be measured based on a point plane.

In some embodiments, as shown in FIG. 5 , the step S450 may further include selecting some or all of the detected objects as subjects (S452).

The step (S450) includes a step (S455) of calculating the actual size of the subject represented on the CCTV screen using the mapped 3D environment model.

In one embodiment, the step S455 includes: converting the pixel size of the subject in step S453 into a three-dimensional size on the 3D environment model. A 3D location on the 3D environment model corresponding to the 2D location of the object in the CCTV screen is determined through the mapped 3D environment model. In addition, a 3D actual size mapped to a pixel size at a corresponding 2D location is retrieved through the mapped 3D environment model.

As a result, the 3D actual size of the subject having the pixel length of step S453 is obtained (S455).

The operation by the method of measuring the actual size of the subject using the CCTV observation system 1 according to the embodiments described above and the 3D environment model is at least partially implemented as a computer program and stored in a computer-readable recording medium. can be recorded. For example, implemented together with a program product consisting of a computer-readable medium containing program code, which may be executed by a processor to perform any or all steps, operations, or processes described.

The computer may be any device that may be integrated into or may be a computing device such as a desktop computer, laptop computer, notebook, smart phone, or the like. A computer is a device that has one or more alternative and special purpose processors, memory, storage, and networking components (whether wireless or wired). The computer may run, for example, an operating system compatible with Microsoft's Windows, Apple's OS X or iOS, a Linux distribution, or an operating system such as Google's Android OS.

The computer-readable recording medium includes all types of recording and identification devices in which data readable by a computer is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage and identification devices, and the like. In addition, computer-readable recording media may be distributed in computer systems connected through a network, and computer-readable codes may be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing this embodiment can be easily understood by those skilled in the art to which this embodiment belongs.

The present invention reviewed above has been described with reference to the embodiments shown in the drawings, but this is only exemplary, and those skilled in the art will understand that various modifications and variations of the embodiments are possible therefrom. However, such modifications should be considered within the technical protection scope of the present invention. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

Claims (12)

In the CCTV control system for measuring the actual size of a dynamically moving subject photographed by one or more CCTVs,
a 3D modeling unit for generating a 3D environment model in which the screen space is implemented as a three-dimensional virtual space based on actually measured information related to the screen space captured on the CCTV screen;
a mapping unit for mapping the 3D environment model to the CCTV screen;
An object detection unit for detecting a dynamic object in the image of the CCTV screen captured by the CCTV; and
A real size measurement unit for measuring a real size of a subject included in the detected dynamic object in a 3D real environment using a mapped 3D environment model;
The mapping unit,
In order to map the CCTV screen and the 3D environment model, based on the ground plane captured on the screen of the virtual camera and the ground plane implemented on the 3D virtual space Matching the screen of the virtual camera and the view of the 3D environment model CCTV control system.
The method of claim 1, wherein the 3D modeling unit,
It is further configured to generate a virtual camera corresponding to the one or more CCTVs on the three-dimensional virtual space,
The virtual camera is a CCTV control system, characterized in that having the same view as the corresponding CCTV screen.
delete The method of claim 2, wherein the mapping unit,
Match the virtual camera screen and the 3D environment model through the following equation,
[mathematical expression]

Here, {Ψi, λi} represents the position of the ith projected point on the 3D model space and the position of the ith point on the 2D screen of the virtual camera image, respectively, and dk is the 3D space considering the camera distortion (k). As a function of projecting a point of into a camera 2-dimensional space and calculating the distance to a point in another 2-dimensional space, based on the specifications of CCTV, R and t represent the angle and position of the virtual camera in 3-dimensional space, respectively Featured CCTV control system.
The method of claim 1, wherein the object detection unit,
A CCTV control system characterized by extracting a feature from an input image and detecting the object by applying the image of the CCTV screen to a machine learning model pre-learned to detect the object included in the CCTV screen based on the extracted feature. .
The method of claim 1, wherein the actual size measuring unit,
Searching for a real size value mapped to a pixel size value in a 3D virtual location on the mapped 3D environment model corresponding to the location of the detected object in the CCTV screen, and
CCTV control system, characterized in that for determining the actual size value retrieved as the actual size of the object
A method for measuring the actual size of a dynamically moving subject using a 3D environment model, performed by a processor, comprising:
Obtaining an image of a CCTV screen including a subject by a CCTV; and
Measuring the actual size of a dynamically moving subject on a CCTV screen using a pre-stored 3D environment model,
The pre-stored 3D environment model is mapped to the CCTV screen,
The 3D environment model is
Based on the ground plane captured on the screen of the virtual camera and the ground plane implemented in the 3D virtual space, the screen of the virtual camera and the view of the 3D environment model are matched and mapped to the CCTV screen.
The method of claim 7, wherein the 3D environment model,
A model generated by 3D rendering of the screen space based on actual measurement information of the screen space captured on the CCTV screen, characterized in that it comprises a virtual camera corresponding to the CCTV.
delete The method of claim 8, wherein the 3D environment model,
It is matched to the virtual camera screen through the following equation,
[mathematical expression]

Here, {Ψi, λi} represents the position of the ith projected point on the 3D model space and the position of the ith point on the 2D screen of the virtual camera image, respectively, and dk is the 3D space considering the camera distortion (k). As a function of projecting a point of into a camera 2-dimensional space and calculating the distance to a point in another 2-dimensional space, based on the specifications of CCTV, R and t represent the angle and position of the virtual camera in 3-dimensional space, respectively How to characterize.
The method of claim 8, wherein the step of measuring the actual size of the dynamically moving subject on the CCTV screen,
Retrieving an actual size value mapped to a pixel size value at a 3D virtual location on a 3D environment model mapped to a CCTV screen in which a dynamic object is detected; and
Determining the retrieved actual size value as the actual size of the object,
Wherein the 3D virtual location is a virtual location on a mapped 3D environment model corresponding to a location of a detected object in a CCTV screen.
A computer-readable recording medium that is readable by a computing device and stores program instructions operable by the computing device, wherein the processor, when the program instructions are executed by a processor of the computing device, A computer readable recording medium for performing a method of measuring a real size of a dynamically moving subject using the 3D environment model according to any one of claims 10 to 11.

Images