KR20230121951A - Image tracking system using object recognition information based on Virtual Reality, and image tracking method thereof - Google Patents

Abstract

The present invention relates to an image tracking system using object recognition information based on VR, and an image tracking method. The present invention comprises an image shooting system and a monitoring system to provide an effect of facilitating tracking and enabling search and classification of objects.

Description

Image tracking system using object recognition information based on Virtual Reality, and image tracking method thereof

[0001] The present invention relates to an image tracking system using VR-based object recognition information and an image tracking method.

In terms of volume, the surveillance system searches for space background images, and if necessary, images and information for each object in the background

Based on providing, a number of interlocked imaging systems are used to track objects found in the spatial background.

to facilitate tracking, and the tracking information is converted into 2D or 3D virtual reality (VR) images.

An image tracking system using VR-based object recognition information to facilitate tracking by providing information, and

It is about high image tracking method.

[0002] As modern society becomes more complex, society faces various social problems such as social crimes and disasters.

[0003] In particular, in recent years, violent crimes from women's social problems such as sexual harassment and sexual violence to serial murder are rapidly increasing.

In the meantime, each autonomous district is trying to reduce it even a little by installing CCTVs in each autonomous district.

[0004] As such, the CCTV system is a visual means designed to monitor situations or actions occurring in the surroundings. this

Such a CCTV system provides important information every week in arresting criminals, so it has an important position as one of the security systems.

The number of CCTV cameras is rapidly increasing.

[0005] Such a CCTV system is usually an algorithm for recognizing objects and people such as faces, objects, license plates, and recognition.

To provide information about the shooting time and location of image data taken centering on the subject to be photographed and such image data

this is commonly used.

[0006] However, these algorithms are close to the image taken from a camera far from the subject to be recognized.

Since it is not filmed, it is difficult to recognize objects or people, and wide-area shooting is not possible for high-resolution close-up images.

Therefore, since it is not possible to recognize objects or people in a wider range by using

There is a need for a synchronization program that provides possible context (situation) information together.

[0007] In order to solve this problem, Republic of Korea Patent Registration Publication No. 10-0967718 corresponding to prior art 1 "water

Video security system using chain-type information" allows the surveillance system to search for spatial background images and, if necessary,

It provides an image security system within spatial coordinates that can provide images and information for each object in the background.

[0008] However, in the prior art 1, a technique for tracking a recognized object by focusing only on enlargement information on an image

However, there are limitations that cannot be presented.

[0009] Republic of Korea Patent Publication Publication No. 10-2016-0008668 corresponding to prior art 2 "object recognition information display device and its

The present invention relates to a display device using object recognition information and a method thereof, and is particularly installed on a street.

Recognizes an object passing by and moves in the direction in which the object moves according to the moving speed of the object.

A display device and method using object recognition information are provided.

[0010] However, in the prior art 2, only the speed of the object is displayed, and like the prior art 1, object tracking through object recognition is performed.

There is a limitation that cannot suggest a method for this.

[0011] Republic of Korea Patent Publication No. 10-2015-0107423 corresponding to prior art 3 "pan-tilt integrated surveillance camera

"Video tracking control system" detects a moving object in a dark place and operates a narrow-angle camera and high-brightness LED auxiliary light source.

It relates to an image tracking control system for a pan-tilt-integrated surveillance camera capable of acquiring precise object images simultaneously.

[0012] However, in the prior art 3, there is a technical limitation in that an object image can be obtained only with a high-brightness LED auxiliary light source.

The main focus is on image acquisition through investigation of the object recognition of the main light source and auxiliary light source, and object recognition

There is a limit that cannot be presented for the continuity of object tracking through .

[0013] Republic of Korea Patent Registration Publication No. 10-1051715 corresponding to prior art 4 "image tracking target device" is an external server

A system interlocking unit that interlocks with data, a heating element that generates heat, and heat control data received from the external server

According to the heating control unit for controlling the amount of heat generated by the heating element, formed in a specific shape, receiving the heat of the heating element

Ah, verifying the performance of image tracking by including a thermal target shape module that causes a temperature change in the specific shape

Provides an image tracking targeting device and an image tracking targeting system used for image tracking verification in the laboratory before

there is.

[0014] However, in the prior art 4, it is used to verify image tracking in real time according to temperature change or light emission change.

It provides video tracking target devices, and sets connection status, communication status, inspection results, and scenarios between video tracking target devices.

We focus on providing a user interface (GUI) that shows status, analysis results, and target conditions, but based on image recognition,

There is a limitation that cannot be done, and, like the conventional technique 3, for the continuity of object tracking through object recognition

There are limitations that cannot be presented.

[0016] The present invention is to solve the above problem, in a panoramic image taken at a specific place at a specific time.

By synchronizing the generated spatial coordinates with the coordinates of objects and people in the close-up image, all objects or objects to be recognized are synchronized.

generates a VR image that can indicate the location of people, and information on the subject, which is an object to be recognized, as well as

Check detailed context (situation) information of other objects and people around you that can be related to the recognition of the subject through VR images

To provide an image tracking system using VR-based object recognition information and an image tracking method

It is for

[0017] In addition, the present invention allows the surveillance system to search for spatial background images, and, if necessary, images for each object in the background and

Based on providing information, a number of interlocked imaging systems for tracking objects found in the spatial background

to facilitate tracking, and convert tracking information into 2D or 3D virtual reality image information.

video tracking system using VR-based object recognition information to facilitate tracking by providing

This is to provide a tracking method.

[0018] In addition, the present invention recognizes an object based on a VR background image synthesized in 2D or 3D, and converts the recognized object to VR

By providing an object image, it is possible to track the desired object as it is possible to search and classify the object.

In addition, to selectively provide 2D VR image information or 3D VR image information according to the request of the tracker

It is to provide an image tracking system using VR-based object recognition information and an image tracking method.

[0019] However, the objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned are the following

It will be clear to those skilled in the art from the description.

In order to achieve the above object, an image tracking system using VR-based object recognition information according to an embodiment of the present invention,

A number of units are installed in succession in the surveillance area to photograph a specific surveillance space and provide the video information of the surveillance space.

The video is photographed and transmitted, and the original image of the close-up video is transmitted according to the search request from the remote central monitoring system.

Video recording system configured for each monitoring area provided; And, sensing from the image provided from the imaging system

Tracker mobile device to monitor space-time and provide monitoring information to the holder of the tracker mobile device

Vise and a monitoring system connected to each imaging system through a network;

The coordinates of a specific object are created from the background image of the projected panoramic image and their feature information is provided,

The monitoring system not only requests and receives an object's proximity image, but also predicts the path to the object.

To track through the video recording system and finally provide 3D image information to the tracker's mobile device

In the video tracking system using VR-based object recognition information, characterized in that, the video shooting system is a VR car

It has a camera module and a camera control module, and the VR camera module is a background image for capturing a background from a distance.

It includes a camera unit and a close-up image camera unit for taking close-up pictures of an object, wherein the background image camera unit can be viewed from a distance.

It is a background image recording means that allows you to capture an image of the surroundings in a panoramic form.

It is configured to take pictures of the surroundings at specific times according to the control of the camera control module,

The proximity image camera unit is a means for obtaining an image of an object by taking a close-up image of the object, and the background image camera unit

It consists of a PTZ (Pan/Tilt/Zoom) camera that rotates 360 degrees as well as adjusts the tilt angle up and down.

The camera control module includes an object recognition unit, a shooting control unit, an object map generation unit, a transmission control unit, an object map database, a first image database, and a pattern image database.

Create background coordinates in a specific space from the image obtained from the source and object to be recognized within the created background coordinates

As a means for recognizing (person, face, vehicle, etc.) or flying object, the image obtained from the background image camera unit

Receive the input and use the object pattern data on the pattern image database to extract the object area and create the object.

When recognizing and extracting the feature information of an object, the object is compared with the pattern image stored in the pattern image database.

Recognizes and extracts feature information of the recognized object, and the shooting control unit controls the background image camera unit to capture the background image

Taking an image and controlling the proximity image camera unit to take a close-up image of the object recognized by the object recognition unit

As a means for obtaining an image by controlling the background image camera unit and the proximity image camera unit in a manner,

By synchronizing the object recognized through the object recognition unit with the background coordinates to create the coordinates of the object, the object map generation unit

To generate object map information including solution coordinates and feature information of an object, and the object map generating unit

The object recognized through planting is synchronized with the background image to create the coordinates of the object, and the created 'coordinate' and 'object's

Object map information including 'characteristic information' is created, and when generating the object map information, identification information (ID) of the image capture system, object

Object coordinates and feature information for each frame of , photographing time information, close-up image camera unit from the central axis (x1) of the background image camera unit

Including the rotation angle (a1, a2) information of and the vertical tilt angle information of the close-up image camera unit, the equation of the image capture system

The star information (ID) includes a unique number and location information of the image capturing system 100, and the transmission control unit includes object map information and background information.

The compressed image of the video is controlled to be transmitted to the monitoring system, and according to the search request from the monitoring system

It provides the original image of the proximity image of the corresponding object, and compresses the object map information and background image with the monitoring system.

It is a means for controlling the transmission of images, and is requested according to the object search request and object tracking request from the monitoring system.

A control means for providing an original image of a proximity image of a requested object and other object map information, the object map data

The base stores and manages object map information, and the first image database stores proximity images and panoramic images.

and the pattern image database manages pattern images for object recognition by the object recognition unit.

The monitoring system includes a video reception monitoring unit, a search interface unit, a monitoring control unit, and a second video database.

system, reception information database, VR image database, background video pattern image database, and operator input

The image reception monitoring unit receives the image and object map information provided from the image capture system and transmits the image and

Characteristic information is displayed to the operator to provide a monitoring image, and the search interface unit for the monitoring image

The operator selects a search area through the operator input unit and selects a specific 'object search request' and 'object tracking request'.

It provides an operator interface so that the operator can select a recognition area for the monitoring image and select a specific object.

As a means to provide an operator interface to search and track the monitoring video, the monitoring operator

A selection and input operator interface (GUI; Graphic User) to select a search area and a tracking area for

Interface), and the monitoring and control unit designates an object map for the recognition area designated by the search interface unit.

Feature information is extracted from beams, displayed and controlled, and according to the operator's selection, a close-up image for the selected area is requested to take an image.

It displays and manages the proximity image information transmitted from the Young System, and displays and controls the search area designated by the search interface unit.

, display and control the feature information transmitted from the image capture system, and select the corresponding selection area according to the operator's choice.

A process for displaying and managing the proximity image information transmitted from the imaging system by requesting a proximity image for

In addition to providing the feature information of the object selected by the selection area according to the search request of the operator input unit

Compared with the object map information registered in the existing received information database, the object captured by the corresponding other imaging system

It provides a search process that searches and provides the result, and according to the operator's selection for the object corresponding to the proximity image

Another tracking request is transmitted to the video recording system, and the VR video transmitted from the video recording system is displayed, managed, and controlled.

The second image database is provided according to the background image (compressed image) and the search received from the image capturing system.

When the original image of the proximity image is stored and managed, the second image database includes information received for each identification information of the image capturing system.

The background image (compressed image) and the original image of the proximity image provided according to the search are stored and managed, and the received information database

When storing and managing the object map information received from the image capturing system, the received information database stores the image when capturing

The object map information received from the system is stored and managed according to the identification information of the video recording system, and the VR video database

IS uses the background image, object map information, and original image received from at least two image capture systems to

Stores and manages VR images generated by the monitoring and control unit, and the operator input unit 280 allows the operator to enter the video recording system.

For command input and search area selection from the monitoring screen, object search request (original video request), and object tracking request

and the monitoring and control unit includes a location tracking unit, a location prediction unit, a location transmission unit, a VR image generating unit, and a VR video

A transmission means is provided, and the location tracking means includes a background image received from an image capture system recognizing a current object, an object

Position tracking of the object selected by the operator from the operator input unit is performed using the body map information and the original image.

360° panorama is captured from the imaging system where the current object is located in real time for the selected object.

A background image and an original image are received, information on a rotation angle (a1) of the proximity image camera is extracted from the central axis (x1) of the background image camera from object map information, and then transferred to the location estimation means, and the location is predicted. Sudan is present

The background image captured as a 360° panorama received from the image capture system where the current object is located, the original image, and the

In terms of business, the object moves using the rotation angle (a1) information of the close-up camera unit from the central axis (x1) of the camera unit.

Next, the image capture system is extracted, and the orientation information of the object in the background image is extracted, which is received from the image capture system.

The background image and the original image captured as a 360° panorama are the background image pattern of the background image pattern image database.

Azimuth information in the background image captured as a 360° panorama of the object by extracting the azimuth area using the data

(east, west, south, north, or azimuth information) is recognized, and feature information of the object corresponding to the original image is extracted, and among the feature information

Recognizes distance information from the proximity image camera unit using a change in perspective according to a change in size, and the location example

The measuring unit uses the recognized azimuth information and distance information, and the unique number and location information of the current imaging system to determine the current location.

Among a plurality of imaging systems excluding the imaging system, at least one imaging system in which the object proceeds to the next

The above extraction is performed, and the current imaging of the object is performed by each imaging system extracted by the position prediction means.

The object map information generated by the Young System is transmitted through the network to request object tracking, and the VR video production

In Seongsudan, the recognition of an object matching the object feature information on the object map information by the imaging system to be progressed is

Upon completion, the background image captured as a 360° panorama from the recognized image capture system and the original image for the object

Example of processing after receiving information on the rotation angle (a2) of the proximity image camera from the central axis (x1) of the image and background image camera

A background image and an original image photographed as a 360° panorama received from the Jungin imaging system, and the location tracking means

Using the background image and original image received from the image capturing system recognizing the current object received by

When generating 3D virtual reality ('VR') image information, received from the image shooting system to be proceeded

The background image and the original image captured as a 360° panorama are stored in the background image pattern data of the background image pattern image database.

Azimuth information (East, West, South, North

or azimuth information), extracts the feature information of the object corresponding to the original image, and detects changes in size among the feature information.

After recognizing the distance information from the proximity imaging camera unit using the change in perspective,

A background image captured as a 360° panorama received from

After synthesizing the background image received from the imaging system to create VR image background information, the image capturing system to be proceeded

and orientation information received from each of the imaging systems recognizing the current object received by the location tracking means and

Recognizes the location coordinates of the object on the VR image background information generated by synthesis using the information, and converts the two original images into

Creates VR image information in which object information corresponding to the generated 3D polygon set is inserted, and the VR image

The moon unit provides VR image information to the tracker's mobile device through the network, and tracks the object in 3 steps.

It is tracked on a dimensional space, and the location tracking means is obtained from the object map information received from the image capturing system.

In addition to the rotation angle (a1) information of the proximity image camera unit from the central axis (x1) of the background image camera unit, the

The position predicting means when extracting the vertical tilt angle information of the proximity image camera unit and transmitting the information to the position predicting means

height information can be additionally generated, and when the VR image information is created, height information can also be included if the object is an air vehicle.

It is characterized by allowing

[0025] An image tracking system and an image tracking method using VR-based object recognition information according to an embodiment of the present invention are specific

Spatial coordinates generated from images captured as a panorama at a specific time in a place and objects and people in close-up images

By synchronizing the coordinates of, creating a VR image that can indicate the position of all objects or people to be recognized,

It provides an effect of being able to check not only the information of the subject, which is the object to be recognized, but also the detailed context (situation) information of other objects and people around the subject that can be related to the recognition of the subject through VR images.

In addition, an image tracking system using VR-based object recognition information according to another embodiment of the present invention, and an image tracking room

The law enables the surveillance system to search for spatial background images and provides images and information for each object in the background as needed.

Based on this, a number of interlocked imaging systems are used to track objects found in the space background.

It facilitates tracking, and tracking information is provided as 2D or 3D virtual reality image information.

provides the effect of

[0027] In addition, an image tracking system using VR-based object recognition information according to another embodiment of the present invention, and an image

The tracking method recognizes an object based on a VR background image synthesized in 2D or 3D, and converts the recognized object into a VR object.

By providing images, it is possible to track the desired object as searching and classifying the object is possible.

In addition, according to the request of the tracker, 2D VR image information or 3D VR image information can be selectively provided.

provide effect.

1 is a diagram showing an image tracking system using VR-based object recognition information according to an embodiment of the present invention.
FIG. 2 is a schematic diagram illustrating a concept of image tracking of an image tracking system using VR-based object recognition information of FIG. 1 .
Figure 3 is a monitoring control unit 230 of the monitoring system 200 of the video tracking system using the VR-based object recognition information of Figure 1
It is a block diagram showing the configuration of in detail.
FIG. 4 shows a plurality of image capturing systems 100 constituting the image tracking system using VR-based object recognition information of FIG. 1 and
It is a diagram showing the overall configuration including the monitoring system 200.
5 is a flowchart illustrating an image tracking method using VR-based object recognition information according to an embodiment of the present invention.

Hereinafter, a detailed description of a preferred embodiment of the present invention will be described with reference to the accompanying drawings. In the following the present invention

In describing, a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention.

If it is determined that it is possible, the detailed description will be omitted.

[0030] In the present specification, when one component 'transmits' data or signals to another component,

An element may transmit the data or signals directly to another element, or through at least one other element.

This means that data or signals can be transmitted to other components.

1 is a diagram showing an image tracking system using VR-based object recognition information according to an embodiment of the present invention. do

2 is a schematic diagram showing the image tracking concept of the image tracking system using VR-based object recognition information of FIG. Figure 3

The structure of the monitoring control unit 230 in the monitoring system 200 of the video tracking system using the VR-based object recognition information of FIG.

It is a block diagram showing the castle in detail. Figure 4 configures the image tracking system using the VR-based object recognition information of Figure 1

It is a diagram showing the overall configuration including a plurality of imaging systems 100 and a monitoring system 200.

[0032] Referring to FIGS. 1 to 4, a video tracking system using VR-based object recognition information captures an image of a surveillance space

Transmission control, and providing the original image of the proximity image according to the search request from the remote central monitoring system 200

The video recording system 100 configured for each monitoring area and the surveillance space from the images provided from the video recording system 100

In order to monitor and provide monitoring information to the holder of the tracker mobile device 400, the tracker mobile device

It consists of the monitoring system 200 connected to the IS 400 and each image capturing system 100 and the network 300.

[0033] The video recording system 100 is divided into a VR camera module 100a and a camera control module 100b.

[0034] Here, the VR camera module 100a includes a background image camera unit 110 for photographing a background from a distance and a close-up photograph of an object.

It includes a proximity image camera unit 120 for

[0035] At this time, the background image camera unit 110 allows the background image to be captured in a panoramic form from a distance.

is a means of filming. The background image camera unit 110 is configured to capture an image of the surroundings while rotating 360 degrees,

Images are captured at specific times according to the control of the shooting control unit 140 .

[0036] The proximity image camera unit 120 is a means for obtaining a detailed image of the object by taking a close-up of the object, and not only rotates 360 degrees in the same way as the background image camera unit 110, but also adjusts the tilt angle up and down. Possible PTZ (Pan/Tilt/Zoom) car

It is composed of Mera.

In addition, the camera control module 100b includes an object recognition unit 130, a shooting control unit 140, an object map generation unit 150, a transmission control unit 160,

It includes an object map database 170, a first image database 180, and a pattern image database 190.

[0038] The object recognition unit 130 receives the image obtained from the background image camera unit 110, and the pattern image database 190

The object area is extracted using the object pattern data, the object is recognized, and the feature information of the object is extracted.

[0039] More specifically, the object recognition unit 130 determines the background in a specific space from the image obtained from the background image camera unit 110.

Create coordinates and recognize objects (persons, faces, vehicles, etc.) or flying objects to be recognized within the created background coordinates

As a means for this, the object recognition unit 130 recognizes the object by comparing it with the pattern image stored in the pattern image database 190.

and extract feature information of the recognized object.

[0040] The shooting control unit 140 controls the background image camera unit 110 to capture a background image, and the object recognized by the object recognition unit 130

Controls the proximity image camera unit 120 to take a close-up image of the body. That is, the shooting control unit 140 is a background image camera unit.

As a means for obtaining an image by controlling the 110 and the proximity image capture unit 120, the object recognized through the object recognition unit 130

is synchronized with the background coordinates to create the coordinates of the object, and an object containing the coordinates created in this way and the characteristic information of the object.

It is a means for generating map information by the object map generator 150. Object map information is the expression of the imaging system

Star information (ID), object coordinates and feature information for each frame of the object, photographing time information, and the central axis (x1) of the background image camera unit 110

information about rotation angles (a1, a2) of the proximity image camera unit 120 and tilt angle information up and down of the proximity image camera unit 120 from

include The identification information (ID) of the imaging system includes a unique number and location information of the imaging system 100 .

[0041] The object map generation unit 150 synchronizes the object recognized through the object recognition unit 130 with the background image to determine the coordinates of the object.

and object map information including the generated 'coordinates' and 'characteristic information of the object'.

[0042] The transmission control unit 160 controls the transmission of the object map information and the compressed image of the background image to the monitoring system 200, and monitors

According to a search request from the system 200, an original image of a close-up image of a corresponding object is provided. That is, the transmission control unit

160 is a means for controlling transmission of object map information and a compressed image of a background image to the monitoring system 200, and also

According to the object search request and object tracking request from the monitoring system 200, the original image of the proximity image of the corresponding object,

It is a control means for providing external object map information.

[0043] The object map database 170 stores and manages object map information, and the first image database 180 stores proximity images and

Stores and manages Norama images.

[0044] The pattern image database 190 manages and stores pattern images for object recognition by the object recognition unit 130.

On the other hand, the monitoring system 200 includes an image reception monitoring unit 210, a search interface unit 220, a monitoring and control unit 230, a second video

database 240, reception information database 250, VR image database 260, background image pattern image database

s 270 and an operator input unit 280.

[0046] The image reception monitoring unit 210 receives the image and object map information provided from the image capture system 100 and receives the image and

Characteristic information is displayed to the operator to provide monitoring images.

[0047] The search interface unit 220 allows the operator to select a search area through the operator input unit 280 with respect to the monitoring image.

It provides operator interface to make specific 'object search request' and 'object tracking request'. That is, search inter

The face unit 220 allows an operator to search for and track a specific object by selecting a recognition area for the monitoring image.

As a means for providing an operator interface, a monitoring operator can search for a monitoring image and track an image.

A means for providing a selection and input operator interface (Graphic User Interface (GUI)) to select a station is provided.

all.

[0048] The monitoring and control unit 230 extracts feature information from the object map information for the recognition area designated by the search interface unit 220.

display control, and according to the operator's selection, a close-up image for the corresponding selection area is requested and the image capture system 100

Controls display management of proximity image information transmitted from

[0049] That is, the monitoring and control unit 230 is configured to search the search area designated by the search interface unit 220 from the image capturing system 100.

The received feature information is displayed and controlled, and according to the operator's selection, a close-up image is requested for the corresponding selected area.

In addition to providing a process for display management and control of the proximity image information transmitted from the photographing system 100, the reception information database 250 transmits characteristic information of an object selected by the selection area according to a search request of the operator input unit 280. to

Compared with the registered object map information, search for an object photographed by the corresponding other imaging system 100, and provide the result.

The ball provides a search process.

In addition, the monitoring and control unit 230 sends a tracking request according to the operator's selection for the object corresponding to the proximity image to the video recording system.

(100), display management control of the VR image transmitted from the image capturing system (100).

[0051] The second image database 240 is stored according to the background image (compressed image) received from the image capturing system 100 and the search.

Stores and manages the original image of the shared proximity image. More specifically, the second image database 240 includes an imaging system

Stores and manages the background image (compressed image) received for each identification information of the system 100 and the original image of the proximity image provided according to the search

do.

[0052] The reception information database 250 stores and manages object map information received from the imaging system 100. more specific

Ideally, the object map information received from the image capture system 100 is included in the reception information database 250 as the image capture system.

It is stored and managed according to the identification information of (100).

[0053] The VR image database 260 defines background images and object maps received from at least two or more image capturing systems 100

In addition, the VR video generated by the monitoring and control unit 230 is stored and managed using the original video.

[0054] The operator input unit 280 allows the operator to manually input commands to the video recording system 100 and select a search area from the monitoring screen,

It is configured for object search request (original video request) and object tracking request.

[0055] On the other hand, the monitoring and control unit 230 includes a location tracking unit 230a, a location prediction unit 230b, a location transmission unit 230c, and a VR image generating unit.

(230d) and VR image delivery means (230e).

[0056] The location tracking unit 230a includes the background image, object map information, and original text received from the image capturing system 100 recognizing the current object.

The location tracking of the object selected by the operator from the operator input unit 280 is performed using the image. more specific

Ideally, the position tracking means (230a) is the imaging system 100 where the current object is located in real time for the selected object.

Receives a background image and an original image captured in a 360 ° panorama from the camera unit 110

After extracting information on the rotation angle (a1) of the proximity imaging camera unit 120 from the central axis (x1) of the position prediction means (230b)

convey

[0057] In another embodiment of the present invention, the location tracking unit 230a is distributed from the object map information received from the image capturing system 100.

In terms of business, not only information on the rotation angle a1 of the proximity image camera 120 from the central axis (x1) of the camera unit 110, but also proximity image

Up and down tilt angle information of the image camera unit 120 may be extracted and transmitted to the position predicting means 230b, in which case

The position predicting means (230b) additionally generates information about the height, so that when the VR image information is generated, the object is inclined toward the aircraft.

height information should also be included.

[0058] The position prediction unit 230b is a 360° panoramic view received from the imaging system 100 where the object is currently located.

Rotation angle (a1) of the proximity image camera unit 120 from the central axis (x1) of the business image, original image, and background image camera unit 110

The next image capturing system 100 to which the object proceeds is extracted using the information.

[0059] More specifically, the position prediction means (230b) extracts the orientation information of the object in the background image, the image capture system (100)

The background image and the original image taken as a 360 ° panorama received from the background image pattern image database 270

By using the background image pattern data, the azimuth area is extracted and the azimuth is determined in the background image captured as a 360° panorama of the object.

Recognizes the beam (east, west, south, north or azimuth information), extracts the feature information of the object corresponding to the original image, and among the feature information

Distance information from the proximity image camera unit 120 is recognized by using a change in perspective according to a change in size.

[0060] The position predicting means (230b) is the recognized azimuth information and distance information, and the unique number and location information of the current imaging system 100

Among the plurality of imaging systems 100 excluding the current imaging system 100, the object progresses to the next using the beam

The imaging system 100 performs at least one extraction.

[0061] The position transmission means 230c is the current image of the object with each image capturing system 100 extracted by the position prediction means 230b.

The object map information generated by the imaging system 100 is transferred through the network 300 to request object tracking.

[0062] The VR image generating means (230d) matches the object feature information on the object map information by the image capturing system 100 to be in progress

When the recognition of the object is completed, the background photographed as a 360 ° panorama from the image capturing system 100 where the recognition is completed

An image, an original image of an object, and rotation angle a2 information of the proximity image camera unit 120 are received from the central axis x1 of the background image camera unit 110.

Accordingly, the VR image generating unit 230d captures a 360° panorama received from the image capturing system 100 to be proceeded.

The video recording system 100 recognizing the recorded background image and the original image and the current object received by the location tracking means 230a

2D or 3D virtual reality ('VR') image information using the background image and original image received from

generate More specifically, the VR image generating means (230d) is received from the video recording system 100 to proceed

Background image pattern data of background image pattern image database 270

Azimuth information (East, West, South, North

or azimuth information), extracts the feature information of the object corresponding to the original image, and detects changes in size among the feature information.

The distance information from the proximity image camera unit 120 is recognized by using the perspective change according to the distance. After that, VR image generating means

(230d) is a background image captured as a 360° panorama received from the image capture system 100 to be performed, and location tracking

The background image received from the image capturing system 100 recognizing the current object received by the means 230a is synthesized to create a VR image.

After creating the image background information, the current object received by the image capturing system 100 and the location tracking means 230a to be proceeded

A VR image generated by synthesis using the azimuth information and distance information received from each of the imaging systems 100 recognizing

It is recognized as the positional coordinates of an object on the image background information, and corresponds to a set of 3D polygons created using two original images.

generates VR image information into which object information is inserted.

[0064] The VR image transmission means 230e provides VR image information to the tracker mobile device 400 through the network 300,

Tracking of objects can be quickly tracked in 3D space.

[0065] Through such a configuration, the present invention is installed in a plurality of consecutively in the surveillance area to photograph a specific space to be monitored and capture the image.

An imaging system that provides information and monitoring that monitors images provided from the imaging system 100

In the image tracking system composed of the system 200, the coordinates of a specific object from the background image of the panoramic image

Generates and provides their characteristic information, requesting a close-up image of the corresponding object from the monitoring system 200 as needed

Not only can it be requested and received, but it can also predict the path to the object and track it through the imaging system to

It is characterized in that the original image information is finally provided to the tracker mobile device 400 .

5 is a flowchart illustrating an image tracking method using VR-based object recognition information according to an embodiment of the present invention. do

Referring to 5, the monitoring system 200 receives the background image and object from the imaging system 100 recognizing the current object.

The location tracking of the object selected by the operator from the operator input unit 280 is performed using map information and original images.

C (S110)

[0067] More specifically, the monitoring system 200 monitors objects in the area selected by the operator input unit 280 in real time.

The background image, the original image, and

Receives object map information, and from the object map information, the proximity image camera from the central axis (x1) of the background image camera unit 110

Information on the rotation angle a1 of the unit 120 is extracted.

[0068] In another embodiment of the present invention, the monitoring system 200 is distributed from the object map information received from the image capturing system 100

In terms of business, not only information on the rotation angle a1 of the proximity image camera 120 from the central axis (x1) of the camera unit 110, but also proximity image

Up and down tilt angle information of the image camera unit 120 may be extracted, and in this case, height information may be additionally generated.

By creating VR image information, height information can also be included if the object is an air vehicle.

[0069] After step (S110), the monitoring system 200 is a 360 ° wave received from the imaging system 100 where the current object is located

The background image captured in Norama, the original image, and the close-up image camera unit from the central axis (x1) of the background image camera unit 110

The next image capture system 100 to which the object proceeds is extracted using the rotation angle a1 information of (120) (S120).

[0070] More specifically, the monitoring system 200 extracts orientation information of an object from a background image, and the image capturing system 100

The background image and the original image taken as a 360 ° panorama received from the background image pattern image database 270

By using the background image pattern data, the azimuth area is extracted and the azimuth is determined in the background image captured as a 360° panorama of the object.

Recognizes the beam (east, west, south, north or azimuth information), extracts the feature information of the object corresponding to the original image, and among the feature information

Distance information from the proximity image camera unit 120 is recognized by using a change in perspective according to a change in size. After that, the monitor

The ring system 200 transmits the recognized azimuth information and distance information, and the unique number and location information of the current imaging system 100.

Among the plurality of imaging systems 100 excluding the current imaging system 100, an object proceeds to the next

The Young System 100 performs at least one extraction.

[0071] After step S120, the monitoring system 200 transmits the object map information generated by the current imaging system 100 for the object to each imaging system 100 extracted in step S120 to the network. Request object tracking by passing through (300)

(S130)

After step S130, the monitoring system 200 determines the object characteristics on the object map information by the imaging system 100 [0072] to be performed.

When the recognition of the object matching the information is completed, a 360° panoramic view is obtained from the image capturing system 100 where recognition is completed.

The background image captured by the camera, the original image of the object, and the background image close-up image from the central axis (x1) of the camera unit 110

Information on the rotation angle a2 of the camera unit 120 is received (S140)

[0073] After step (S140), the monitoring system 200 is a panorama of 360 ° received from the imaging system 100 to proceed.

The captured background image and original image, and the background image and original image received from the image capture system 100 recognizing the current object.

2D or 3D virtual reality ('VR') image information is generated using the image (S150)

[0074] More specifically, the monitoring system 200 is a 360 ° panorama received from the imaging system 100 to be progressed.

Orientation of the captured background image and the original image using the background image pattern data of the background image pattern image database 270

Azimuth information (East, West, South, North or azimuth information) in the background image captured as a 360° panorama of the object by extracting the area

Recognizes and extracts the feature information of the object corresponding to the original image, and analyzes the change in perspective according to the change in size among the feature information.

It recognizes the distance information from the proximity image camera unit 120 by using. After that, the monitoring system 200 is the image to be progressed

The background image captured as a 360° panorama received from the photographing system 100 and the received by the location tracking means 230a

After synthesizing the background image received from the imaging system 100 recognizing the current object to create VR image background information,

An image capturing system recognizing the current object received by the image capturing system 100 and the location tracking means 230a to be progressed

(100) of the object on the VR image background information generated according to the synthesis using the direction information and distance information received from each

Recognized as positional coordinates, object information corresponding to a set of 3D polygons created using two original images is inserted.

Create VR image information.

[0075] After step (S150), the monitoring system 200 sends VR image information to the tracker mobile device 400 through the network 300.

By providing, it is possible to quickly track the object in the 3D space (S160).

[0076] The present invention can also be implemented as computer readable codes on a computer readable recording medium.

A computer-readable recording medium is any type of device that stores data that can be read by a computer system.

Includes lock device.

[0077] Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices.

and the like, and also includes those implemented in the form of a carrier wave (eg, transmission over the Internet).

[0078] In addition, the computer-readable recording medium is distributed in a computer system connected to a network, and the computer is distributed in a distributed manner.

Readable code can be stored and executed. And a functional program for implementing the present invention

RAM, code and code segments can be easily inferred by programmers skilled in the art.

[0079] As described above, the present specification and drawings disclose preferred embodiments of the present invention, although specific terms are used.

Although used, it is only used in a general sense to easily explain the technical content of the present invention and aid understanding of the invention.

However, it is not intended to limit the scope of the present invention. In addition to the embodiments disclosed herein, the technical aspects of the present invention

It is common knowledge in the art to which the present invention belongs that other modifications based on the above are possible.

It is self-evident to you.

[0080] 100: imaging system
100a: VR camera module
100b: camera control module
110: background image camera unit
120: close-up camera unit
130: object recognition unit
140: shooting control unit
150: object map generation unit
160: transmission control unit
170: object map database
180: first image database
190: pattern image database
200: monitoring system
210: video receiving monitoring unit
220: search interface unit
230: monitoring and control unit
230a: location tracking means
230b: position prediction means
230c: position transmission means
230d: VR image generating means
230e: VR image transmission means
240: second image database
250: reception information database
260: VR image database
270: background image pattern image database
280: operator input unit
300: network
400: tracker mobile device

Claims (1)

A number of units are installed in succession in the visual area to photograph a specific space for monitoring and provide video information to the area of the surveillance space.
The image is photographed and transmitted, and the original image of the close-up image is transmitted according to the search request from the remote central monitoring system.
Video recording system configured for each monitoring area provided; And, sensing from the image provided from the imaging system
Tracker mobile device to monitor space-time and provide monitoring information to the holder of the tracker mobile device
Vise and a monitoring system connected to each imaging system through a network;
The coordinates of a specific object are created from the background image of the projected panoramic image and their feature information is provided,
The monitoring system not only requests and receives an object's proximity image, but also predicts the path to the object.
To track through the video recording system and finally provide 3D image information to the tracker's mobile device
In the image tracking system using VR-based object recognition information characterized by,
The video recording system includes a VR camera module and a camera control module, and the VR camera module controls the background from a distance.
It includes a background image camera unit for photographing and a proximity image camera unit for closely photographing an object,
The number of background image captures that allow the background image camera unit to capture surrounding images in a panoramic form from a distance.
In other words, it is configured to capture an image of the surroundings while rotating 360 degrees, and the image of the surroundings is controlled by the camera control module.
Depending on the language, it is filmed at a specific time,
The proximity image camera unit is a means for obtaining an image of the object by taking a close-up image of the object, and the background image camera
It consists of a PTZ (Pan/Tilt/Zoom) camera that not only rotates 360 degrees, but also adjusts the tilt angle up and down.
becomes,
The camera control module includes an object recognition unit, a shooting control unit, an object map generation unit, a transmission control unit, an object map database, a first
It includes an image database and a pattern image database,
The object recognition unit generates background coordinates in a specific space from an image obtained from the background image camera unit,
As a means for recognizing objects (persons, faces, vehicles, etc.) or flying objects to be recognized within the created background coordinates,
Receiving the image obtained from the background image camera unit, object pattern data on the pattern image database
When the object is recognized by extracting the object area using the pattern image database when the feature information of the object is extracted,
Contrast with the stored pattern image to recognize the object, extract feature information of the recognized object,
The shooting control unit controls the background image camera unit to capture a background image, and the object recognized by the object recognition unit
The background image camera unit and the proximity image camera unit are controlled by a method of controlling the proximity image camera unit to capture a close-up image of the body.
As a means for obtaining an image by controlling the video camera unit, the object recognized through the object recognition unit is assigned to background coordinates.
By synchronizing and generating the coordinates of an object, the object map including the coordinates and feature information of the object is generated by the object map generating unit.
To generate road information,
The object map generation unit synchronizes the object recognized through the object recognition unit with the background image to determine the coordinates of the object.
and object map information including the created 'coordinates' and 'characteristic information of the object' is created, and when generating the object map information,
Identification information (ID) of the imaging system, object coordinates and feature information for each frame of the object, shooting time information, middle of the background image camera
Rotation angle (a1, a2) information of the proximity imaging camera unit from the mandrel axis (x1), tilt angle information of the proximity imaging camera unit up and down
Including, the identification information (ID) of the imaging system includes a unique number and location information of the imaging system 100,
The transmission control unit transmits and controls object map information and a compressed image of a background image to the monitoring system,
According to the search request from the monitoring system, the original image of the close-up image of the corresponding object is provided, and the monitoring system
It is a means for controlling the transmission of object map information and compressed images of background images to the system, and from the monitoring system
To provide the original image of the proximity image of the object requested according to the object search request and object tracking request, and other object map information
is a means of control,
The object map database stores and manages object map information, the first image database stores and manages proximity images and panoramic images, and the pattern image database stores patterns for object recognition by the object recognition unit.
Manage and store the unknown,
The monitoring system includes a video reception monitoring unit, a search interface unit, a monitoring and control unit, a second video database, and a reception center.
It includes a information database, a VR image database, a background image pattern image database, and an operator input unit,
The image reception monitoring unit receives the image and object map information provided from the image capture system, and displays images and features.
Information is displayed to the operator to provide monitoring images,
The search interface allows the operator to select a search area for the monitoring image through the operator input unit.
It provides an operator interface to make a specific 'object search request' and 'object tracking request'.
To provide an operator interface so that the operator can select a recognition area to search for and track a specific object.
As a means for this, the monitoring operator can select a search area and a tracking area for monitoring images
Provides an input operator interface (GUI; Graphic User Interface),
The monitoring and control unit extracts feature information from object map information for a recognition area designated by the search interface unit.
display control, and according to the operator's selection, a close-up image for the selected area is requested and transmitted from the image capture system.
Display management and control of proximity image information, and for the search area designated by the search interface unit, the image capture system
Display and control the feature information transmitted from the operator, and request a close-up image for the corresponding selected area according to the operator's selection
to provide a process for display management control of the proximity image information transmitted from the imaging system,
According to the search request of the operator input unit, the characteristic information of the object selected by the selection area is stored in the received information database.
It compares with the object map information registered in the corresponding image capture system to search for objects photographed and provides the result.
It provides a search process that performs image capture according to the operator's selection of a tracking request for an object corresponding to a close-up image.
system to display, manage, and control VR images transmitted from the video recording system;
The second image database is provided according to the background image (compressed image) and the search received from the image capturing system.
When the original image of the proximity image is stored and managed, the second image database includes information received for each identification information of the image capturing system.
The background image (compressed image) and the original image of the proximity image provided according to the search are stored and managed.
The reception information database stores and manages the object map information received from the image capturing system, the reception information data base.
In this database, object map information received from the imaging system is stored and managed according to the identification information of the imaging system.
as,
The VR image database includes background images, object map information, and original images received from at least two image capturing systems.
Using this video, the VR video generated by the monitoring and control unit is stored and managed,
The operator input unit 280 allows the operator to manually input commands to the video recording system and select a search area from the monitoring screen,
It is configured for object search request (original video request) and object tracking request,
The monitoring and control unit obtains a position tracking means, a position prediction means, a position transmission means, a VR image generation means, and a VR image transmission means.
compare,
The location tracking unit detects a background image, object map information, and an original image received from an imaging system recognizing a current object.
When tracking the location of an object selected from the operator input unit by the operator using the
background image and original image captured as a 360° panorama from the image capture system where the object is located in real time.
and rotation of the proximity image camera unit from the central axis (x1) of the background image camera unit based on object map information.
After extracting the angle (a1) information, it is transmitted to the position predicting means,
The position predicting means is a background image captured as a 360° panorama received from an image capture system where the object is currently located.
Information on the original image and the rotation angle (a1) of the proximity image camera unit from the central axis (x1) of the background image camera unit
The next image capture system to be processed by the object is extracted using , and orientation information of the object is extracted from the background image.
The background image and the original image recorded as a 360° panorama received from the imaging system are converted to the background image pattern image data.
The background image captured as a 360° panorama of the object by extracting the azimuth area using the background image pattern data of the database.
Recognizes azimuth information (east, west, south, north or azimuth information) in
By extracting, distance information from the proximity image camera unit is obtained by using a change in perspective according to a change in size among feature information.
recognize,
The location predicting means uses the recognized azimuth information and distance information, and the unique number and location information of the current imaging system to determine the next imaging system from among a plurality of imaging systems excluding the current imaging system.
At least one or more extractions are performed, and each image capturing system extracted by the position prediction means determines the object.
Request object tracking by transmitting object map information generated by the current imaging system through the network,
The VR image generating means is an object that matches the object feature information on the object map information by the image capturing system to be in progress.
When the recognition of the object is completed, the background image and the object
The original image for the body and the rotation angle (a2) information of the proximity image camera unit from the central axis (x1) of the background image camera unit are obtained.
After Shinhan, the background image and original image captured as a 360° panorama received from the image capture system to be in progress,
The background image and the original image received from the image capture system recognizing the current object received by the existing location tracking means
When generating 2-dimensional or 3-dimensional virtual reality ('VR') image information using
The background image and the original image taken as a 360 ° panorama received from the background image pattern image database
By using the background image pattern data, the azimuth area is extracted and the azimuth is determined in the background image captured as a 360° panorama of the object.
Recognizes the beam (east, west, south, north or azimuth information), extracts the feature information of the object corresponding to the original image, and among the feature information
After recognizing the distance information from the proximity image camera unit using the perspective change according to the size change, the image to be progressed
A background image photographed as a 360° panorama received from the photographing system and a current object received by the location tracking means
After synthesizing the background image received from the image shooting system that recognizes the body to create VR image background information,
Received from each of the imaging system and the imaging system recognizing the current object received by the location tracking means
Recognizes the location coordinates of the object on the VR image background information generated by synthesis using the azimuth information and distance information,
Creates VR image information in which object information corresponding to a 3D polygon set generated using the original image of is inserted,
The VR image transmission means provides VR image information to the tracker's mobile device through the network,
to be tracked in a three-dimensional space,
The location tracking means is based on the object map information received from the image capturing system, and the central axis of the background image camera unit.
(x1) as well as information on the rotation angle (a1) of the proximity image camera unit, as well as up and down of the proximity image camera unit
When tilt angle information is extracted and transmitted to the position predicting means, the position predicting means additionally provides height information.
When generating VR image information, it is characterized in that height information can also be included in the case of an object flying
Video tracking system using VR-based object recognition information.