KR20140074513A - Monitoring Method using image description information and apparatus thereof - Google Patents

Abstract

Disclosed are a monitoring method and a monitoring device using image description information. A monitoring method using image description information according to an embodiment of the present invention includes steps of receiving an image captured by an image capturing device; detecting at least one interesting object from the received image; extracting the feature of the detected interesting object; and generating description information on the interesting object on the basis of the extracted feature of the interesting object. The method further includes determining whether an image is a monitoring target image on the basis of the generated description information on the interesting object; and combining the description information on the interesting object and information on the image in the case of the monitoring target image and providing the combined information to a user. A large scaled image may be monitored efficiently.

Description

Technical Field [0001] The present invention relates to a monitoring method using image description information,

The present invention relates to monitoring using image description information. More specifically, the present invention relates to a method and apparatus for recognizing an object of interest in an image photographed by a video photographing apparatus such as a CCTV or a DVR to generate description information, (Or a video image pickup apparatus) that needs to be observed or monitored in consideration of the monitoring condition, and further, a monitoring method using image description information that can inform the monitor (or user) will be.

Currently CCTV / DVR video surveillance systems for security and safety purposes are being rapidly installed and installed. Korea's Ministry of Public Administration and Security has established a CCTV integrated control center that integrates and manages CCTVs for about 200 municipalities from 2011 to 2015. These CCTV integrated control centers manage hundreds to thousands of CCTVs, but monitoring personnel monitor between 5 and 50 people.

Currently, CCTV monitoring is a method that people record directly to DVR and search later. Recently, CCTVs have been integrated into large-scale units such as cities or buildings. However, since a small number of CCTVs are monitored by a small number of people, problems such as accident prevention and security situation are not monitored. have.

In other words, current CCTV monitoring monitors several tens to several hundreds of CCTV monitors with a few people, so it is difficult to know what is happening on many monitors and it is difficult to know what objects are passing.

Accordingly, it is necessary to provide effective monitoring apparatus and method for large-scale CCTV operators. In the present invention, a monitoring method and apparatus using video description information are disclosed. Korean Patent Laid-Open No. 10-2011-0106414 entitled "Video descriptor generating device" has been proposed as a prior art using a conventional descriptor.

The prior patent relates to a video descriptor generating device, a video descriptor generating method, and a video descriptor generating program for video retrieval capable of detecting a similar or the same moving picture section among a large number of moving pictures, and can be applied to retrieve similar or the same video from many videos In particular, the illegal copying video is read in the same section of video, and the illegal copying image is searched by expressing the overall shape or shape of the image as a vector value instead of recognizing the object or the like.

Korean Patent Laid-Open No. 10-2011-0106414 (published on September 28, 2011)

In order to solve the above problems, an object of an embodiment of the present invention is to efficiently monitor a large-scale image using descriptive information of an object of interest in an image photographed by a video photographing apparatus such as a CCTV or a DVR And a monitoring method and apparatus using the image depiction information that can be monitored.

More specifically, the present invention generates object description information of interest objects in an image captured through face recognition and face recognition, such as a person and a vehicle, in order to efficiently monitor a plurality of CCTVs with a small number of persons, The system automatically detects the image or image capturing device that needs to be observed or monitored in consideration of the generated description information, the information about the captured image, and further the monitoring condition, and notifies the monitor agent thereof to efficiently monitor the large-scale CCTV with a small number of persons The present invention is directed to a method and an apparatus that can be used in a mobile communication system.

According to another aspect of the present invention, there is provided a monitoring method using image description information, the method comprising: detecting at least one object of interest from the received image, ; Extracting characteristics of the detected object of interest; Generating descriptive information of the object of interest based on the extracted characteristic of the object of interest; Determining whether the image is a monitoring target image based on the generated descriptive information of the object of interest; And combining the descriptive information of the object of interest and the information about the image in the case of the monitoring object image, and providing the combined information to the user.

The determining step may determine whether the object of interest is a preset monitored object of interest based on the generated descriptive information of the object of interest.

The determining step may determine whether the generated descriptive information of the object of interest corresponds to a preset monitoring condition.

The image capturing apparatus may include a CCTV, and the information about the image may include at least one of an installation place of the image capturing apparatus, a time at which the image was captured, and a space where the image was captured.

The characteristic of the object of interest may include a vehicle type, a vehicle type, a vehicle color, license plate recognition if the object of interest is a vehicle, and includes gender, key, body shape, clothes color, .

A monitoring apparatus using image description information according to an exemplary embodiment of the present invention includes a detector for detecting at least one object of interest from the received image upon receiving an image captured by the image capturing apparatus; An extracting unit for extracting characteristics of the detected object of interest; A generating unit for generating a script of the object of interest based on the extracted characteristic of the object of interest; A determination unit for determining whether the image is a monitoring target image based on the generated description information of the object of interest; And a providing unit for combining the description information of the object of interest and the information about the image in the case of the monitoring object image and providing the combined information to the user.

The determination unit may determine whether the object of interest is a preset monitored object of interest based on the generated descriptive information of the object of interest.

The determination unit may determine whether the generated descriptive information of the object of interest corresponds to a preset monitoring condition.

According to the present invention, it is possible to efficiently monitor or monitor a large-scale image using descriptive information about an object of interest in an image captured by a video photographing apparatus such as CCTV, DVR, and the like.

More specifically, the present invention generates object description information of interest objects in an image captured through face recognition and face recognition, such as a person and a vehicle, in order to efficiently monitor a plurality of CCTVs with a small number of persons, The system automatically detects video or image capturing devices that need to be observed or monitored in consideration of the generated description information, information about captured images, and further monitoring conditions, and informs monitor personnel to efficiently monitor a large-scale CCTV with a small number of persons have.

In other words, the present invention can reduce the monitoring personnel for CCTV deployed not only in public institutions such as police, but also in the private sector through efficient monitoring of large-scale CCTV with a small number of personnel, thereby reducing monitoring cost.

In addition, the present invention can contribute to not only security but also CCTV industry by efficiently managing integrated control center and spreading it to private monitoring center in future and more effective CCTV operation.

1 shows a system configuration of an embodiment for explaining a monitoring apparatus according to the present invention.
FIG. 2 shows a configuration of a monitoring apparatus using image description information according to an embodiment of the present invention.
FIG. 3 is a flowchart illustrating an operation of a monitoring method using image description information according to an embodiment of the present invention.
FIG. 4 shows an operational flow diagram of an embodiment of step S330 shown in FIG.

Other objects and features of the present invention will become apparent from the following description of embodiments with reference to the accompanying drawings.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the term "comprising" or " comprising " is intended to specify the presence of stated features, integers, steps, operations, elements, parts or combinations thereof, , But do not preclude the presence or addition of one or more other features, elements, components, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

Preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Hereinafter, a monitoring method and apparatus using image description information according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 to 4.

1 shows a system configuration of an embodiment for explaining a monitoring apparatus according to the present invention.

1, the system according to the present invention includes an image capturing apparatus 110 and an integrated control center or a monitoring center, and the image capturing apparatus 110 and the monitoring center are connected through a wired or wireless network .

The image capturing apparatus 110 is a device for capturing an image, which is shown as CCTV, but it is not limited thereto and may include all apparatuses for capturing an image such as a DVR.

In this case, each of the image capturing apparatuses 110 may acquire not only the captured image but also information about the captured image, for example, an installation place where the image capturing apparatus is installed or position information of the image capturing apparatus, Information on the space, identification information given to the image photographing apparatus, and the like to the monitoring center.

The monitoring center includes a storage device 120, a monitoring device 130, and a monitor 140.

The storage device 120 stores all information related to the present invention, such as images received from the image capturing device 110, information about images, and scripts for objects generated by the monitoring device 130.

The monitoring device 130 is a monitoring device using a script according to the present invention and is a monitoring device for monitoring a target object included in the image, for example, a vehicle, a person, etc., based on information about the image and the image received from the image- Generates descriptive information, generates monitoring information in consideration of information on the generated script and image, and information previously set by the user, and provides the generated monitoring information to the monitor 140 so that the user, that is, Provide the necessary information.

Here, the monitoring device 130 extracts the characteristics of the object of interest included in the image, generates descriptive information using the extracted characteristics of the object of interest, generates information about the generated descriptive information and the image, For example, if the corresponding image is determined as an image requiring monitoring, the monitoring unit 140 may convert the image into a natural language that can be easily read by the monitoring personnel, for example, Korean.

In this case, the characteristics of the object of interest may include gender, key, body shape, clothes color, and face recognition if the object of interest is a person, and may include vehicle type, vehicle type, vehicle size, vehicle color, And the characteristics of the other objects may include size and color. Of course, the objects of interest are not limited to persons and vehicles, but may include objects such as bicycles and motorcycles.

The monitoring apparatus according to the present invention will be described in detail with reference to FIG.

FIG. 2 shows a configuration of a monitoring apparatus using image description information according to an embodiment of the present invention.

2, a monitoring apparatus 130 according to the present invention includes a receiving unit 210, a detecting unit 220, an extracting unit 230, a generating unit 240, a determining unit 250, and a providing unit 260 .

The receiving unit 210 receives information on the image and the image photographed by the image photographing apparatus.

Here, the information about the image may include location information or location information of the image photographing apparatus, time at which the image was photographed, identification number of the image photographing apparatus, and information about the space where the image was photographed.

The detection unit 220 detects at least one object of interest from the image received by the reception unit 210.

In this case, the detection unit 220 may detect an object of interest through comparison of frames constituting the image, and may detect an object of interest through comparison of real-time or sequentially received images. But is not limited thereto.

The extraction unit 230 extracts characteristics of the object of interest detected by the detection unit 220.

The characteristics of the object of interest may include gender, key, body shape, clothes color, face recognition if the object of interest is a person, and may include vehicle type, vehicle type, vehicle size, vehicle color, license plate recognition And the characteristics of the other objects may include size and color. Of course, the objects of interest are not limited to persons and vehicles, but may include objects such as bicycles and motorcycles.

The generating unit 240 generates descriptive information on each of the objects of interest based on the characteristics of the object of interest extracted by the extracting unit 230.

In this case, the descriptive information about the object of interest includes all the characteristics of the object of interest, and is used to identify the feature of the object of interest. The descriptive information includes information about the object of interest, such as a child, It can also be used for searching for missing persons, searching for people to be searched, etc. through comparison.

The determining unit 250 determines whether the image received by the receiving unit 210 is a monitoring target image based on the descriptive information of the object of interest generated by the generating unit 240. [

That is, the determination unit 250 determines that the CCTV is a CCTV that needs to be monitored.

At this time, the judging unit 250 compares the monitoring condition set by the monitoring staff or the system, for example, night, woman, alley, etc. with the description information of the interested object, and if the condition is appropriate, The photographing apparatus can be determined as the monitoring target image or the monitoring target apparatus.

Further, the determination unit 250 may determine that the monitoring target image is a monitoring target image that needs to be monitored when a situation that does not match the video position occurs.

When the determination unit 250 determines that the corresponding image is a monitoring target image, the providing unit 260 generates monitoring information by combining the description information of the interested object and the image information, and transmits the generated monitoring information to the user, Provide to monitoring personnel.

For example, if a woman is detected and the time is night and the place where the woman is walking is an alley, the slave 260 uses the condition of night time and alleyway with one woman of interest, "I am walking alone" and by providing this interpreted monitoring information to the monitoring personnel, I can monitor the situation that can happen to the woman. Therefore, it is possible to protect the woman by monitoring in depth the situation of the woman who may occur in the dark night and the alley.

As described above, the monitoring apparatus according to the present invention recognizes and depicts an object of interest in an image photographed by the image photographing apparatus, informs the monitoring agent of the monitored object image determined using the descriptive information of the object of interest, As a monitoring agent, you can monitor large CCTVs, which can reduce the cost of monitoring.

In addition, the present invention stores the description information of an object of interest by matching it with a corresponding image photographing device, thereby making it possible to easily apply it to searching for missing persons and searching for a missing person through comparison with characteristics of objects of interest such as a lost person or a wanted person, Furthermore, it is possible to easily detect an event, a path to an accident, and the like, thereby reducing the time for resolving an incident or the like.

FIG. 3 is a flowchart illustrating an operation of a monitoring method using image description information according to an embodiment of the present invention.

Referring to FIG. 3, a method according to the present invention receives an image photographed by a photographing apparatus from a photographing apparatus (S310).

Here, the image capturing apparatus may include a CCTV and a DVR, and step S310 may receive not only the image from the image capturing apparatus, but also information on the image capturing apparatus and information on the image together.

The information about the image capturing apparatus may include an identification number for the image capturing apparatus, place information or location information where the image capturing apparatus is installed, and space information that is captured by the image capturing apparatus. Time, space information on which the image was captured, and the like.

The image received in step S310 may be images or moving images to be photographed in real time.

When the image is received in step S310, the target object is detected from the received image, the characteristic of each detected target object is extracted, and the description information of each target object is generated based on the extracted characteristic of the target object (S320, S330).

Here, the characteristic of the object of interest may include gender, key, body shape, clothes color, face recognition if the object of interest is a person, and vehicle type, vehicle type, vehicle size, vehicle color, And the properties of the other objects may include size and color. Of course, the objects of interest are not limited to persons and vehicles, but may include objects such as bicycles and motorcycles.

If the descriptive information about the object of interest is generated, it is determined whether the image or the image capturing apparatus that photographed the corresponding image is to be monitored based on the generated descriptive information of the object of interest (S340).

Here, in step S340, based on the generated description information about the object of interest, it can be determined whether the object of interest is a monitored object by comparing the object of interest, that is, the monitored object of interest preset by the monitoring person.

In step S340, it is determined whether the object corresponds to the condition set by the user based on the generated script for at least one object of interest. For example, if the image corresponds to a situation condition of "female "," alleyway ", "night ", step S340 may determine the image as a monitoring target image.

If it is determined in operation S340 that the corresponding image is a monitoring target image, the monitoring information is generated by combining the script of the object of interest and the image information, and the generated monitoring information is provided to the user, that is, the monitoring agent in operation S350.

Here, the method of providing the monitoring information may include displaying the monitoring information to the monitor, providing the monitoring information to the terminal of the monitoring agent in the form of a text message, or various other methods.

FIG. 4 shows an operational flow diagram of an embodiment of step S330 shown in FIG.

Referring to FIG. 4, the step of generating descriptive information (S330) checks the type of the detected interest object and determines whether the identified interest object is a vehicle or a person (S410, S420).

If it is determined in step S420 that the detected object of interest is a vehicle, the license plate of the vehicle is recognized and the shape of the vehicle, the color of the vehicle, and the size of the vehicle are detected (S430, S440).

Here, the items of steps S430 and S440 may be detected through image processing, and the type of vehicle may be detected using the detected items. Of course, in order to detect the type of the vehicle, information about the vehicle corresponding to the shape of the vehicle and the size of the vehicle must be stored in advance. At this time, the vehicle type may be previously modeled and stored according to the vehicle size and the vehicle type.

When the characteristics of the vehicle are extracted in steps S430 and S440, the vehicle description information is generated using the characteristics of the extracted vehicle (S450).

On the other hand, if it is determined in step S420 that the type of the object of interest is a human, the face of the detected person is recognized, and the sex, key, body shape, color of clothes to be worn, and the like are detected (S460, S470).

Here, the items about the characteristics of a person can also be obtained through image processing. The gender and the key can be obtained through various modeling previously set for the gender and the key, and the predetermined reference size or the like in the image can be used The person's height may be acquired, and the sex of the person may be acquired in consideration of the body shape and the like.

When the characteristics for the person are extracted in steps S460 and S470, the person description information is generated using the characteristics of the extracted person (S480).

The monitoring method using the video description information according to an embodiment of the present invention may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described with reference to particular embodiments, such as specific elements, and specific embodiments and drawings. However, it should be understood that the present invention is not limited to the above- And various modifications and changes may be made thereto by those skilled in the art to which the present invention pertains. Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

Claims (1)

Detecting at least one object of interest from the received image upon receiving the image captured by the image capture device;
Extracting characteristics of the detected object of interest;
Generating descriptive information of the object of interest based on the extracted characteristic of the object of interest;
Determining whether the image is a monitoring target image based on the generated description information of the object of interest; And
Combining the descriptive information of the object of interest and information about the image in the case of the monitoring object image and providing the combined information to a user;
The method comprising the steps of:

Images