KR102353724B1 - Apparatus and method for monitoring city condition - Google Patents

Abstract

A monitoring device is provided. The monitoring device may include: an acquisition unit configured to acquire image data and location information of the set area from an image unit that captures the set area; an integrator for integrating the plurality of image data obtained from the plurality of image units by location; a setting unit for setting a region of interest in the integrated image integrated by the integrating unit; and an analysis unit that analyzes the state information of the region of interest.

Description

CITY CONDITION MONITORING APPARATUS AND METHOD FOR MONITORING CITY CONDITION

The present invention relates to an apparatus and method for monitoring city conditions.

Recognition of dangerous situations and intrusions using camera image information is a universal technology, and research is being conducted in many areas. However, the research is focused on autonomy and advancement of recognition using artificial intelligence technology, rather than monitoring real-time city conditions by integrating multiple information. In addition, it is difficult to find examples of implementation and application of a system that analyzes information obtained from mobile and stationary robots, reflects and analyzes real-time city state information, and proposes city state-based city management matters.

Korean Patent Publication No. 10-1285169 discloses an automatic maintenance system for lighting lamps using a track moving body. However, there is a limitation in that the monitoring of the facility is performed only locally.

Korean Patent Publication No. 10-1285169

An object of the present invention is to provide a monitoring apparatus and method for analyzing city state information to prevent various disasters.

The monitoring apparatus of the present invention includes: an acquisition unit configured to acquire image data and location information of the set area from an image unit that captures the set area; an integrator for integrating the plurality of image data obtained from the plurality of image units by location; a setting unit for setting a region of interest in the integrated image integrated by the integrating unit; and an analysis unit that analyzes the state information of the region of interest.

A monitoring apparatus of the present invention includes: a setting unit for setting a region of interest in image data obtained by photographing a setting region; and an analysis unit that analyzes the state information of the region of interest, wherein the analysis unit compares a normal state of the region of interest with a current state of the region of interest, and the analysis unit determines that the usual state and the current state are greater than or equal to a set value If it is analyzed that there is a difference, it may be determined that an abnormal situation has occurred in the ROI.

The monitoring method of the present invention includes: acquiring image data and location information for each city area through cameras of a plurality of mobile robots or fixed robots; filtering the personal information included in the image data; analyzing city state information including at least one of a city security and safety state, an urban environment state, and an urban facility state by using the image data from which the personal information has been filtered; The city state abnormal situation is monitored by using the analysis result of the city state information, and the city state information, the analysis result of the city state information, and the urban state abnormal situation in at least one of a police server, a firefighting server, and a city management server providing at least one; may include.

The monitoring apparatus and method of the present invention can monitor the entire city in real time by integrating image data (including voice) captured by a plurality of robots.

According to the present invention, an ROI may be set in an integrated image representing an entire city, and different types of city states may be automatically recognized for each ROI.

The automatically identified city state may be provided to one of a firefighting server, a police server, and a city management server capable of handling a city state of a corresponding area of interest, for example, crime, damage to facilities, and the like.

According to the present invention, a multi-robot-based city state integrated control system that collects multi-robot-based information, analyzes city state information for disaster prevention, and presents problems can be provided. Through this, the problem of disaster prevention through city information collection and the problem of manpower utilization in the city management environment and the additional resource consumption problem required for information analysis can be solved, and real-time city status monitoring can be guaranteed.

In the present invention, a system and method for recognizing and analyzing information obtained from a plurality of robots to monitor real-time city status, and to propose a management operation based on the monitored city status can be provided.

1 is a schematic diagram showing a monitoring device of the present invention.
2 is a schematic diagram showing the operation of the analysis unit.
3 is a schematic diagram showing the movement of the acquisition unit.
4 is a schematic diagram showing the operation of the setting unit.
5 is a flowchart illustrating a monitoring method of the present invention.
6 is a diagram illustrating a computing device according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, the embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

In the present specification, duplicate descriptions of the same components will be omitted.

Also, in this specification, when it is said that a certain element is 'connected' or 'connected' to another element, it may be directly connected or connected to the other element, but other elements in the middle It should be understood that there may be On the other hand, in this specification, when it is mentioned that a certain element is 'directly connected' or 'directly connected' to another element, it should be understood that the other element does not exist in the middle.

In addition, the terms used herein are used only to describe specific embodiments, and are not intended to limit the present invention.

Also, in this specification, the singular expression may include the plural expression unless the context clearly dictates otherwise.

Also, in this specification, terms such as 'include' or 'have' are only intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and one or more It is to be understood that the existence or addition of other features, numbers, steps, operations, components, parts, or combinations thereof, is not precluded in advance.

Also in this specification, the term 'and/or' includes a combination of a plurality of described items or any item of a plurality of described items. In this specification, 'A or B' may include 'A', 'B', or 'both A and B'.

Also, in this specification, detailed descriptions of well-known functions and configurations that may obscure the gist of the present invention will be omitted.

1 is a schematic diagram showing a monitoring device 100 of the present invention. 2 is a schematic diagram illustrating the operation of the analysis unit 170 . 3 is a schematic diagram illustrating the movement of the acquisition unit 110 . 4 is a schematic diagram illustrating the operation of the setting unit 150 .

The monitoring apparatus 100 shown in the drawing may include an acquisition unit 110 , an integrator 130 , a setting unit 150 , an analysis unit 170 , and a provision unit 190 .

A monitoring target of the monitoring device 100 of the present invention may include an administrative area such as a city. Administrative areas may have a size that is difficult to photograph with only a single robot. In order to monitor an administrative district such as a city in real time, it is necessary to collect a plurality of image data of different set regions within the administrative district, and to form a single integrated image representing the entire large administrative district.

When a single integrated image is formed, the monitoring device 100 applies the same analysis algorithm to determine whether there are any abnormalities in crime occurrence, crime prevention, road environment, street environment, atmospheric environment, public transportation facilities, city safety facilities, resident welfare facilities, etc. It can automatically generate or analyze state information indicating

The obtaining unit 110 may obtain image data and location information of the set area from the imaging unit 10 that captures the set area.

The acquisition unit 110 may acquire the image data i and the position information n from the imaging unit 10 that is fixed at a set position or moves to a plurality of positions. The imaging unit 10 may include various robots that automatically or semi-automatically capture a set area. The robot fixed to the set position may include a Closed Circuit Television (CCTV) or the like. A robot moving via a plurality of locations may include an unmanned vehicle equipped with a camera. In some cases, a photographing machine mounted on a vehicle driven by a driver may also be included in the category of a mobile robot.

The acquisition unit 110 may filter the personal information p included in the image data. As an example, the acquisition unit 110 may mosaic-process an individual's face appearing in the image data. In the case of identification of the crime prevention state, filtering of personal information may be completed to a degree of mosaic processing of the face. On the other hand, in the case of grasping the state of a facility in India, the person (personal information p) present in the current image data i1 is not required. In this case, the final image data i2 in which the entire person is filtered or removed may be generated by overwriting the entire person in the current image data i1 with the background b of the previously captured image data i0.

The acquisition unit 110 may communicate with a plurality of robots by wire or wireless. In order to reduce the communication load, a plurality of acquisition units 110 may be provided.

The integrator 130 may integrate the plurality of image data i obtained from the plurality of image units 10 for each location.

The integrator 130 may generate an integrated image j in which a plurality of image data i classified for each location is arranged according to the location information of the map m by using the location information n. The integrated image j may include all of the entire area z that the monitoring device 100 is a monitoring target.

The setting unit 150 may set a region of interest in the integrated image integrated by the integrating unit 130 . The setting unit 150 may set a region of interest in image data obtained by photographing the setting region.

The analyzer 170 may automatically analyze the state information of the ROI.

The analyzer 170 may analyze the state information indicating the state of the city by using the image data from which the personal information has been filtered.

The providing unit 190 may provide the state information analyzed by the analyzing unit 170 to at least one of the fire fighting server 92 , the police server 91 , and the city management server 93 . The provider 190 may include a server communication module that communicates with the firefighting server 92 , the police server 91 , and the city management server 93 by wire or wireless.

The state information of the ROI may be classified into a plurality of types. A region of interest may be set according to the type of state information. The region of interest is an area unit that serves as a standard for automatically determining whether a state is abnormal, and one or more may exist in one screen. The size of each ROI may be the same as or different from each other.

The urban state may be classified into an urban security state, an urban safety state, an urban environment state, an urban facility state, and the like. In accordance with this, the setting unit 150 may divide the region of interest into a first region ①, a second region ②, and a third region ③.

The first area ① is an area related to an urban security state or an urban safety state, and may be set by the setting unit 150 .

The second area ② is an area related to the urban environment, and the third area ③ is an area related to urban facilities, which may be set by the setting unit 150 , respectively.

The analysis unit 170 may analyze the state information by dividing the first area ①, the second area ②, and the third area ③, respectively.

Each area can be classified or divided as shown in Table 1 below.

city state classification Details monitoring items city security and
safe state crime State of Crime Against Persons state of crime crime prevention city patrol status urban environment road environment Road management conditions (road conditions, lanes, crosswalks, etc.) Road congestion management (illegal parking, etc.) status street environment Attachment management status (signs, banners, advertisements, etc.) Street management status (sidewalks, street lights, street vendors, etc.) atmospheric environment Air pollution status (fine dust, etc.) city facilities public transport facilities
(bicycle, bus, etc.) Conditions of bike paths and related facilities Maintenance status of public transportation facilities city safety facility Management status of natural disaster prevention facilities (water and sewerage, etc.) Man-made disaster prevention facility management status (traffic lights, etc.) Resident Welfare Facility Status of living infrastructure (childcare, park, school, culture, physical education, health care, etc.) Status of resident welfare facilities (social welfare center, dementia support center, etc.)

The first area ① can be divided into crime occurrence items and crime prevention items. The crime occurrence item may include a state in which a crime has occurred against a person or a state in which a crime against objection has occurred. In order to monitor the occurrence of crimes against people, the setting unit 150 may set a dark alley between buildings, a sidewalk, and the like, as the first area ①. The analysis unit 170 may recognize an image-based violence situation or determine an emergency situation based on voice recognition. The analysis unit 170 may recognize a violent situation or determine an emergency situation through the analysis of image data (possible to include audio data), in particular, the first area ①. As an example, the analysis unit 170 may determine an emergency situation in which a person has fallen by using an emergency model that determines that a person has fallen through machine learning. Alternatively, when a person's fist or foot moves along a preset trajectory, the analysis unit 170 may determine that it is violence and determine that it is a violent situation. The analysis result of the analyzing unit 170 may be provided to the police server 91 , the fire fighting server 92 , and the city management server 93 by the providing unit 190 .

The crime prevention item may include city patrol status.

The analysis unit 170 may determine the patrol state based on the moving path record of the mobile robot. The analysis unit 170 may analyze the search area in the image-based map.

The setting unit 150 may set image data captured by the mobile robot as the second area. If the image of the mobile robot is fixed in one place, the analysis unit 170 may determine that the city patrol by the mobile robot is abnormal. Alternatively, the analysis unit 170 may analyze the position information of the mobile robot and determine that the city patrol state is abnormal when the mobile robot does not move.

The second area ② may be divided into a road environment item, a street environment item, and an atmospheric environment item.

The road environment item may include a road management status or a road congestion occurrence management status.

The analysis unit 170 may recognize the painted state of the lane and crosswalk, or recognize the state of the vehicle in the illegal parking area.

The setting unit 150 may set a road through which the vehicle passes and a place where the vehicle is parked as the second area ②. The analysis unit 170 may check the state of the lane drawn on the road and the state of the crosswalk through the analysis of the second area ② set by the setting unit 150, or may recognize illegal parking and stopping. The analysis unit 170 may set the lane or crosswalk photographed at the time when the lane or crosswalk is newly drawn on the road as the reference image, and may compare the subsequently photographed image with the reference image. When a set value of a ratio of a color other than the original color is satisfied based on the painting area of the reference image, the analysis unit 170 may determine the painting state as defective. The painting state defect information may be provided to the city management server 93 by the providing unit 190 .

The street environment item may include an attachment management state or a distance management state.

The analysis unit 170 may recognize the state of the attachment of the outer wall of the building, or recognize the state of the sidewalk and the obstacle in the sidewalk. The analysis unit 170 may analyze a state of an attachment attached to a wall of a building, a street lamp, or the like, or analyze a street management state such as an amount of garbage existing on the street.

The setting unit 150 may set a wall of a building, a street lamp, a sidewalk where garbage is concentrated, and the area around the trash can as the second area ②. The analysis unit 170 may apply a machine learning algorithm or an image analysis algorithm for analyzing the attachment management state or distance management state to the analysis of the region set as the second region ②. If the analysis result satisfies the set value, the analysis unit 170 may determine the attachment management state or the distance management state as defective. The bad state information may be provided to the city management server 93 by the providing unit 190 .

The air environment item may include air pollution conditions.

The analysis unit 170 may recognize the atmospheric state based on the atmospheric color and the cloud state. Alternatively, the analysis unit 170 may acquire the atmospheric state of the corresponding area from the Meteorological Agency database.

The setting unit 150 may set the sky part included in the image data as the second area ② in order to analyze the atmospheric color and the cloud state. The analysis unit 170 may apply the atmospheric color analysis algorithm and the cloud analysis algorithm to the second area ② set by the setting unit 150 to analyze the atmospheric state of the corresponding area.

The third area ③ can be divided into a public transportation facility item, a city safety facility item, and a resident welfare facility item.

The public transportation facility item may include a bicycle road and related facility condition or a public transportation facility maintenance condition.

The analysis unit 170 may recognize whether the facility is damaged or obtain a maintenance state from the maintenance record database of the used facility.

For example, the setting unit 150 may set a bicycle road or bicycle road-related facility included in the image data as the third area ③. The setting unit 150 may set the bus stop facilities (such as awnings, information boards, etc.) and taxi stop facilities included in the image data as the third area ③.

The analysis unit 170 may recognize or determine whether the used facility is damaged by using a comparison between the reference image and the current image of the third area ③, a damage analysis algorithm, and the like. The determination result of the analysis unit 170 may be provided to the city management server 93 by the providing unit 190 .

The urban safety facility item may include a management status of a natural disaster prevention facility or a management status of a man-made disaster prevention facility.

The analysis unit 170 may recognize whether the preventive facility operates normally, or may obtain an inspection record of the disaster prevention facility.

Natural disaster prevention facilities may include storage bins to compensate for sand that can be sprinkled on roads in snowy conditions. The human-made disaster prevention facility may include a traffic light on the road.

The setting unit 150 may set the storage box or the traffic light to the third area ③. The analysis unit 170 may recognize or analyze whether the third area ③ set by the setting unit 150 is damaged. The analysis result of the analysis unit 170 may be provided to the city management server 93 or the fire fighting server 92 by the providing unit 190 .

The resident welfare facility item may include a living infrastructure status or a resident welfare facility status.

The analysis unit 170 may recognize whether there is external damage in the living infrastructure and welfare facilities.

The setting unit 150 may set a part of the image data in which resident welfare facilities such as social welfare centers and dementia support centers are displayed as the third area ③. The analysis unit 170 may analyze and determine whether a building appearing in the third area (3) is damaged or not.

Basically, the analyzer 170 may compare the usual state of the ROIs ①, ②, and ③ with the current state of the ROI. When it is analyzed that the normal state and the current state have a difference greater than or equal to a set value, the analysis unit 170 may determine that an abnormal situation has occurred in the ROI.

The setting unit 150 may classify objects included in the image data according to public security, road, street, atmosphere, and facilities. The setting unit 150 may set one or more regions of interest ①, ②, and ③ in the set region according to the classification result.

The analyzer 170 may apply different setting values to each ROI set in the image data according to a classification result.

The analyzer 170 may classify and monitor each ROI included in the image data by using each set value set differently.

5 is a flowchart illustrating a monitoring method of the present invention.

The monitoring method of the present invention may be performed by the monitoring device 100 shown in FIG. 1 .

The monitoring apparatus 100 may acquire image data and location information for each city area through cameras of a plurality of mobile robots or stationary robots ( S510 ). When a plurality of robots are provided as an operation performed by the acquisition unit 110 , a plurality of acquisition units 110 may also be provided to reduce a communication load and a processing load.

The monitoring apparatus 100 may filter the personal information included in the image data (S520). Except in special circumstances such as face identification of criminals, personal information such as an individual's face should be protected. The acquisition unit 110 may mosaic the face included in the image data or bring an existing image of an area in which a person exists to overwrite the person.

The monitoring apparatus 100 may analyze urban state information including at least one of an urban security and safety state, an urban environment state, and an urban facility state by using the image data from which the personal information has been filtered ( S530 ). The city state information may be analyzed by the analysis unit 170 . If necessary, the integrator 130 may be provided for integrating a plurality of image data obtained from a plurality of robots corresponding to the image unit 10 into a single integrated image. In addition, a setting unit 150 for setting a region of interest in the integrated image may be additionally provided.

The monitoring apparatus 100 may monitor an abnormal city state by using the analysis result of the city state information. The monitoring device 100 may provide at least one of city state information, an analysis result of urban state information, and an urban state abnormal situation to at least one of the police server 91, the fire fighting server 92, and the city management server 93. There is (S540). Monitoring of the urban state abnormal situation may be performed by the analysis unit 170 . The city state information, the analysis result, and the abnormal situation may be provided to the external servers 91 , 92 , and 93 by the providing unit 190 .

6 is a diagram illustrating a computing device according to an embodiment of the present invention. The computing device TN100 of FIG. 6 may be a device described herein (eg, the monitoring device 100 , etc.).

In the embodiment of FIG. 6 , the computing device TN100 may include at least one processor TN110 , a transceiver device TN120 , and a memory TN130 . In addition, the computing device TN100 may further include a storage device TN140 , an input interface device TN150 , an output interface device TN160 , and the like. Components included in the computing device TN100 may be connected by a bus TN170 to communicate with each other.

The processor TN110 may execute a program command stored in at least one of the memory TN130 and the storage device TN140. The processor TN110 may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to an embodiment of the present invention are performed. The processor TN110 may be configured to implement procedures, functions, and methods described in connection with an embodiment of the present invention. The processor TN110 may control each component of the computing device TN100 .

Each of the memory TN130 and the storage device TN140 may store various information related to the operation of the processor TN110. Each of the memory TN130 and the storage device TN140 may be configured as at least one of a volatile storage medium and a nonvolatile storage medium. For example, the memory TN130 may include at least one of a read only memory (ROM) and a random access memory (RAM).

The transceiver TN120 may transmit or receive a wired signal or a wireless signal. The transceiver TN120 may be connected to a network to perform communication.

On the other hand, the embodiment of the present invention is not implemented only through the apparatus and/or method described so far, and a program for realizing a function corresponding to the configuration of the embodiment of the present invention or a recording medium in which the program is recorded may be implemented. And, such an implementation can be easily implemented by those skilled in the art from the description of the above-described embodiment.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims are also presented. It belongs to the scope of the invention.

10...video unit 91...police server
92...Fire Server 93...City Management Server
100...monitoring device 110...acquisition department
130...Integration unit 150...Setting unit
170...analysis department 190...provider department

Claims (8)

a setting unit for setting a region of interest in image data obtained by photographing a setting region;
an analysis unit analyzing state information of the region of interest;
Including; providing unit that communicates with the city management server;
The analysis unit confirms the state of the lane drawn on the road through which the vehicle passes or the state of the crosswalk,
The analysis unit sets the image of the lane or crosswalk taken at the time when the lane or crosswalk is newly drawn on the road as a reference image, and compares the image of the lane or the crosswalk taken later with the reference image,
When defining the image of the lane or crosswalk taken after the time when the lane or crosswalk is newly drawn on the road as 'image taken later',
In the image taken after the above, if the ratio of the color other than the original color based on the painted area of the lane or the crosswalk of the reference image satisfies the set value, the painting state is determined as defective, ,
The providing unit provides information indicating the failure of the painting state to the city management server,
The setting unit divides the region of interest into a first region, a second region, and a third region,
The first area is an area related to a city policing state or a city safety state,
The second area is an area related to an urban environment,
The third area is an area related to urban facilities,
The analysis unit analyzes the state information by dividing the first region, the second region, and the third region, respectively,
The first area is divided into crime occurrence items and crime prevention items,
The crime occurrence item includes a state of occurrence of a crime against a person or a state of occurrence of an objectionable crime,
The analysis unit recognizes an image-based violence situation or determines an emergency situation based on voice recognition,
The crime prevention item includes city patrol status,
The analysis unit determines the patrol state based on the moving path record of the mobile robot, or analyzes the search area in the image-based map,
The second area is divided into a road environment item, a street environment item, and an atmospheric environment item,
The road environment item includes a road management state or a road congestion occurrence management state,
The analysis unit recognizes the paint state of the lane and crosswalk, or recognizes the state of the vehicle in the illegal parking area,
The street environment item includes an attachment management state or a distance management state,
The analysis unit recognizes the state of attachment of the outer wall of the building, or recognizes the state of the sidewalk and the obstacle in the sidewalk,
The air environment item includes an air pollution state,
The analysis unit recognizes the atmospheric state based on the atmospheric color and cloud state, or obtains the regional atmospheric state from the Meteorological Agency database,
The third area is divided into a public transportation facility item, a city safety facility item, and a resident welfare facility item,
The public transportation facility item includes a bicycle road and related facility condition or a public transportation facility maintenance condition,
The analysis unit recognizes whether the facility is damaged or acquires the maintenance status from the maintenance record database of the facility used,
The city safety facility item includes a natural disaster prevention facility management state or a man-made disaster prevention facility management state,
The analysis unit recognizes whether the preventive facility operates normally, or obtains an inspection record of the disaster prevention facility,
The resident welfare facility item includes a living infrastructure status or a resident welfare facility status,
The analysis unit recognizes whether there is external damage inside living infrastructure and welfare facilities,
The setting unit classifies the objects included in the image data according to security, road, street, atmosphere, and facilities,
The setting unit sets one or more regions of interest in the setting region according to the classification result,
The analysis unit applies different setting values according to the classification result to each ROI set in the image data,
The analysis unit monitors each region of interest included in the image data by using each set value set differently from each other.
According to claim 1,
An acquisition unit for obtaining image data and location information of the set area from an image unit for photographing the set area, and an integrator for generating an integrated image by integrating a plurality of the image data obtained from a plurality of the image units for each location are provided become,
The acquisition unit acquires the image data and the position information from a robot fixed to a set position or moving a plurality of positions,
The integrator generates the integrated image in which a plurality of the image data classified for each location is arranged according to the location information by using the location information.
According to claim 1,
An acquisition unit is provided for acquiring image data and location information of the set area from an image unit that captures the set area,
The acquisition unit filters the personal information included in the image data,
The analyzing unit is a monitoring device for analyzing the state information indicating the state of the city by using the image data from which the personal information has been filtered.
delete delete delete delete delete

Images