KR102282373B1 - Position Verification System For Confirming Change In MMS Image - Google Patents

Abstract

The present invention relates to a location verification system for each section of the function to check change in image information according to the data collection location of MMS. The system includes: an MMS image storage DB (10); a section setting module (70); an editing module (80); an update module (90); deep learning information DB (20); a human body image extraction deep learning module (30); a mask image analysis module (40); a human face image analysis module (50); and a blurring processing module (60). When some changes are confirmed in the collected MMS images, only the corresponding section can be updated by verifying the location of the section, and the MMS video can be updated efficiently.

Description

Position Verification System For Confirming Change In MMS Image for each section of image information change confirmation function according to the data collection location of MMS

The present invention relates to a location verification system for each section of the function to check image information change according to the data collection location of the MMS.

The data collection device of the MMS (Mobile Mapping System) is installed in the vehicle and constitutes the lidar, camera, and GPS device to obtain geometrical structure information of the surrounding space centered on the current location, collected video image information, and location information of the current location. to collect

The geometrical structure information collected in this way is used as drawing data for the 3D numerical map image, and the collected image image information is used as an image providing service (hereinafter, 'street view service') of a street scene provided by an Internet portal site.

However, the collected image image of the street scene photographed through the data collection device includes the face of the pedestrian. That is, the personal information of pedestrians is exposed as it is at the time of information collection through MMS.

Therefore, before providing the street view service in the collected video image, it must be blurred so that a third party cannot identify it.

Conventionally, an image analysis method of finding each human face in an MMS image is used to blur the human face.

The image analysis method is to find a human face by analyzing pixel values (RGB values) in the entire MMS image, and since it is necessary to find a human face by analyzing the pixel values of the entire MMS image, the processing time is long even if the accuracy is improved. There was a problem with taking

Meanwhile, due to the recent virus epidemic, most people wear a mask. In this way, the person wearing the mask does not need to do blurring because the face is exposed inaccurately.

Also, conventionally, when a change occurs in facilities (buildings, etc.) around the road, MMS data (images) are newly collected for the entire section.

However, in this case, it takes a long time to work, and there is a problem of re-storing unnecessary data.

Republic of Korea Patent Registration No. 10-1979913

The present invention is to solve the above problems, and when a partial change is confirmed in the collected MMS image, the section of the function to verify the change of image information according to the data collection location of the MMS is updated by verifying the location of the section and updating only the section The task to be solved is to provide a 'star location verification system'.

In addition, in the present invention, after rapidly extracting a human body image from an MMS image through a deep learning method, and accurately finding a face image from the human body image through image analysis of the extracted human body image, the face image is added to the face image. It is a task to solve the problem to provide a 'location verification system for each section of the function to check image information change according to the data collection location of MMS', which enables faster and more accurate human face blurring processing by performing blurring processing on the face.

In addition, the present invention excludes a human body image with a mask image among human body images extracted through deep learning, and then searches for a human face only for a human body image without a mask and performs blurring processing within the MMS image. It is a task to solve the problem to provide a 'position verification system for each section of the function to check image information change according to the data collection location of MMS', which improves the processing speed of human face blur in .

The present invention for solving the above problems,

an MMS image storage DB 10 for storing MMS images;

a section setting module 70 for setting a certain section in the MMS video;

an editing module 80 for cutting the set predetermined section in the MMS video;

an update module 90 for confirming the location of a certain section of the MMS video cut by the editing module 80, and inserting a new video into the predetermined section;

It has a basic information DB 21 that stores the first basic information for deep learning and a learning information DB 22 that stores learning information for deep learning, and stores deep learning information consisting of basic information and learning information. Deep learning information DB (20) and;

a human body image extraction deep learning module 30 for extracting a human body image from an MMS image by deep learning based on the deep learning information stored in the deep learning information DB 20;

Human body image extraction The human body image extracted by the deep learning module 30 is subjected to pixel RGB analysis and pattern analysis to determine whether a mask image exists on the human body image, and if there is a mask image on the human body image a mask image analysis module 40 for classifying the person's body image into a blurring non-processing image;

a human face image analysis module 50 for identifying a human face by performing pixel RGB analysis and pattern analysis on a human body image that is not classified as a blurring unprocessed image by the mask image analysis module 40;

A blurring processing module (60) for blurring the human face identified by the human face image analysis module (50)

It is characterized in that it includes.

According to the present invention according to the means for solving the above problems, when some changes are confirmed in the collected MMS image, the position of the corresponding section can be verified and only the section can be updated, so that the MMS video can be updated efficiently. .

In addition, in the present invention, after rapidly extracting a human body image from an MMS image through a deep learning method, and accurately finding a face image from the human body image through image analysis of the extracted human body image, the face image is added to the face image. There is an effect that faster and more accurate human face blurring processing is possible by performing the blurring process.

In addition, the present invention excludes a human body image with a mask image among human body images extracted through deep learning, and then searches for a human face only for a human body image without a mask and performs blurring processing within the MMS image. It has the effect of improving the human face blurring processing speed in

1 is a system diagram for explaining the configuration of the present invention,
2 is a flowchart for explaining an image processing process according to the present invention;
3 is a view for explaining the human body image stored in the basic information DB,
4 is a view for explaining the human body image stored in the basic information DB and the learning information DB,
5 is a diagram for explaining an MMS image before blurring;
6 is a diagram for explaining an MMS image after blurring.

Hereinafter, the present invention will be described so that those skilled in the art can easily practice it.

1 is a system diagram for explaining the configuration of the present invention. The configuration according to the present invention will be described with reference to FIG. 1 as follows.

'Position verification system for each section of the function to check image information change according to the data collection location of MMS' according to the present invention includes the MMS image storage DB 10, the section setting module 70, the editing module 80, and the update Module 90, deep learning information DB 20, human body image extraction deep learning module 30, mask image analysis module 40, human face image analysis module 50, and blurring processing module (60).

The MMS storage DB 10 stores the MMS image collected by the data collection device (DC). In this embodiment, the data collection device DC is mounted on the vehicle V, and collects MMS images using the camera DC1. The data collection device DC used in this embodiment is a configuration provided in a normal MMS (Mobile Mapping System), and in addition to the camera DC1, a GPS device, a lidar device, a positioning sensor, etc. includes configuration.

The section setting module 70 sets a certain section in the MMS video.

The editing module 80 cuts the set predetermined section from the MMS video. That is, when a certain section is set by the section setting module 70 , the editing module 80 cuts the set section.

The update module 90 checks the location of a predetermined section of the MMS video cut by the editing module 80 and inserts a new video into the predetermined section. Here, the updated MMS image is stored in the MMS storage DB (10).

The deep learning information DB 20 stores the deep learning information consisting of basic information and learning information.

The basic information DB 21 stores initial basic information for deep learning (machine learning). In this embodiment, the basic information DB 21 stores various types of human body images as initial basic information.

The learning information DB 22 stores learning information for deep learning. In this embodiment, the learning information DB 22 stores various types of human body images learned by the human body image extraction deep learning module 30 as learning information.

The human body image extraction deep learning module 30 extracts a human body image from the MMS image by searching for a constituent object on the MMS image, and deep learning based on the deep learning information stored in the deep learning information DB 20 .

As an example, the human body image extraction deep learning module 30 is configured to extract an initial human body image stored in the basic information DB 21 and a predetermined area similar to the first human body image (eg, similarity of 70 to 80%). Search for human body images on MMS images with

Here, the operator checks the human body image searched for in the human body image extraction deep learning module 30, so that the searched human body image is similar to the first human body image stored in the basic information DB 21 below the standard. In this case (if the similarity is less than or equal to the range), it is designated that the searched human body image is not stored in the learning information DB 22 by the human body image extraction deep learning module 30 .

Then, the human body image extraction deep learning module 30 includes a searched human body image 100% identical to the first human body image stored in the basic information DB 21 among the searched human body images, and the learning information DB 22 . After excluding the searched human body image designated not to be stored in the , the remaining searched human body images are stored in the learning information DB 22 .

On the other hand, when deep learning is performed based on the first human body image stored in the basic information DB 21 , thereafter, deep learning is performed based on the human body image stored in the deep learning information DB 20 .

In this embodiment, deep learning is performed by the deep learning method as described above, but a human body image may be searched through various existing deep learning methods.

The mask image analysis module 40 analyzes the human body image extracted by the human body image extraction deep learning module 30 to determine whether a mask image exists on the human body image, and a mask on the human body image If an image exists, the person's body image is classified as a non-blurring image.

For example, the mask image analysis module 40, through pixel RGB analysis and pattern analysis, on the human body image extracted from the human body image extraction deep learning module 30, a mask shape (eg, white or black) It is determined whether a color rectangle, oval, rhombus, etc.) exists, and if a mask shape exists on the human body image, the corresponding human body image is classified as a non-blurring image. Through this, in the present embodiment, the blurring process is not performed on the face of the person wearing the mask, so that the work speed can be improved.

The human face image analysis module 50 analyzes a human body image that is not classified as a blurring non-processed image by the mask image analysis module 40 to identify a human face. For example, the human face image analysis module 50 detects a human face by finding parts where eyes, nose, and mouth exist in a human body image through pixel RGB analysis and pattern analysis.

The blurring processing module 60 blurs (or blurs) the human face identified by the human face image analysis module 50 in real time.

Meanwhile, the present invention may further include an input unit IP for inputting a data signal and a display unit DS for outputting a data signal.

2 is a flowchart for explaining an image processing process according to the present invention, FIG. 3 is a diagram for explaining a human body image stored in the basic information DB, and FIG. 4 is a human body image stored in the basic information DB and the learning information DB. , FIG. 5 is a view for explaining an MMS image before blurring, and FIG. 6 is a view for explaining an MMS image after blurring, as viewed with reference to FIGS. 2 to 6 . The operation of the invention is described as follows.

First, the MMS image collected by the data collection device DC is transmitted to the MMS image storage DB 10 through communication (wired or wireless communication) between the present invention and the data collection device DC.

On the other hand, when the MMS image is received from the data collection device (DC) and a new object (new building, new terrain, etc.) is identified in a certain section of the MMS image, the section setting module 70 'exists of a new object in the new MMS image' Based on the 'section', the location of the 'existing section of the existing MMS video (original MMS video)' corresponding to the new object existence section is verified and confirmed, and an update section is set in the existing MMS video.

That is, when the operator drives the section setting module 70 , the section setting module 70 sets the 'updating section of the existing MMS image' corresponding to the 'new object existence section of the new MMS image'.

When the update section is set in this way, the operator drives the editing module 80 to cut the update section from the existing MMS image.

Thereafter, the operator drives the update module 90 , checks the position of the update section of the existing MMS video cut by the editing module 80 , and inserts the new video into the predetermined section.

Then, the new object is updated in the existing MMS image, so that only the necessary update period can be efficiently updated without replacing the entire existing MMS image.

Meanwhile, an example of the human face blurring process according to the present invention will be described as follows.

First, the basic information DB 21 stores the first basic information for deep learning.

As the initial basic information, various human body images are stored as shown in FIG. 3 .

In this embodiment, 100 human body images are stored in the basic information DB 21 .

In this state, the human body image extraction deep learning module 30 searches for a constituent object on the MMS image (existing MMS image or updated MMS image), and the first human body image stored in the basic information DB 21 and the first person A human body image on the MMS image having a certain area similar to the body image (eg, similarity of 70-80%) is searched for.

At this time, the human body image extraction deep learning module 30 searches, for example, 1000 human body images.

In the 1000 human body images, for example, there are 200 identical human body images 100% identical to the original human body image, 650 suitable human body images having a similarity of 70 to 80%, and a similarity of less than 70 to 80%. There may be 150 non-conforming human body images with

Here, since the same human body image is the same as the initial human body image, the human body image extraction deep learning module 30 does not need to learn and store the same human body image in the learning information DB 22 .

In addition, since the non-conforming human body image is not similar to the actual human body, the human body image extraction deep learning module 30 does not need to learn and store the non-conforming human body image in the learning information DB 22 .

Therefore, in this embodiment, the operator checks the human body image searched by the human body image extraction deep learning module 30, and then the same human body image and the non-conforming human body image are combined with the human body image extraction deep learning module ( 30) to designate not to be stored in the learning information DB 22.

Then, the human body image extraction deep learning module 30 removes the 200 identical human body images and 150 non-conforming human body images from the searched human body images, and then extracts 650 suitable human body images as learning information. It is stored in DB 22. Accordingly, 750 human body images are stored in the deep learning information DB 20 as shown in FIG. 4 .

On the other hand, when deep learning is performed based on the first human body image stored in the basic information DB 21 , thereafter, deep learning is performed based on the human body image stored in the deep learning information DB 20 .

Through the deep learning process as described above, the human body image extraction deep learning module 30 extracts a human body image that matches the similarity (70-80% similarity).

Then, the mask image analysis module 40, through RGB pixel analysis and pattern analysis, on the human body image extracted from the human body image extraction deep learning module 30, the mask shape (eg, white or black) formed rectangle, oval, rhombus, etc.) is checked, and if a mask shape exists on the human body image, the corresponding human body image is classified as a non-blurring image.

Then, the human face image analysis module 50 identifies the human face by finding parts where the eyes, nose, and mouth exist in the human body image that is not classified as a blurring unprocessed image through pixel RGB analysis and pattern analysis. .

As described above, when a human face is recognized from the human body image by the human face image analysis module 50, the blurring processing module 60 blurs the recognized human face in real time.

Through the above method, a human face may be searched for in the MMS image shown in FIG. 5 and subjected to blurring, but after excluding the face of a person wearing a mask, blurring may be performed as shown in FIG. 6 .

In this embodiment, after excluding the body image of a person wearing a mask from the human body image extracted by the human body image extraction deep learning module 30, the human face is identified only for the body image of a person who is not wearing a mask, and the corresponding person Blur the face. However, in the present invention, without considering wearing a mask, after identifying a human face from the human body image extracted by the human body image extraction deep learning module 30, the corresponding human face may be blurred.

On the other hand, in this embodiment, when the existing (original) MMS image is blurred, the entire updated MMS image is blurred, or the existing MMS image that has been blurred is updated, only the update section can be blurred. .

In the present invention, it is preferable to update the blurring-processed existing MMS image and to perform the blurring process only on the update section within the updated existing MMS image (updated MMS image).

In addition, in this embodiment, a temperature sensor (not shown) may be mounted on a case in which each module of the present invention is mounted. In this case, the temperature sensor outputs a warning signal when the internal temperature of the case rises above a certain level, so that an administrator can quickly check the inside of the case.

According to the present invention as described above, when a change in image information (new object discovery) is confirmed at the MMS image (MMS data) collection location, only the changed position can be updated while using the existing MMS image as it is.

On the other hand, when searching for human face images one by one using the image extraction method through pixel RGB analysis, it is necessary to analyze the RGB values of many pixels in the MMS image and analyze each pattern one by one. It takes a very long time to do.

Therefore, the present invention extracts a human body image having a large size within the MMS image through a deep learning method, and then identifies a human face from the extracted human body image through pixel RGB analysis, so that the human face is quickly and accurately displayed in the MMS image. can figure out

At this time, the image extraction of the deep learning method is based on the deep learning information stored in the deep learning information DB 20 to search for a human body image with a certain degree of similarity in the MMS image, so the image extraction method through RGB pixel analysis. Compared to that, the processing speed of searching for human body images is faster.

In addition, the present invention extracts a human body image from an MMS image, recognizes a human face from a human body image, and blurs the person's face, so that the face of a person wearing a mask may not be blurred. there is. Since the face of a person wearing a mask is not easily identified by itself, there is no need to perform a blurring process. It can improve the working speed.

On the other hand, the present invention verifies the position of the new MMS video section based on the section position of the existing MMS video in order to update the MMS video when the video information change (new object discovery) is confirmed at the MMS video (MMS data) collection location. In the sense of including a configuration for checking and further including various configurations for MMS image processing, the title and claimed subject of the present invention are defined as 'Position verification system for each section of image information change confirmation function according to the data collection location of MMS' ' is defined as

10; MMS image storage DB 20; Deep Learning Information DB
21; Basic information DB 22; Learning information DB
30; human body image extraction deep learning module 40; Mask image analysis module
50; human face image analysis module 60; Blur processing module
70; section setting module 80; editing module
90; update module

Claims (1)

an MMS image storage DB 10 for storing MMS images;
a section setting module 70 for setting a certain section in the MMS video;
an editing module 80 for cutting the set predetermined section in the MMS video;
an update module 90 for confirming the location of a certain section of the MMS video cut by the editing module 80, and inserting a new video into the predetermined section;
It has a basic information DB 21 that stores the first basic information for deep learning and a learning information DB 22 that stores learning information for deep learning, and stores deep learning information consisting of basic information and learning information. Deep learning information DB (20) and;
a human body image extraction deep learning module 30 for deep learning based on the deep learning information stored in the deep learning information DB 20 to extract a human body image from an MMS image;
Human body image extraction The human body image extracted by the deep learning module 30 is pixelated Mask image analysis module 40 that performs RGB analysis and pattern analysis to determine whether a mask image exists on a human body image, and classifies the human body image as a blurring non-processing image if a mask image exists on the human body image class;
a human face image analysis module 50 for recognizing a human face by performing pixel RGB analysis and pattern analysis on a human body image that is not classified as a blurring unprocessed image by the mask image analysis module 40;
A blurring processing module (60) for blurring the human face identified by the human face image analysis module (50)
Position verification system for each section of the image information change confirmation function according to the data collection position of the MMS, characterized in that it comprises a.

Images