KR20200003971A - Image Processing Apparatus and Image Verification Apparatus for Protecting Privacy and Preventing Alteration, and Method thereof - Google Patents

Abstract

According to the present invention, provided are an image processing device, an image verification device, and a method thereof which extract first hash data from a first image which pixelates the entire area for each of the frames constituting an original image, generate indexes indicating positions of an important block and a second image which is pixelated for only the important block including an object, which is a privacy protection target, generate a third image which is pixelated with respect to a non-important block in a second image, provide a privacy-protected second image by comparing second hash data extracted from the third image with the first hash data, and verify integrity of the second image. The present invention can efficiently protect privacy in the image and verify whether the image is modulated.

Description

Image Processing Apparatus and Image Verification Apparatus for Protecting Privacy and Preventing Alteration, and Method

The present disclosure relates to an apparatus and a method for processing an image to protect privacy included in the image and to verify whether the image is modulated.

Recently, with the development of video technology and network technology, various low cost Internet of Things (IoT) cameras supported by a network have been widely spread. Various images captured by these cameras are shared through a network.

These shared videos can be used as video evidence of various events and can benefit the public. However, there is a risk that privacy information, such as an individual's face, license plate or address sign in a video, may be exposed to the public, and thus a protection method is required.

In addition, a wide variety of image processing applications are being deployed, including copy-move, splicing, retouching, erasing, changing lighting conditions, frame insertion, As modulation on an image becomes easy, such as frame deletion and frame switch, there is a need for a method of guaranteeing time information and integrity of a shared image.

According to such a necessity, a method of protecting a privacy through a process of identifying an area corresponding to privacy when capturing an image and blurring the identified area, etc. is provided. There is a problem in that the image quality is degraded due to the limited processing capacity. In addition, such blurring may have a problem that invalidates the authenticated hash through time stamping.

Therefore, there is an increasing need for research on a method for efficiently protecting privacy in an image and verifying the integrity of the image.

In the above-described background, when the original image is captured, the present disclosure extracts hash data from a pixelated image of the entire area of each frame, and detects an object subject to privacy protection when the image is requested. The present invention proposes an image processing apparatus and method capable of efficiently generating a privacy protected image regardless of the performance of the apparatus.

In addition, the present disclosure verifies the integrity of the privacy protection image by extracting hash data from a pixelated image of a region in which the object to be protected for privacy is not included in the privacy protection image to determine the integrity of the privacy protection image. An apparatus and method for verifying an image can be provided.

Disclosure of the Invention The present disclosure devised to solve the above-described problem stores the original image photographed by the memory, the image capturing unit, and the image capturing unit in a memory, and stores each of the frames included in the original image captured by a predetermined number of blocks. An image processor configured to generate a first image by performing pixelation on all of a predetermined number of blocks, and extract and store first hash data from the first image. Objects to be protected for privacy are detected from the frames included in the original image, each of the frames included in the stored original image is divided into a predetermined number of blocks, and a critical block including the object among the predetermined number of blocks. a pixel to generate a second image by performing pixelation on a block, and to store an index indicating a location of a critical block and a second image in a memory. It provides an image processing apparatus comprising image generation unit protection.

Also, the present disclosure receives first hash data, a second image, and an index from an image processing apparatus through a communication unit and a communication unit, and distinguishes each of the frames included in the second image from the first image from which the first hash data is extracted. The same classification as one predetermined number of blocks is performed, and pixelization is performed on the remaining blocks except for the pixelated critical block by including an object to be protected for privacy among the frames of the second image by using an index. A control apparatus for generating an image and comparing the second hash data extracted from the third image with the first hash data to determine an integrity of the second image is provided.

The present disclosure also provides a method of storing an original image photographed by an image capturing unit, dividing each of the frames included in the original image photographed into a predetermined number of blocks, and performing pixelization on all of the predetermined number of blocks. Generating a first image, extracting and storing first hash data from the first image, detecting an object to be protected in privacy from frames included in the stored original image, and including the stored original image Dividing each of the frames into a predetermined number of blocks, and performing pixelation on a critical block including an object among the predetermined number of blocks to generate a second image, and an index indicating a location of the critical block and the first image. 2 provides an image processing and verification method comprising storing an image.

According to the present disclosure, when capturing an original image, hash data is extracted from an image obtained by pixelating an entire area of each frame, and a pixelated privacy protection image is generated by detecting an object subject to privacy protection when a request for the image is made. As a result, the privacy protection image can be efficiently generated regardless of the performance of the device.

In addition, according to the present disclosure, by extracting the hash data from the pixelized image of the area that does not include the object of privacy protection in the privacy protection image to determine the integrity of the privacy protection image, the integrity of the privacy protection image with high reliability There is an effect that can be verified.

1 is a block diagram of an image processing and verification system according to an embodiment of the present disclosure.
2 is a block diagram of an image processing apparatus according to an exemplary embodiment.
3 is a block diagram of an image processor included in an image processing apparatus according to an exemplary embodiment.
4 to 6 are diagrams for describing an operation of an image processor according to an exemplary embodiment.
7 is a block diagram of a privacy protection image generator included in an image processing apparatus according to an exemplary embodiment.
8 and 9 are diagrams for describing an operation of a privacy protection image generator according to an embodiment of the present disclosure.
10 is a block diagram of an image verification apparatus according to an exemplary embodiment.
11 is a diagram for describing an operation of an image verification apparatus according to an exemplary embodiment.
12 is a flowchart illustrating an image processing and verification method according to an embodiment of the present disclosure.
13 is a flowchart illustrating a start time of generation of a second image according to an embodiment of the present disclosure.
14 is a flowchart illustrating an image processing and verification method according to an embodiment of the present disclosure.
15 is a flowchart illustrating an operation of an image processing and verification system according to an embodiment of the present disclosure.

Some embodiments of the present disclosure are described in detail below with reference to exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In addition, in describing the present disclosure, when it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present disclosure, the detailed description thereof will be omitted.

In addition, in describing the components of the present disclosure, terms such as first, second, A, B, (a), and (b) may be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. If a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected to or connected to that other component, but there may be another configuration between each component. It is to be understood that the elements may be "connected", "coupled" or "connected".

In the present disclosure, an "original image" is an image photographed by an image processing apparatus, and means an image before processing for privacy protection such as blurring. In addition, the term "block" refers to an area, such as a grid, that divides the entire area of the frame constituting the image. In addition, "pixelate" refers to a process that makes it difficult to identify an image in a block, such as a blurring process or a mosaic process, for each block. In addition, the term "critical block" refers to a block containing an object whose privacy may be problematic. In addition, "index" refers to information indicating the position of the critical block among the blocks for dividing the frame.

Hereinafter, an image processing apparatus, an image verification apparatus, and a method thereof according to embodiments of the present disclosure will be described with reference to related drawings.

1 is a block diagram of an image processing and verification system according to an embodiment of the present disclosure.

Referring to FIG. 1, the image processing and verification system 10 includes an image processing apparatus 100, an image verification apparatus 200, and a time stamp server 300.

The image processing apparatus 100 may include a device for capturing an image and transmitting the captured image when requested. If the image processing apparatus 100 may capture an image and may perform the image processing method according to the present disclosure, the image processing apparatus 100 is not limited to a specific type. For example, the image processing apparatus 100 may include electronic devices in various fields such as a security camera, a black box camera installed in a vehicle, or a camera provided in a drone.

In particular, according to the present disclosure, even in a low performance device, an excellent privacy protection effect and reliability of an image can be ensured, and an image having a good image quality can be generated. Therefore, the image processing apparatus 100 according to the exemplary embodiment of the present disclosure may be implemented as a low-cost camera including a low specification processor (700 MHz to 1.2 GHz CPU) and a built-in wireless network interface for IoT connection. . However, this is only an example, and the present invention is not limited thereto, and it will be obvious that the present invention can be implemented in various performance devices.

The image processing apparatus 100 may detect an object whose privacy may be problematic in the captured image. For example, an object whose privacy may be a problem may mean an object including personal information such as a face of a person, a license plate of a car, or a home address sign. The image processing apparatus 100 may generate a privacy protection image by processing an object that may have a privacy problem so that it may not be recognized.

In relation to the generation of a privacy-protected image, a conventional blurring procedure acquires a real time frame from a camera module, blurs a detected area, and records a frame in which a face / number plate or the like is blurred into a video file. do. Among the I / O time for acquiring and recording the real-time frame, the detection time for the face detection, and the pixelate time for the blur processing, the most time is generated during the generation of the privacy protection image. The most consumed operation is detection time. Table 1 below is an example of time spent when real-time blur processing of an image using a Raspberry Pi single board computer. Referring to Table 1, it is shown that the most time is spent when detecting the face / number plate.

Processing time taken per frame Object blurring Resulting frame rate Detect Pixelate I / O Human face 2.3 fps 431.5 ms (89.7%) 0.05 ms (0.01%) 47.4 ms (10.2%) Car plate 5.1 fps 146.2 ms (75.5%) 0.05 ms (0.02%) 47.1 ms (24.4%)

By using this characteristic, the image processing apparatus 100 according to an exemplary embodiment of the present disclosure may preferentially store the original image without performing the detection of the most time-consuming object in real time together with the image capturing. In addition, as a measure for verifying the integrity of the image, the image processing apparatus 100 may pixelate each frame of the original image in real time and extract hash data. In addition, to verify the recording time of the image, the image processing apparatus 100 may receive the time stamped hash data by transmitting the hash data to a time stamp server 300 on a network. have. Ensuring the reliability of data through the time stamp server follows a known method, and a detailed description thereof will be omitted.

Thereafter, when an image transmission request is received from the image verification apparatus 200, the image processing apparatus 100 may detect an object with respect to the stored original image. Object detection at this stage does not need to proceed in real time with the image capture, and thus can be efficiently performed even with a lower specification CPU. The image processing apparatus 100 may generate a privacy protection image by performing pixelization processing on the detected object, and store index information indicating the pixelated region. The image processing apparatus 100 may transmit the privacy protection image and the index information to the image verification apparatus 200.

The image verifying apparatus 200 may be a device such as a mobile terminal or a computer used by a person or an institution requesting an image captured by the image processing apparatus 100. The image verification apparatus 200 may request an image transmission to the image processing apparatus 100, and if the image verification method according to the present disclosure can be performed, the image verification apparatus 200 is not limited to a specific type.

The image verification apparatus 200 may receive the time stamped hash data, the privacy protection image, and the index information from the image processing apparatus 100. The image verification apparatus 200 may perform pixelation processing of the non-pixelation process on the privacy-protected image by using the index information. Therefore, an image in which the entire area of each frame is pixelated may be generated. The image verification apparatus 200 may extract hash data from the generated image. The image verification apparatus 200 may verify the integrity of the privacy protection image by comparing the extracted hash data with the time stamped hash data received from the image processing apparatus 100.

If the integrity is verified, the image verification apparatus 200 may use the privacy protected image such as publicly presenting or submitting as evidence. In this case, since the original image that is not pixelated is stored only in the image processing apparatus 100, the image verification apparatus 200 may not access the original image. Therefore, problematic privacy included in the original image captured may be protected.

According to this, the protection and integrity of the privacy of the captured image can be efficiently verified. Hereinafter, an image processing and verification method according to embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings.

2 is a block diagram of an image processing apparatus according to an exemplary embodiment. 3 is a block diagram of an image processor included in an image processing apparatus according to an exemplary embodiment. 4 to 6 are diagrams for describing an operation of an image processor according to an exemplary embodiment. 7 is a block diagram of a privacy protection image generator included in an image processing apparatus according to an exemplary embodiment. 8 and 9 are diagrams for describing an operation of a privacy protection image generator according to an embodiment of the present disclosure.

Referring to FIG. 2, the image processing apparatus 100 according to the exemplary embodiments of the present disclosure stores an image captured by the image capturing unit 110 and an image captured by the image capturing unit in a memory, and includes a frame included in the captured original image. Each of the blocks is divided into a predetermined number of blocks, and pixelation of all of the predetermined number of blocks is performed to generate a first image, and first hash data is generated from the first image. ), The image processor 120, the memory 130, and the frames included in the stored original image, respectively, to detect the object to be protected in privacy, and each of the frames included in the stored original image is a predetermined number of blocks. And generate a second image by pixelating a critical block including an object among a predetermined number of blocks, and write down an index and a second image indicating the position of the critical block. The privacy protection image generating unit 140 and the communication unit 150 are stored in the memory.

The image capturing unit 110 may include an image sensor to capture an image in frame units. The image capturing unit 110 may transmit the captured frames to the image processing unit 120. The image capturing unit 110 may be capable of capturing an image, and is not limited to a specific device.

Referring to FIG. 3, the image processor 120 includes an encoder 121, an original image storage unit 122, a decoder 123, a first image generator 124, a hash data extractor 125, and a hash data storage. It may include a portion 126.

The encoder 121 included in the image processor 120 may generate an original image by encoding a frame photographed by the image capturing unit 100. For example, the encoder 121 may encode the frame into a JPEG image. Accordingly, the original image may be stored in MJPEG (Motion JPEG) format. However, this is merely an example and the present invention is not limited thereto. In the encoder 121, encoding may be performed in other ways as necessary.

The image processor 120 may store the generated original image in the memory 130 through the original image storage 122 in real time. Here, storing in real time may mean storing the original image in the memory 130 while the image is captured by the image capturing unit 100. That is, the image processor 120 may generate original images by encoding frames sequentially photographed and store the generated original images.

The image processor 120 may decode the generated original image into frames again in the decoder 123 while storing the original image generated by the encoder 121. That is, the storage path of the original image and the decoding path of the original image may be implemented as separate paths.

The encoding of the original image is performed in common first, and then the storage of the original image and the first hash data are extracted in order to prevent an error caused by lossy compression scheme encoding during the integrity verification of the privacy protected image. That is, by storing the original image in a lossy compression method and extracting the first hash data based on the encoded state, an error caused by the lossy compression method may be prevented in the later integrity verification.

For example, it is assumed that the first hash data is extracted directly from the frames photographed by the image capturing unit 110. In this case, the privacy protection image is generated based on the original image in which the loss due to the lossy compression is generated, and the second hash data thereof is extracted, whereas the first hash data is extracted based on the frames without such loss. Subsequently, when comparing the second hash data and the first hash data to verify the integrity of the privacy protected image, an error due to lossy compression may occur. Therefore, in order to prevent such an error, the storage path of the original image and the start point of the decoding path are set to the encoded original image.

The first image generator 124 included in the image processor 120 may divide each of the frames included in the captured original image into a predetermined number of blocks. The image processor 120 may generate a first image by performing pixelation on all of a predetermined number of blocks included in each of the frames constituting the original image. That is, the first image may be an image in which the entire area of each of the frames constituting the original image is pixelated.

Referring to FIG. 4, one frame i captured by the image capturing unit 110 is illustrated. The frame i contains two people 1 and 2. Referring to FIG. 5, according to an example, a frame i is divided into 16 equally rectangular blocks b, which are 4 * 4 matrices. Herein, frames are divided into 16 blocks for convenience of description and the present invention is not limited thereto. In addition, each block is configured as a rectangle, but is not limited thereto, and may be set in various shapes as necessary.

The first image generator 124 may perform pixelization on each block. The intensity of pixelation for each block can be set equally. Referring to FIG. 6, the pixelated processing (bp) of all blocks included in the frame is indicated by diagonal lines. Accordingly, people included in the frame also appear to be pixelated (1p, 2p). As such, the frames i1 in which the entire area is pixelated may be generated as the first image.

According to an embodiment, the pixelation intensity for pixelating each block may be determined in consideration of an object's identification possibility and an image's modulation detection possibility. The pixelated intensity indicates a degree of lowering the original resolution. For example, the pixelated intensity X may mean reducing the resolution at a ratio of X ² by averaging pixel values of the image.

The higher the pixelation intensity, the lower the object's recognizability. Also, if the pixelation intensity is too high, the possibility of modulation detection of the image may be lowered. Accordingly, an appropriate pixelation intensity may be selected to increase the detection probability of the modulation of the image while lowering the identification probability of the object. According to one example, it may be desirable for the pixelation intensity X to be set between 11 and 15. However, this is only an example, and the present disclosure is not limited thereto and may be set differently depending on the quality of the image.

The hash data extractor 125 included in the image processor 120 may extract and store first hash data from the first image. According to an embodiment, the image processor 120 may extract frame hash data from each of the frames included in the first image, and re-extract the first hash data from the extracted frame hash data. According to an embodiment, the image processor 120 may simultaneously perform the operation of generating the first image and the extraction of the first hash data in parallel with the operation of storing the original image.

According to an embodiment, the hash data extractor 125 may delete the first image after extracting the first hash data. According to the present disclosure, since only the first hash data is used when verifying the integrity of the privacy protection image, the first image is deleted because it is no longer needed, thereby preventing unnecessary consumption of the memory.

The image processor 120 may transmit the first hash data to the time stamp server 300 through the communication unit 150. The image processor 120 may receive the first hash data time stamped from the time stamp server 300 through the hash data storage unit 126 and store the first hash data in the memory 130. When the image transmission request is received from the image verification apparatus 200, the hash data storage unit 126 may transmit the time stamped first hash data to the image verification apparatus 200.

The memory 130 stores image data processed by the image processor 120 and the privacy protection image generator 140 and data related thereto. The memory 130 may store a program for performing an operation such as dividing the frame into blocks or pixelating the frame. According to an example, the memory 130 may be implemented as a card type memory (for example, SD or XD memory). For example, the low-end model image processing apparatus may include a 64-128 GB onboard SD memory card. However, this is only an example, and the present disclosure is not limited thereto, and the memory 130 may be implemented as various storage media as necessary.

Referring to FIG. 7, the privacy protection image generator 140 may include an object detector 141, a second image generator 142, and an index acquirer 143.

The object detector 141 included in the privacy protection image generator 140 may detect an object to be privacy protected from the frames included in the stored original image. When the privacy protection image generating unit 140 receives an image transmission request, the privacy protection image generating unit 140 may read an original image from the memory 130 and detect an object included in each frame. A well-known object detection algorithm may be applied to the detection of an object, and is not limited to a specific algorithm.

The privacy protection image generating unit 140 may detect an object such as a face of a person or a license plate of a car for each frame. Referring to FIG. 8, the faces of people 1 and 2 included in the frame i of the stored original image may be a privacy protection target. The object detector 141 may detect the faces of the people 1 and 2 and acquire the detected areas d1 and d2.

The second image generator 142 included in the privacy protection image generator 140 may divide each of the frames included in the stored original image into a predetermined number of blocks. The privacy protection image generating unit 140 may distinguish the frames included in the stored original image in the same format as the blocks in which the frames are divided in order to generate the first image in the image processing unit 120. That is, each frame of the first image and each frame of the second image corresponding to each frame of the first image may be divided into blocks having the same number and the same shape.

For example, as described above, it is assumed that each frame is divided into 16 identical rectangular blocks having a 4 * 4 matrix to generate the first image. In this case, the second image generator 142 may also classify the frames included in the stored original image into 16 identical rectangular blocks having a 4 * 4 matrix.

The second image generator 142 may generate a second image by performing pixelation on critical blocks including an object to be protected for privacy among a predetermined number of blocks that divide each frame. That is, the second image may be a privacy protection image in which only an important block of pixels constituting the original image is pixelated. The privacy protection image generator 140 may perform pixelization on the critical block with the same pixelation intensity as the pixelation intensity where each of the blocks is pixelated to generate the first image in the image processing unit 120. .

Referring to FIG. 9, the pixels processed for the critical blocks cb1 and cb2 including the areas where a human face is detected among the blocks included in the frame are indicated by diagonal lines. Accordingly, the faces of people included in the important blocks cb1 and cb2 are also shown to be pixelated. As such, the frames i2 in which the critical blocks cb1 and cb2 are pixelated in the entire area may be generated as the second image.

The index obtaining unit 143 included in the privacy protection image generating unit 140 may obtain an index indicating the position of the important block among a predetermined number of blocks for dividing each frame. The privacy protection image generator 140 may store the index and the second image in the memory 130. When the image protection request is received from the image verification apparatus 200, the privacy protection image generator 140 may transmit the index and the second image to the image verification apparatus 200.

According to an embodiment, the privacy protection image generator 140 may compress the second image by a lossless compression method and transmit the second image to the image verification apparatus 200. As described above, in order to prevent an error due to lossy compression when verifying the integrity of the second image, after encoding the original image first, the storage of the original image and the extraction of the first hash data are performed. Even in the transmission of two images, in order to prevent an error due to lossy compression, it may be transmitted in a lossless compression method.

The communicator 150 may include one or more modules that enable wireless communication between the image processing apparatus 100 and the image verification apparatus 200 or between the image processing apparatus 100 and the time stamp server 300. According to an example, the communication unit 150 may transmit and receive hash data using the time stamp server 300 and the IoT communication channel. The communication unit 150 may include a mobile communication module, a wireless Internet module, or a short-range communication module for exchanging data with the image verification apparatus 200.

According to this, when capturing an original image, hash data is extracted from an image obtained by pixelating the entire area of each frame, and when a request for the image is made, an object to be protected is generated to generate a pixelated privacy protection image. Regardless of the performance of the device, it is possible to efficiently generate a privacy-protected image.

10 is a block diagram of an image verification apparatus according to an exemplary embodiment. 11 is a diagram for describing an operation of an image verification apparatus according to an exemplary embodiment.

Referring to FIG. 10, the image verifying apparatus 200 according to an exemplary embodiment receives the first hash data, the second image, and the index from the image processing apparatus 100 described above through the communication unit 210 and the communication unit. Using the index to generate a third image by performing pixelation on the remaining blocks except for the pixelated important block including the object to be protected for privacy among the frames of the second image, and extracting from the third image. The controller 220 may include a controller 220 that compares the hash data with the first hash data to determine the integrity of the second image.

The communication unit 210 may include one or more modules that enable wireless communication between the image processing apparatus 100 and the image verification apparatus 200. According to an example, the communication unit 210 may include a mobile communication module, a wireless Internet module, or a short-range communication module for exchanging data with the image processing apparatus 100.

The controller 220 may request to transmit an image to the image processing apparatus 100 through the communication unit 210. The controller 220 may receive the first hash data, the second image, and the index, which are time stamped from the image processing apparatus 100 according to the request. As described above, the time-stamped first hash data is extracted in a state in which the entire block that distinguishes each frame of the original image is subjected to pixelation processing. In addition, the second image is a privacy protection image obtained by pixelating an important block indicated by the index.

The controller 220 may classify each of the frames included in the second image in the same manner as a predetermined number of blocks that separate the first image from which the first hash data is extracted. That is, the frames included in the first image and the second image may be equally divided into blocks having the same number and the same shape, respectively.

The controller 220 may perform pixelation on the remaining blocks except for the pixelated critical block by including an object to be protected for privacy among the frames of the second image by using the index. Referring to FIG. 9, the important blocks cb1 and cb2 included in the frames of the second image are pixelated. Similarly, referring to FIG. 11, the important blocks cb1 and cb2 are pixelated.

The controller 220 may perform pixelation processing on the remaining blocks bp other than the critical blocks cb1 and cb2 using indexes indicating the critical blocks cb1 and cb2. In order to distinguish it from the important blocks cb1 and cb2, in FIG. 11, the remaining blocks bp are pixelated.

However, this is illustrated for convenience of description, and the pixelation method and intensity applied to the critical blocks cb1 and cb2 and the remaining blocks bp may be set to be the same and pixelation may be performed. Accordingly, the people 1p and 2p included in the image are also pixelated as a whole. As such, the frames i3 in which the entire area is pixelated may be generated as the third image. Accordingly, in the third image, the entire area of each frame of the original image is pixelated, similarly to the first image from which the first hash data is extracted.

According to an embodiment, the controller 220 may extract frame hash data from each frame included in the third image, and re-extract second hash data from the extracted frame hash data. The controller 220 may determine the integrity of the second image by comparing the second hash data extracted from the third image with the first hash data.

As described above, the third image, like the first image, is subjected to the same pixelation process for all blocks of the frames constituting the original image. Therefore, when the first hash data extracted from the first image and the second hash data extracted from the third image processed with the second image have the same value, the second image may be verified as an image without modulation. It can also be verified for the recording time by the time stamp of the first hash data.

According to this, the hash data is extracted from the pixelated image of the area where the object to be protected in the privacy protected image is not included in the privacy protected image to determine the integrity of the privacy protected image, thereby reliably verifying the integrity of the privacy protected image. Can be.

12 is a flowchart illustrating an image processing and verification method according to an embodiment of the present disclosure. 13 is a flowchart illustrating a start time of generation of a second image according to an embodiment of the present disclosure. 14 is a flowchart illustrating an image processing and verification method according to an embodiment of the present disclosure.

The image processing and verification method according to the present disclosure may be implemented in the image processing apparatus 100 and the image verification apparatus 200 described with reference to FIG. 1. Hereinafter, a method of processing and verifying an image according to the present disclosure, and operations of the image processing apparatus 100 and the image verification apparatus 200 for implementing the same will be described in detail with reference to the accompanying drawings.

Referring to FIG. 12, the image processor of the image processing apparatus may store an original image photographed by the image capturing unit [S110].

The image processor may generate an original image by encoding the frame photographed by the image capturing unit. The image processor may store the generated original image in memory in real time. That is, the image processor may encode the frames sequentially photographed to generate an original image and store the generated original image.

Referring back to FIG. 12, each of the frames included in the original photographed image may be divided into a predetermined number of blocks, and a first image may be generated by performing pixelation on all of the predetermined number of blocks. [S120].

The image processor may divide each of the frames included in the captured original image into a predetermined number of blocks. The image processor may generate the first image by performing pixelation on all of the predetermined number of blocks included in each of the frames constituting the original image. That is, the first image may be an image in which the entire area of each of the frames constituting the original image is pixelated.

The image processor may perform pixelization on each block. The intensity of pixelation for each block can be set equally. Frames in which the entire area is pixelated may be generated as the first image.

Referring back to FIG. 12, the image processor may extract and store first hash data from the first image [S130].

According to an embodiment, the image processor may extract frame hash data from each of the frames included in the first image, and may re-extract the first hash data from the extracted frame hash data. According to an embodiment, the image processor may simultaneously perform the operation of generating the first image and the extraction of the first hash data in parallel with the operation of storing the original image.

According to an embodiment, the image processor may delete the first image after extracting the first hash data. According to the present disclosure, since only the first hash data is used when verifying the integrity of the privacy protection image, the first image is deleted because it is no longer needed, thereby preventing unnecessary consumption of the memory.

According to an embodiment, the image processor may transmit the first hash data to the time stamp server. The image processor may receive the first hash data time stamped from the time stamp server and store the first hash data in the memory.

In FIG. 12, steps S110, S120, and S130 are illustrated in sequence, but are not limited thereto. According to an embodiment, the operation of storing the original image of step S110 and the operation of generating the first image of steps S120 and S130 and extracting the first hash data may be performed after encoding the captured frames to obtain the original image. May be performed as This is to prevent an error that may occur in the subsequent verification of the integrity of the image due to the loss due to encoding. Since this has been described above, a detailed description thereof will be omitted.

Referring back to FIG. 12, the privacy protection image generator of the image processing apparatus may detect an object to be privacy protected from the frames included in the stored original image [S140].

When the privacy protection image generation unit receives the image transmission request, the privacy protection image generation unit may read an original image from the memory and detect an object included in each frame. A well-known object detection algorithm may be applied to the detection of an object, and is not limited to a specific algorithm.

Privacy protection may include the face of a person included in the frame of the original image, the license plate of the car or a sign with the address. The privacy protection image generator may detect an object such as a face of a person or a license plate of a vehicle for each frame.

According to an embodiment, when the image protection request is received from the image verification apparatus, the privacy protection image generator may start detecting an object to be privacy protected from the original image. Referring to FIG. 13, when an image transmission request is received from the image verifying apparatus [S210, Yes], the privacy protection image generating unit may start detecting an object to be privacy protected [S220]. If the image transmission request is not received from the image verification apparatus [S210, No], the privacy protection image generating unit may perform operations S110 to S130 in the image processing unit without performing a separate operation.

Referring back to FIG. 12, the privacy protection image generator divides each of the frames included in the stored original image into a predetermined number of blocks and performs pixelization on an important block including an object among the predetermined number of blocks. The second image may be generated [S150].

The privacy protection image generator may divide each of the frames included in the stored original image into a predetermined number of blocks. The privacy protection image generator may distinguish the frames included in the stored original image in the same format as the blocks that divide the frames in order to generate the first image in the image processor. That is, each frame of the first image and each frame of the second image corresponding to each frame of the first image may be divided into blocks having the same number and the same shape.

The privacy protection image generator may generate a second image by performing pixelation on important blocks including an object to be privacy protected among a predetermined number of blocks that divide each frame. That is, the second image may be a privacy protection image in which only an important block of pixels constituting the original image is pixelated. The privacy protection image generator may perform pixelization on the critical block at the same pixelation intensity as the pixelation intensity where the blocks are pixelated to generate the first image. As such, frames in which the critical block of the entire area is pixelated may be generated as the second image.

Referring back to FIG. 12, the privacy protection image generation unit stores an index indicating the location of the critical block and the second image [S160], the privacy protection image generation unit is the second image and the index, and the image processing unit is the first hash data. May be transmitted to the image verification apparatus [S170].

The privacy protection image generator may acquire an index indicating the position of the critical block among the predetermined number of blocks for dividing each frame. The privacy protection image generator may store the index and the second image in a memory. When the image protection request is received from the image verification apparatus, the privacy protection image generator may transmit the index and the second image to the image verification apparatus. When the image transmission request is received from the image verification apparatus, the image processor may transmit the time stamped first hash data to the image verification apparatus.

According to this, when capturing an original image, hash data is extracted from an image obtained by pixelating the entire area of each frame, and when a request for the image is made, an object to be protected is generated to generate a pixelated privacy protection image. Regardless of the performance of the device, it is possible to efficiently generate a privacy-protected image.

Referring to FIG. 14, the image verification apparatus performs pixelation on the remaining blocks except for the pixelated critical block by including an object to be protected for privacy among the frames of the second image by using an index to perform a third image. Can be generated [S310].

The image verification apparatus may request transmission of an image to the image processing apparatus. The image verification apparatus may receive the first hash data, the second image, and the index, which are time stamped from the image processing apparatus, according to the request.

The image verification apparatus may classify each of the frames included in the second image in the same manner as a predetermined number of blocks that separate the first image from which the first hash data is extracted. That is, the frames included in the first image and the second image may be equally divided into blocks having the same number and the same shape, respectively. For example, as described above, it is assumed that each frame is divided into 16 identical rectangular blocks having a 4 * 4 matrix to generate the first image. In this case, the apparatus for verifying an image may equally classify the frames of the received second image into 16 identical rectangular blocks that are 4 * 4 matrices.

The image verification apparatus may perform pixelation on the remaining blocks except for the pixelated critical block by including an object to be protected for privacy among the frames of the second image by using the index. The image verification apparatus may perform pixelation processing on the remaining blocks other than the critical block by using an index indicating the significant block. The pixelation scheme and intensity applied to the critical block and the remaining blocks may be set to be the same and pixelation may be performed.

As such, the frames in which the entire area is pixelated may be generated as the third image. Accordingly, in the third image, the entire area of each frame of the original image is pixelated, similarly to the first image from which the first hash data is extracted.

Referring back to FIG. 14, the image verification apparatus may extract second hash data from the third image [S320].

According to an embodiment, the apparatus for verifying image may extract frame hash data from each frame included in the third image, and re-extract second hash data from the extracted frame hash data.

Referring back to FIG. 14, the apparatus for verifying image may compare second hash data with the first hash data to determine the integrity of the second image [S330].

The image verifying apparatus may determine the integrity of the second image by comparing the second hash data extracted from the third image with the first hash data. Like the first image, the third image is in the same pixelation process with respect to all blocks of the frames constituting the original image. Therefore, when the first hash data extracted from the first image and the second hash data extracted from the third image processed with the second image have the same value, the second image may be verified as an image without modulation. It can also be verified for the recording time by the time stamp of the first hash data.

According to this, the hash data is extracted from the pixelated image of the area where the object to be protected in the privacy protected image is not included in the privacy protected image to determine the integrity of the privacy protected image, thereby reliably verifying the integrity of the privacy protected image. Can be.

15 is a flowchart illustrating an operation of an image processing and verification system according to an embodiment of the present disclosure.

Referring to FIG. 15, the image processing apparatus 100 may store an original image photographed by the image capturing unit [S401]. The image processing apparatus 100 may generate an original image by encoding the frame photographed by the image capturing unit [S403]. The image processing apparatus 100 may decode the generated original image back into frames while storing the generated original image in memory in real time [S423].

The image processing apparatus 100 divides each of the frames included in the original image captured by the encoded original image into a predetermined number of blocks [S407], and pixelates each of the predetermined number of blocks. The first image may be generated [S409].

The image processing apparatus 100 may extract frame hash data from each frame included in the first image [S411], and extract the first hash data again from the extracted frame hash data [S413].

The image processing apparatus 100 may transmit the first hash data to the time stamp server 300 [S415]. The time stamp server 300 may sign the first hash data together with the current time [S417]. The image processing apparatus 100 may receive first hash data time-stamped from the time stamp server 300 [S419] and store it in a memory [S421].

The image verification apparatus 200 may request transmission of an image to the image processing apparatus 100 [S425]. When receiving the image transmission request, the image processing apparatus 100 may read an original image from the memory and detect an object included in each frame [S427].

The image processing apparatus 100 divides each of the frames included in the stored original image into a predetermined number of blocks, and generates a second image by performing pixelation on an important block including an object among the predetermined number of blocks. It can be done [S429]. The image processing apparatus 100 may store an index indicating the position of the critical block and the second image [S431]. The image processing apparatus 100 may transmit the second image, the index, and the time stamped first hash data to the image verifying apparatus [S433].

The image verifying apparatus 200 may classify each of the frames included in the second image in the same manner as a predetermined number of blocks that separate the first image from which the first hash data is extracted [S435]. The image verification apparatus 200 generates a third image by performing pixelation on the remaining non-significant blocks except for the pixelated critical block by including an object to be protected for privacy among the frames of the second image by using an index. It can be done [S437].

The image verification apparatus 200 may extract the second hash data from the third image [S439]. The image verification apparatus 200 may determine the integrity of the second image by comparing the second hash data with the first hash data [S441].

According to this, when capturing an original image, hash data is extracted from an image obtained by pixelating the entire area of each frame, and when a request for the image is made, an object to be protected is generated to generate a pixelated privacy protection image. Regardless of the performance of the device, it is possible to efficiently generate a privacy-protected image. In addition, by extracting the hash data from the pixelated image of the area that does not include the object to be protected in the privacy protection image, the integrity of the privacy protection image can be judged with high reliability. have.

In the above, it has been described that the image processing and verification method according to the embodiment of the present disclosure is performed by the same procedure as in FIG. 15, but this is only for convenience of description and within the scope not departing from the essential concept of the present disclosure. Depending on the implementation method, the execution procedure of each step may be changed, two or more steps may be integrated, or one step may be performed in two or more steps.

In the foregoing description, the present disclosure is not necessarily limited to these embodiments, although all components constituting the embodiments of the present disclosure are described as being combined or operating in combination. In other words, within the scope of the present disclosure, all of the components may be selectively operated in combination with one or more. In addition, although all of the components may be implemented in one independent hardware, each or all of the components may be selectively combined to perform some or all functions combined in one or a plurality of hardware. It may be implemented as a computer program having a. Codes and code segments constituting the computer program may be easily inferred by those skilled in the art of the present disclosure. Such a computer program may be stored in a computer readable storage medium and read and executed by a computer, thereby implementing embodiments of the present disclosure. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, and the like.

In addition, the terms "comprise", "comprise" or "having" described above mean that the corresponding component may be included, unless otherwise stated, and thus excludes other components. It should be construed that it may further include other components instead. All terms, including technical and scientific terms, have the same meanings as commonly understood by one of ordinary skill in the art to which this disclosure belongs, unless defined otherwise. Terms commonly used, such as terms defined in a dictionary, should be interpreted to coincide with the contextual meanings of the related art, and shall not be construed in an ideal or excessively formal sense unless explicitly defined in the present disclosure.

The above description is merely illustrative of the technical spirit of the present disclosure, and various modifications and variations may be made by those skilled in the art without departing from the essential characteristics of the present disclosure. Therefore, the embodiments disclosed in the present disclosure are not intended to limit the technical spirit of the present disclosure but to describe the present disclosure, and the scope of the technical spirit of the present disclosure is not limited by these embodiments. The protection scope of the present disclosure should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto shall be construed as being included in the scope of the present disclosure.

10: image processing and verification system 100: image processing device
110: image capturing unit 120: image processing unit
130: memory 140: privacy protection image generation unit
150: communication unit 200: image verification device
300: timestamp server

Claims (20)

Memory;
An image photographing unit;
The original image photographed by the image capturing unit is stored in the memory, and each of the frames included in the captured original image is divided into a predetermined number of blocks, and each of all the predetermined numbers of blocks is provided. An image processor configured to generate a first image by performing pixelation, and extract and store first hash data from the first image; And
Detecting objects which are subject to privacy protection from the frames included in the stored original image, dividing each of the frames included in the stored original image into the predetermined number of blocks, wherein the object among the predetermined number of blocks A privacy protection image generation unit which generates a second image by performing pixelation on an included critical block, and stores an index indicating a position of the critical block and the second image in the memory;
Image processing apparatus comprising a. The method of claim 1,
After the image processing unit acquires the original image by encoding the frames photographed by the image capturing unit, the storing operation of the captured original image and the generating operation of the first image using the captured original image are performed in parallel. Image processing apparatus to perform. The method of claim 1,
Each frame of the first image and each frame of the second image corresponding to each frame of the first image are divided into blocks having the same number and the same type. The method of claim 1,
And the image processor extracts frame hash data from each frame included in the first image, and extracts the first hash data from the extracted frame hash data. The method of claim 1,
It further comprises a communication unit;
The image processor transmits the first hash data to a time stamp server through the communication unit, and receives and stores time-stamped first hash data from the time stamp server. Device. The method of claim 1,
And an intensity of pixelation performed for each block of the first image and an intensity of pixelation performed for the critical block of the second image. The method of claim 1,
It further comprises a communication unit;
When the image transmission request is received from the image verification apparatus through the communication unit, the privacy protection image generation unit generates the second image and transmits the second image to the image verification apparatus along with the index, and the image processing unit transmits the first hash data. And an image processing apparatus for transmitting to the image verification apparatus. Communication unit; And
Receiving first hash data, a second image, and an index from the image processing apparatus of claim 1 through the communication unit, and separating each of the frames included in the second image from the first image from which the first hash data is extracted. By dividing the same number as a predetermined number of blocks, pixelization is performed on the remaining blocks except for the pixelated critical block by including the object to be protected for privacy among the frames of the second image by using the index. A controller configured to generate a third image and compare the second hash data extracted from the third image with the first hash data to determine an integrity of the second image;
Image verification apparatus comprising a. The method of claim 8,
The controller extracts frame hash data from each frame included in the third image, and extracts the second hash data from the extracted frame hash data. The method of claim 8,
And an intensity of pixelation performed on the significant block of the second image and a pixelization intensity performed on the remaining blocks except for the significant block of the second image. Storing the original image photographed by the image capturing unit;
Dividing each of the frames included in the captured original image into a predetermined number of blocks, and performing pixelation on all of the predetermined number of blocks to generate a first image;
Extracting and storing first hash data from the first image;
Detecting an object to be privacy protected from the frames included in the stored original image;
Generating a second image by dividing each of the frames included in the stored original image into the predetermined number of blocks, and performing pixelation on an important block including the object among the predetermined number of blocks; And
Storing an index indicating a location of the critical block and the second image;
Image processing and verification method comprising a. The method of claim 11,
And storing the original image and generating the first spirituality are performed in parallel after encoding the frames photographed by the image capturing unit to obtain the original image. The method of claim 11,
And each frame of the second image corresponding to each frame of the first image and each frame of the first image is divided into blocks having the same number and the same type. The method of claim 11,
The extracting and storing the first hash data may include extracting frame hash data from frames included in the first image and extracting the first hash data from the extracted frame hash data. Way. The method of claim 11,
The extracting and storing the first hash data may include transmitting the first hash data to a time stamp server, and receiving and storing the first hash data time stamped from the time stamp server. The method of claim 11,
And the intensity of pixelation performed on each block of the first image and the intensity of pixelation performed on the critical block of the second image are equally set. The method of claim 11,
The generating of the second image is performed when an image transmission request is received from the image verification apparatus.
Transmitting the second image, the index, and the first hash data to the image verification device;
Image processing and verification method further comprising. The method of claim 17,
Generating a third image by performing pixelation on the remaining blocks except for the pixelated important block including an object to be protected for privacy among the frames of the second image by using the index;
Extracting second hash data from the third image; And
Comparing the second hash data with the first hash data to determine the integrity of the second image;
Image processing and verification method further comprising. The method of claim 18,
The extracting of the second hash data may include extracting frame hash data from respective frames included in the third image, and extracting the second hash data from the extracted frame hash data. The method of claim 18,
And the intensity of pixelation performed on the significant block of the second image and the intensity of pixelation performed on the remaining blocks except for the significant block of the second image.

Images