KR102095968B1 - Image Processing Apparatus and Image Verification Apparatus for Protecting Privacy and Preventing Alteration, and Method thereof - Google Patents

Abstract

In the present disclosure, the first hash data is extracted from the first image in which the entire area is pixelated for each of the frames constituting the original image, and the second image and the important pixelized only for the important block including the object to be protected by privacy By creating an index indicating the position of the block, generating a third image pixelated for a non-critical block from the second image, and comparing the second hash data extracted from the third image with the first hash data, privacy is achieved. Provided is an image processing apparatus, an image verification apparatus, and a method for providing a protected second image and verifying the integrity of the second image. According to the present disclosure, it is possible to efficiently protect privacy in an image and verify whether the image is tampered with.

Description

Image processing apparatus, image verification apparatus, and method for preventing privacy and tampering of images {Image Processing Apparatus and Image Verification Apparatus for Protecting Privacy and Preventing Alteration, and Method thereof}

The present disclosure relates to an apparatus and method for processing an image in order to protect the privacy included in the image and verify whether the image is falsified.

In recent years, with the development of image technology and network technology, various and low-cost Internet of Things (IoT) cameras supporting a network are widely spread. Various images captured through these cameras are shared through the network.

These shared videos can be used as video evidence for various events, which can benefit the public. However, since there is a risk that privacy information such as an individual's face in a video, a vehicle license plate, or an address sign may be exposed to the public, a protection method is required.

In addition, various image processing applications have been spread, such as copy-move, splicing, retouching, erasing, changing lighting conditions, frame insertion, As it becomes easy to modulate an image, such as frame deletion and frame switch, a method for guaranteeing time information and integrity for a shared image is needed.

According to this need, a method for identifying a region corresponding to privacy when photographing an image, and protecting privacy through blurring or the like on the identified region has been provided. There are problems such as deterioration of image quality due to limited processing power. In addition, such blurring may have a problem of invalidating the authenticated hash through time stamping.

Accordingly, there is an increasing need for research on a method for effectively protecting privacy in an image and verifying the integrity of the image.

In the above-described background, the present disclosure extracts hash data from an image in which the entire area of each frame is pixelated when photographing an original image, and detects an object that is a privacy-protected object when an image is requested. By creating, to propose an image processing apparatus and method capable of efficiently generating a privacy-protected image regardless of the performance of the apparatus.

In addition, the present disclosure extracts hash data from an image obtained by performing pixelation on an area that does not include an object to be protected in a privacy-protected image to determine the integrity of the privacy-protected image, thereby verifying the integrity of the privacy-protected image with high reliability. We would like to propose an image verification device and method that can be done.

The present disclosure, devised to solve the above-described problems, stores a memory, an image capturing unit, and an original image captured by the image capturing unit in a memory, and blocks each of the frames included in the captured original image by a predetermined number of blocks. The image processing unit for storing the first hash data extracted from the first image and storing the first hash data from the first image by dividing them into pixels and performing pixelation on all of the predetermined number of blocks, respectively, and stored An object that is a privacy-protected object is detected from frames included in the original image, each of the frames included in the stored original image is divided into a predetermined number of blocks, and an important block including an object among the predetermined number of blocks (critical) A blocker that performs pixelation on a block to generate a second image, and stores an index indicating the location of the important block and a second image in memory. It provides an image processing apparatus comprising image generation unit protection.

In addition, the present disclosure receives the first hash data, the second image and the index from the image processing apparatus through the communication unit, the communication unit, and distinguishes the first image from which the first hash data is extracted from each of the frames included in the second image. It is divided into the same as one predetermined number of blocks, and the third block is obtained by performing pixelation on the remaining blocks except for the important block that is the pixel-protected object among the frames of the second image using the index. Provided is an image verification apparatus including a control unit that generates an image and compares the second hash data extracted from the third image with the first hash data to determine the integrity of the second image.

In addition, the present disclosure is a step of storing the original image photographed by the image capturing unit, each of the frames included in the photographed original image is divided into a predetermined number of blocks, and pixelation is performed for all of the predetermined number of blocks. Performing, generating a first image, extracting and storing first hash data from the first image, detecting each object that is a privacy object in frames included in the stored original image, included in the stored original image The step of generating a second image by dividing each of the frames into a predetermined number of blocks and performing pixelation on the important block including the object among the predetermined number of blocks, and the index and the index indicating the location of the important block It provides an image processing and verification method comprising the step of storing 2 images.

According to the present disclosure, when photographing an original image, hash data is extracted from a pixelated image of the entire area of each frame, and when an image is requested, an object to be protected is detected to generate a pixelated privacy-protected image. By doing so, it is possible to efficiently create a privacy-protected image regardless of the performance of the device.

In addition, according to the present disclosure, by extracting hash data from an image in which pixelation is performed on an area that does not include an object to be protected in a privacy-protected image, the integrity of the privacy-protected image is determined with high reliability. There is an effect that can be verified.

1 is a block diagram of an image processing and verification system according to an embodiment of the present disclosure.
2 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure.
3 is a block diagram of an image processing unit included in an image processing apparatus according to an embodiment of the present disclosure.
4 to 6 are views for explaining the operation of the image processing unit according to an embodiment of the present disclosure.
7 is a block diagram of a privacy-protected image generator included in an image processing apparatus according to an embodiment of the present disclosure.
8 and 9 are views for explaining the operation of the privacy-protected image generating unit according to an embodiment of the present disclosure.
10 is a block diagram of an image verification apparatus according to an embodiment of the present disclosure.
11 is a view for explaining the operation of the image verification apparatus according to an embodiment of the present disclosure.
12 is a flowchart of an image processing and verification method according to an embodiment of the present disclosure.
13 is a flowchart illustrating a start timing of generation of a second image according to an embodiment of the present disclosure.
14 is a flowchart of an image processing and verification method according to an embodiment of the present disclosure.
15 is a flowchart illustrating an operation of an image processing and verification system according to an embodiment of the present disclosure.

Hereinafter, some embodiments of the present disclosure will be described in detail through exemplary drawings. It should be noted that in adding reference numerals to the components of each drawing, the same components have the same reference numerals as possible even though they are displayed on different drawings. In addition, in describing the present disclosure, when it is determined that detailed descriptions of related known configurations or functions may obscure the subject matter of the present disclosure, detailed descriptions thereof will be omitted.

In addition, in describing the components of the present disclosure, terms such as first, second, A, B, (a), and (b) may be used. These terms are only for distinguishing the component from other components, and the nature, order, or order of the component is not limited by the term. When a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected to or connected to the other component, but another component between each component It should be understood that elements may be "connected", "coupled" or "connected".

In the present disclosure, the “original image” refers to an image captured by an image processing apparatus, and refers to an image before processing for privacy protection such as blurring. In addition, "block (block)" refers to an area such as a grid shape that divides the entire area of a frame constituting an image. Further, "pixelate" refers to a process that makes it difficult to identify an image in a corresponding block, such as blurring or mosaic processing, for each block. In addition, "critical block (critical block)" means a block containing an object that may be a privacy problem among the blocks. In addition, "index (index)" means information indicating the location of the important block among the blocks that separate the frame.

Hereinafter, an image processing apparatus, an image verification apparatus, and a method according to embodiments of the present disclosure will be described with reference to related drawings.

1 is a block diagram of an image processing and verification system according to an embodiment of the present disclosure.

Referring to FIG. 1, the image processing and verification system 10 includes an image processing apparatus 100, an image verification apparatus 200, and a time stamp server 300.

The image processing apparatus 100 may include an apparatus that photographs an image and transmits the photographed image when requested. The image processing apparatus 100 is not limited to a specific type, as long as it can take an image and perform the image processing method according to the present disclosure. For example, the image processing apparatus 100 may include electronic devices in various fields, such as a security camera, a black box camera installed in a vehicle, or a camera provided in a drone.

In particular, according to the present disclosure, an excellent privacy protection effect and reliability of an image can be ensured even in a low performance device, and an image with good image quality can be generated. Accordingly, the image processing apparatus 100 according to an embodiment of the present disclosure may include a low specification processor (700 MHz-1.2 GHz CPU), and may be implemented as an inexpensive camera with a wireless network interface for IoT connection. . However, this is not limited to this as an example, and it will be understood that it may be implemented with various performance devices.

The image processing apparatus 100 may detect an object in which privacy may be a problem in the captured image. For example, the object in which the privacy may be a problem may mean an object including personal information such as a human face, a license plate of a car, or a home address sign. The image processing apparatus 100 may generate a privacy-protected image by processing an object in which privacy may be a problem so that it cannot be recognized.

In relation to the generation of the privacy-protected image, the conventional blurring procedure acquires a real-time frame from the camera module, blurs the detected area, and records the frame with the face / number plate blurred as a video file. do. Of the I / O time for acquiring and recording a real-time frame, detection time for a detection operation such as a face, and pixelate time for a blur processing operation, the most time in the process of generating a privacy-protected image The time consuming operation is detection time. Table 1 below is an example of time spent when performing real-time blur processing of an image using a Raspberry Pi single board computer. Referring to Table 1, it is shown that the most time is consumed when detecting the face / number plate.

Processing time taken per frame Object blurring Resulting frame rate Detect Pixelate I / O Human face 2.3 fps 431.5 ms (89.7%) 0.05 ms (0.01%) 47.4 ms (10.2%) Car plate 5.1 fps 146.2 ms (75.5%) 0.05 ms (0.02%) 47.1 ms (24.4%)

By using these characteristics, the image processing apparatus 100 according to an embodiment of the present disclosure may preferentially store an original image without performing real-time detection of an object that takes the most time along with image capture. In addition, as a measure for verifying the integrity of the image, the image processing apparatus 100 may pixelate each frame of the original image in real time and extract hash data. In addition, in order to verify the recording time of the image, the image processing apparatus 100 may receive the time stamped hash data by transmitting hash data to a time stamp server 300 on the network. have. Securing the reliability of data through the time stamp server follows a known method, and a detailed description will be omitted.

Thereafter, when an image transmission request is received from the image verification apparatus 200, the image processing apparatus 100 may detect an object for the stored original image. Since the detection of the object in the corresponding step does not need to be performed in real time along with the imaging of the image, it can be efficiently performed even with a lower specification CPU. The image processing apparatus 100 may generate a privacy-protected image by performing pixelation processing on the detected object, and may store index information indicating the pixelated region. The image processing apparatus 100 may transmit the privacy-protected image and index information to the image verification apparatus 200.

The image verification device 200 may be a device such as a mobile terminal or a computer used by a person or institution requesting an image captured by the image processing device 100. The image verification apparatus 200 may request transmission of an image to the image processing apparatus 100, and is not limited to a specific type if it is possible to perform the image verification method according to the present disclosure.

The image verification apparatus 200 may receive the time stamped hash data and the privacy-protected image and index information from the image processing apparatus 100. The image verification apparatus 200 may pixelize a portion that is not pixelated in the privacy-protected image using the index information. Accordingly, an image in which the entire area of each frame is pixelated may be generated. The image verification apparatus 200 may extract hash data from the generated image. The image verification apparatus 200 may verify the integrity of the privacy-protected image by comparing the extracted hash data with time-stamped hash data received from the image processing apparatus 100.

When the integrity is verified, the image verification device 200 may use it, such as publishing a privacy-protected image or submitting it as evidence. In this case, since the original image that is not pixelated is stored only in the image processing apparatus 100, the image verification apparatus 200 cannot access the original image. Therefore, the privacy that may be involved in the captured original image can be protected.

According to this, protection and integrity of the privacy of the captured image can be efficiently verified. Hereinafter, an image processing and verification method according to embodiments of the present disclosure will be described in more detail with reference to related drawings.

2 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure. 3 is a block diagram of an image processing unit included in an image processing apparatus according to an embodiment of the present disclosure. 4 to 6 are views for explaining the operation of the image processing unit according to an embodiment of the present disclosure. 7 is a block diagram of a privacy-protected image generator included in an image processing apparatus according to an embodiment of the present disclosure. 8 and 9 are views for explaining the operation of the privacy-protected image generating unit according to an embodiment of the present disclosure.

Referring to FIG. 2, the image processing apparatus 100 according to embodiments of the present disclosure stores an image captured by the image photographing unit 110 and the image photographing unit in a memory, and a frame included in the photographed original image Each of these is divided into a predetermined number of blocks, and pixelation is performed on all of the predetermined number of blocks to generate a first image and first hash data from the first image. ), And extracts and stores the image processing unit 120, the memory 130, and objects that are subject to privacy protection from the frames included in the stored original image, and each of the frames included in the stored original image is a predetermined number of blocks The second image is generated by performing pixelation on a critical block including an object among a predetermined number of blocks, and the index and the second image indicating the location of the important block are memoized. It includes a privacy-protected image generation unit 140 and the communication unit 150 to store the data.

The image photographing unit 110 may include an image sensor and the like to photograph an image in frame units. The image photographing unit 110 may transmit the photographed frames to the image processing unit 120. The image photographing unit 110 is sufficient if it is possible to photograph an image, and is not limited to a specific device.

Referring to FIG. 3, the image processing unit 120 includes an encoder 121, an original image storage unit 122, a decoder 123, a first image generation unit 124, a hash data extraction unit 125, and hash data storage Part 126 may be included.

The encoder 121 included in the image processing unit 120 may generate an original image by encoding a frame captured by the image capturing unit 100. For example, the encoder 121 may encode a frame as a JPEG image. Accordingly, the original image may be stored in MJPEG (Motion JPEG) format. However, this is an example, and is not limited thereto. In the encoder 121, encoding may be performed in other ways, if necessary.

The image processing unit 120 may store the generated original image in real time in the memory 130 through the original image storage unit 122. Here, storing in real time may mean that the original image is stored in the memory 130 while the image is captured in the image capturing unit 100. That is, the image processing unit 120 may encode the frames that are sequentially photographed to generate an original image and store the generated original image.

The image processing unit 120 may decode the generated original image into frames again in the decoder 123 while storing the original image generated from the encoder 121. That is, the storage path of the original image and the decoding path of the original image may be implemented as separate paths.

The encoding of the original image is first performed in common, and then the reason for storing the original image and extracting the first hash data is to prevent errors due to lossy compression encoding when verifying the integrity of the privacy-protected image. That is, the original image is stored in an encoded state by the lossy compression method, and the first hash data is extracted based on the encoded state, thereby preventing errors due to the lossy compression method in future integrity verification.

For example, it is assumed that the first hash data is directly extracted from the frames photographed by the image photographing unit 110. In this case, the privacy-protected image is generated based on the original image in which loss due to lossy compression is generated, and second hash data for it is extracted, whereas the first hash data is extracted based on frames without such loss. Then, when comparing the second hash data and the first hash data for verifying the integrity of the privacy-protected image, there is a fear that an error due to lossy compression may occur. Accordingly, in order to prevent such an error, a viewpoint of a storage path and a decoding path of the original image is set as the encoded original image.

The first image generation unit 124 included in the image processing unit 120 may divide each of the frames included in the captured original image into a predetermined number of blocks. The image processor 120 may generate a first image by performing pixelation on all of a predetermined number of blocks included in each of the frames constituting the original image. That is, the first image may be an image in which the entire area of each frame constituting the original image is pixelated.

Referring to FIG. 4, one frame i photographed by the image photographing unit 110 is illustrated. The frame (i) includes two people (1, 2). Referring to FIG. 5, according to an example, it is illustrated that a frame (i) is divided into 16 equal square blocks (b) of a 4 * 4 matrix. Here, the frame is divided into 16 blocks for convenience of description, but is not limited thereto. In addition, each block is composed of a square, but is not limited thereto, and may be set in various shapes as necessary.

The first image generator 124 may perform pixelation on each block. The intensity of pixelation for each block can be set equally. Referring to FIG. 6, pixels that are pixelated (bp) for all blocks included in the frame are indicated by diagonal lines. Accordingly, people included in the frame also appear to be pixelated (1p, 2p). As such, frames i1 in which the entire area is pixelated may be generated as a first image.

According to an embodiment, the pixelation intensity of pixelizing each block may be determined in consideration of an object identification possibility and an image modulation detection possibility. The pixelation intensity indicates the degree of lowering the original resolution. For example, the pixelation intensity X may mean reducing the resolution by a ratio of X ² by averaging the pixel values of the image.

The higher the pixelation intensity, the lower the probability of object identification. In addition, if the pixelation intensity is too high, the possibility of detecting modulation in an image may be reduced. Accordingly, an appropriate pixelation intensity may be selected so as to increase the possibility of detecting modulation of an image while lowering the possibility of identifying an object. According to an example, it may be desirable that the pixelation intensity X is set between 11 and 15. However, this is an example, and is not limited thereto, and may be set differently depending on cases such as image quality.

The hash data extraction unit 125 included in the image processing unit 120 may extract and store the first hash data from the first image. According to an embodiment, the image processor 120 may extract frame hash data from each frame included in the first image, and extract the first hash data again from the extracted frame hash data. According to an embodiment, the image processing unit 120 may simultaneously perform the operation of generating the first image and the operation of extracting the first hash data in parallel with the operation of storing the original image.

According to an embodiment, the hash data extractor 125 may delete the first image after extracting the first hash data. According to the present disclosure, since only the first hash data is used when verifying the integrity of the privacy-protected image, the first image is deleted because it is no longer needed, and accordingly, unnecessary memory consumption can be prevented.

The image processing unit 120 may transmit the first hash data to the time stamp server 300 through the communication unit 150. The image processing unit 120 may receive the first hash data time-stamped from the time stamp server 300 through the hash data storage unit 126 and store it in the memory 130. When the image transmission request is received from the image verification apparatus 200, the hash data storage unit 126 may transmit the time-stamped first hash data to the image verification apparatus 200.

The memory 130 stores image data processed by the image processing unit 120 and the privacy-protected image generation unit 140 and data related thereto. The memory 130 may store a program for performing an operation such as dividing a frame into blocks or pixelating. According to an example, the memory 130 may be implemented as a card-type memory (eg, SD or XD memory, etc.). For example, a low-end model image processing device may include a 64-128 GB onboard SD memory card. However, this is not limited to this as an example, and the memory 130 may be implemented as various storage media as necessary.

Referring to FIG. 7, the privacy-protected image generation unit 140 may include an object detection unit 141, a second image generation unit 142, and an index acquisition unit 143.

The object detection unit 141 included in the privacy-protected image generation unit 140 may detect objects that are privacy-protected objects in frames included in the stored original image. Upon receiving the request for transmitting the image, the privacy-protected image generating unit 140 may read the original image from the memory 130 and detect an object included in each frame. A known object detection algorithm may be applied to the detection of the object, and is not limited to a specific algorithm.

The privacy-protected image generating unit 140 may detect an object such as a human face or a license plate of a car for each frame. Referring to FIG. 8, faces of people 1 and 2 included in the frame i of the stored original image may be subject to privacy protection. The object detection unit 141 may detect the faces of people 1 and 2 and acquire the detected areas d1 and d2.

The second image generator 142 included in the privacy-protected image generator 140 may divide each of the frames included in the stored original image into a predetermined number of blocks. In order to generate the first image in the image processing unit 120, the privacy-protected image generation unit 140 may classify frames included in the stored original image in the same format as blocks in which the frames are divided. That is, each frame of the first image and each frame of the second image corresponding to each frame of the first image may be divided into blocks having the same number and the same shape.

For example, as described above, it is assumed that each frame is divided into 16 equal square blocks, each of which is a 4 * 4 matrix, to generate a first image. In this case, the second image generation unit 142 may equally divide the frames included in the stored original image into blocks of 16 identical squares, which are 4 * 4 matrices.

The second image generation unit 142 may generate a second image by performing pixelation on critical blocks including an object to be protected, among a predetermined number of blocks that divide each frame. That is, the second image may be a privacy-protected image in which only important blocks among the frames constituting the original image are pixelated. The privacy-protected image generation unit 140 may perform pixelation of the important block with the same pixelation intensity as the pixelated intensity of each of the blocks, in order to generate the first image in the image processing unit 120. .

Referring to FIG. 9, pixelated processing of important blocks cb1 and cb2 including a region in which a human face is detected among blocks included in the frame is illustrated by a diagonal line. Accordingly, the faces of people included in the important blocks cb1 and cb2 also appear to be pixelated. As such, frames i2 in which the important blocks cb1 and cb2 of the entire area are pixelated may be generated as a second image.

The index obtaining unit 143 included in the privacy-protected image generating unit 140 may obtain an index indicating the location of an important block among a predetermined number of blocks that divide each frame. The privacy-protected image generating unit 140 may store the index and the second image in the memory 130. When the image transmission request is received from the image verification apparatus 200, the privacy-protected image generation unit 140 may transmit the index and the second image to the image verification apparatus 200.

According to an embodiment, the privacy-protected image generation unit 140 may compress the second image in a lossless compression method and transmit the compressed image to the image verification apparatus 200. As described above, in order to prevent errors due to lossy compression when verifying the integrity of the second image, the original image is preferentially encoded, and then the same as that of storing the original image and extracting the first hash data, 2 In order to prevent the occurrence of errors due to lossy compression even when the image is transmitted, it can be transmitted in a lossless compression method.

The communication unit 150 may include one or more modules that enable wireless communication between the image processing apparatus 100 and the image verification apparatus 200 or between the image processing apparatus 100 and the time stamp server 300. According to an example, the communication unit 150 may transmit and receive hash data using the time stamp server 300 and the IoT communication channel. The communication unit 150 may include a mobile communication module, a wireless Internet module, or a short-range communication module for exchanging data with the image verification device 200.

According to this, when photographing the original image, by extracting hash data from the image in which the entire area of each frame is pixelated, and when the image is requested, an object to be protected is detected to generate a pixelated privacy-protected image, It is possible to efficiently create a privacy-protected image regardless of the performance of the device.

10 is a block diagram of an image verification apparatus according to an embodiment of the present disclosure. 11 is a view for explaining the operation of the image verification apparatus according to an embodiment of the present disclosure.

10, the image verification apparatus 200 according to an embodiment of the present disclosure receives the first hash data, the second image and the index from the above-described image processing apparatus 100 through the communication unit 210 and the communication unit , A third image is generated by performing pixelation on the remaining blocks except for the important block that is pixelated by including an object to be privacy-protected among each frame of the second image using the index, and the third image is extracted from the third image. 2 may include a control unit 220 that determines the integrity of the second image by comparing the hash data with the first hash data.

The communication unit 210 may include one or more modules that enable wireless communication between the image processing apparatus 100 and the image verification apparatus 200. According to an example, the communication unit 210 may include a mobile communication module, a wireless Internet module, or a short-range communication module for exchanging data with the image processing apparatus 100.

The control unit 220 may request transmission of an image to the image processing apparatus 100 through the communication unit 210. The controller 220 may receive the first hash data, the second image, and the index time-stamped from the image processing apparatus 100 according to the request. As described above, the time-stamped first hash data is extracted in a state in which all blocks that separate each frame of the original image are pixelated. In addition, the second image is a privacy-protected image in which the important block indicated by the index is pixelated.

The controller 220 may classify each of the frames included in the second image in the same manner as a predetermined number of blocks that divide the first image from which the first hash data is extracted. That is, the frames included in the first image and the second image may be equally divided into blocks of the same number and the same shape, respectively.

The control unit 220 may perform pixelation on the remaining blocks except for the important block that has been subjected to privacy protection among the frames of the second image, which is an object to be protected, by using the index. Referring to FIG. 9, it is illustrated by diagonal lines that the important blocks cb1 and cb2 included in the frame of the second image are pixelated. Similarly, referring to FIG. 11, it is illustrated by diagonal lines that the important blocks cb1 and cb2 are pixelated.

The controller 220 may perform pixelation processing on the remaining blocks bp other than the important blocks cb1 and cb2 using the indexes indicating the important blocks cb1 and cb2. In order to distinguish from the important blocks cb1 and cb2, in FIG. 11, the remaining blocks bp are pixelated, as shown by vertical lines.

However, this is illustrated separately for convenience of description, and the pixelation method and intensity applied to the important blocks cb1 and cb2 and the rest of the blocks bp are set to be the same and pixelation can be performed. Accordingly, people 1p and 2p included in the image are also entirely pixelated. As such, the frames i3 of which the entire area is pixelated may be generated as a third image. Therefore, in the third image, the entire area of each frame of the original image is pixelated in the same manner as the first image from which the first hash data is extracted.

According to an embodiment, the controller 220 may extract frame hash data from each frame included in the third image, and extract the second hash data from the extracted frame hash data again. The controller 220 may compare the second hash data extracted from the third image with the first hash data to determine the integrity of the second image.

As described above, the third image is in the same state as the first image, in the same pixel processing for all blocks of the frames constituting the original image. Accordingly, when the first hash data extracted from the first image and the second hash data extracted from the third image from which the second image is processed have the same value, the second image may be verified as an image without modulation. Also, the recording time can be verified by the time stamp of the first hash data.

According to this, by extracting hash data from an image that has been subjected to pixelation on an area that does not include an object to be protected in the privacy-protected image, the integrity of the privacy-protected image is determined and the reliability of the privacy-protected image is verified with high reliability. You can.

12 is a flowchart of an image processing and verification method according to an embodiment of the present disclosure. 13 is a flowchart illustrating a start timing of generation of a second image according to an embodiment of the present disclosure. 14 is a flowchart of an image processing and verification method according to an embodiment of the present disclosure.

The image processing and verification method according to the present disclosure may be implemented in the image processing apparatus 100 and the image verification apparatus 200 described with reference to FIG. 1. Hereinafter, a method for processing and verifying an image according to the present disclosure and operations of the image processing apparatus 100 and the image verification apparatus 200 for realizing the image will be described in detail with reference to necessary drawings.

Referring to FIG. 12, the image processing unit of the image processing apparatus may store the original image photographed by the image capturing unit [S110].

The image processing unit may generate an original image by encoding a frame captured by the image capturing unit. The image processing unit may store the generated original image in memory in real time. That is, the image processing unit may encode frames sequentially photographed to generate an original image and store the generated original image.

Referring to FIG. 12 again, each frame included in the photographed original image is divided into a predetermined number of blocks, and pixelation is performed on all of the predetermined number of blocks to generate a first image. [S120].

The image processing unit may divide each of the frames included in the photographed original image into a predetermined number of blocks. The image processing unit may generate a first image by performing pixelation on all of a predetermined number of blocks included in each of the frames constituting the original image. That is, the first image may be an image in which the entire area of each frame constituting the original image is pixelated.

The image processing unit may perform pixelation for each block. The intensity of pixelation for each block can be set equally. Frames in which the entire area is pixelated may be generated as a first image.

Referring to FIG. 12 again, the image processing unit may extract and store the first hash data from the first image [S130].

According to an embodiment, the image processing unit may extract frame hash data from each frame included in the first image, and extract the first hash data from the extracted frame hash data. According to an embodiment, the image processing unit may simultaneously perform the operation of generating the first image and the operation of extracting the first hash data in parallel with the operation of storing the original image.

According to an embodiment, the image processing unit may delete the first image after extracting the first hash data. According to the present disclosure, since only the first hash data is used when verifying the integrity of the privacy-protected image, the first image is deleted because it is no longer needed, thereby preventing unnecessary memory consumption.

According to an embodiment, the image processing unit may transmit the first hash data to the time stamp server. The image processing unit may receive the first hash data time-stamped from the time stamp server and store it in the memory.

In FIG. 12, steps S110, S120, and S130 are sequentially shown, but are not limited thereto. According to one embodiment, the operation of storing the original image in step S110 and the operation of generating the first image in steps S120 and S130 and extracting the first hash data are parallel after encoding the captured frames to obtain the original image. Can be performed. This is to prevent an error that may occur when verifying the integrity of a video due to loss due to encoding, and since it has been described above, a detailed description will be omitted.

Referring back to FIG. 12, the privacy-protected image generation unit of the image processing apparatus may detect objects that are privacy-protected objects in frames included in the stored original image [S140].

Upon receiving the request to transmit the image, the privacy-protected image generation unit may read the original image from the memory and detect an object included in each frame. A known object detection algorithm may be applied to the detection of the object, and is not limited to a specific algorithm.

The face of the person included in the frame of the original video, the license plate of the car or the sign with the address, etc., can be the subject of privacy protection. The privacy-protected image generation unit may detect an object such as a human face or a license plate of a car for each frame.

According to an embodiment, when the image transmission request is received from the image verification apparatus, the privacy-protected image generation unit may start detecting an object that is a privacy-protected object from the original image. Referring to FIG. 13, when an image transmission request is received from the image verification device [S210, Yes], the privacy-protected image generation unit may start detection of an object that is a privacy-protected object [S220]. When an image transmission request is not received from the image verification device [S210, No], the privacy-protected image generation unit may perform operations S110 to S130 in the image processing unit without performing a separate operation.

Referring to FIG. 12 again, the privacy-protected image generation unit divides each of the frames included in the stored original image into a predetermined number of blocks, and performs pixelation on an important block including an object among the predetermined number of blocks. A second image may be generated [S150].

The privacy-protected image generation unit may divide each of the frames included in the stored original image into a predetermined number of blocks. In order to generate the first image in the image processing unit, the privacy-protected image generation unit may classify frames included in the stored original image in the same format as blocks in which the frames are divided. That is, each frame of the first image and each frame of the second image corresponding to each frame of the first image may be divided into blocks having the same number and the same shape.

The privacy-protected image generation unit may generate a second image by performing pixelation on important blocks including an object to be privacy-protected among a predetermined number of blocks that divide each frame. That is, the second image may be a privacy-protected image in which only important blocks among the frames constituting the original image are pixelated. In order to generate the first image in the image processing unit, the privacy-protected image generation unit may perform pixelation of the important block with a pixelation intensity equal to the pixelation intensity of each block. As such, frames in which an important block of the entire area is pixelated may be generated as a second image.

Referring back to FIG. 12, the privacy-protected image generator stores the index indicating the location of the important block and the second image [S160], the privacy-protected image generator generates a second image, an index, and the image processor generates first hash data. Can be transmitted to the image verification device [S170].

The privacy-protected image generation unit may obtain an index indicating the location of an important block among a predetermined number of blocks that divide each frame. The privacy-protected image generation unit may store the index and the second image in memory. When the image transmission request is received from the image verification apparatus, the privacy-protected image generation unit may transmit the index and the second image to the image verification apparatus. When the image transmission request is received from the image verification apparatus, the image processing unit may transmit the time-stamped first hash data to the image verification apparatus.

According to this, when photographing the original image, by extracting hash data from the pixelated image of the entire area of each frame, and when the image is requested, the object to be protected is detected to generate a pixelated privacy-protected image, It is possible to efficiently create a privacy-protected image regardless of the performance of the device.

Referring to FIG. 14, the image verification apparatus performs a pixelation on the remaining blocks except for the important block, which includes the object to be protected, among the frames of the second image, using the index, and performs a pixelation on the third image It can be generated [S310].

The image verification device may request transmission of an image to the image processing device. The image verification apparatus may receive the first hash data, the second image, and the index time stamped from the image processing apparatus according to the request.

The image verification apparatus may classify each of the frames included in the second image in the same manner as a predetermined number of blocks that divide the first image from which the first hash data is extracted. That is, the frames included in the first image and the second image may be equally divided into blocks of the same number and the same shape, respectively. For example, as described above, it is assumed that each frame is divided into 16 equal square blocks, each of which is a 4 * 4 matrix, to generate a first image. In this case, the image verification apparatus may equally divide the frames of the received second image into 16 equal square blocks of 4 * 4 matrices.

The image verification apparatus may perform pixelation on the remaining blocks except for the important block in which the object to be protected is included in each frame of the second image by using the index and the pixelized process. The image verification apparatus may perform pixelation processing on the remaining blocks other than the important block by using an index indicating the important block. The pixelation method and intensity applied to the important block and the remaining blocks are set to be the same as each other, so that pixelation may be performed.

As such, frames in which the entire area is pixelated may be generated as a third image. Therefore, in the third image, the entire area of each frame of the original image is pixelated in the same manner as the first image from which the first hash data is extracted.

Referring to FIG. 14 again, the image verification apparatus may extract second hash data from the third image [S320].

According to an embodiment, the image verification apparatus may extract frame hash data from each frame included in the third image, and extract the second hash data from the extracted frame hash data again.

Referring to FIG. 14 again, the image verification apparatus may determine the integrity of the second image by comparing the second hash data with the first hash data [S330].

The image verification apparatus may determine the integrity of the second image by comparing the second hash data extracted from the third image with the first hash data. The third image is in the same state as the first image, in the same pixel processing for all blocks of the frames constituting the original image. Accordingly, when the first hash data extracted from the first image and the second hash data extracted from the third image from which the second image is processed have the same value, the second image may be verified as an image without modulation. Also, the recording time can be verified by the time stamp of the first hash data.

According to this, by extracting hash data from an image that has been subjected to pixelation on an area that does not include an object to be protected in the privacy-protected image, the integrity of the privacy-protected image is judged with high reliability. You can.

15 is a flowchart illustrating an operation of an image processing and verification system according to an embodiment of the present disclosure.

15, the image processing apparatus 100 may store the original image photographed by the image photographing unit [S401]. The image processing apparatus 100 may generate an original image by encoding a frame captured by the image capturing unit [S403]. The image processing apparatus 100 may decode the generated original image back into frames in parallel while storing the generated original image in real time [S423].

The image processing apparatus 100 divides each of the frames included in the captured original image into a predetermined number of blocks [S407], and performs pixelation on all of the predetermined number of blocks, respectively A first image may be generated [S409].

The image processing apparatus 100 may extract frame hash data from each frame included in the first image [S411], and extract the first hash data from the extracted frame hash data [S413].

The image processing apparatus 100 may transmit the first hash data to the time stamp server 300 [S415]. The time stamp server 300 may sign the first hash data along with the current time [S417]. The image processing apparatus 100 may receive the first hash data time-stamped from the time stamp server 300 [S419] and store it in the memory [S421].

The image verification apparatus 200 may request transmission of an image to the image processing apparatus 100 [S425]. Upon receiving the request to transmit the image, the image processing apparatus 100 may read the original image from the memory and detect an object included in each frame [S427].

The image processing apparatus 100 divides each of the frames included in the stored original image into a predetermined number of blocks, and generates a second image by performing pixelation on an important block including an object among the predetermined number of blocks It can be done [S429]. The image processing apparatus 100 may store an index indicating the location of an important block and the second image [S431]. The image processing apparatus 100 may transmit the second image, the index and the time-stamped first hash data to the image verification apparatus [S433].

The image verification apparatus 200 may classify each of the frames included in the second image in the same manner as a predetermined number of blocks that classify the first image from which the first hash data is extracted [S435]. The image verifying apparatus 200 generates a third image by performing pixelation on the remaining non-important blocks except for the important block that is pixelated by including an object that is a privacy object among each frame of the second image using the index It can be done [S437].

The image verification apparatus 200 may extract second hash data from the third image [S439]. The image verification apparatus 200 may determine the integrity of the second image by comparing the second hash data with the first hash data [S441].

According to this, when photographing the original image, by extracting hash data from the image in which the entire area of each frame is pixelated, and when the image is requested, an object to be protected is detected to generate a pixelated privacy-protected image, It is possible to efficiently create a privacy-protected image regardless of the performance of the device. In addition, by extracting hash data from an image that has been pixelated for an area that does not contain an object to be protected in a privacy-protected image, it is possible to verify the integrity of the privacy-protected image with high reliability by determining the integrity of the privacy-protected image. have.

In the above, the method for processing and verifying an image according to an embodiment of the present disclosure has been described as being performed in the same procedure as in FIG. 15, but this is for convenience of explanation only and does not depart from the essential concept of the present disclosure, Depending on the implementation method, the execution procedure of each step may be changed, two or more steps may be integrated, or one step may be performed separately into two or more steps.

In the above, even if all the components constituting the embodiments of the present disclosure are described as being combined or operated as one, the present disclosure is not necessarily limited to these embodiments. That is, within the scope of the present disclosure, all of the constituent elements may be selectively combined and operated. In addition, although all of the components may be implemented as one independent hardware, a part or all of the components are selectively combined to perform a combined function of some or all of the functions in one or a plurality of hardware. It may be implemented as a computer program having a. The codes and code segments constituting the computer program may be easily deduced by those skilled in the art of the present disclosure. Such a computer program is stored in a computer readable storage medium (Computer Readable Media) to be read and executed by a computer, thereby implementing an embodiment of the present disclosure. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, and the like.

In addition, the terms "include", "consist" or "have" as described above mean that the corresponding component can be inherent, unless specifically stated otherwise, to exclude other components. It should not be interpreted as being able to further include other components. All terms, including technical or scientific terms, unless otherwise defined, have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Commonly used terms, such as predefined terms, should be interpreted as being consistent with the meaning of the context of the related art, and are not to be interpreted as ideal or excessively formal meanings unless explicitly defined in the present disclosure.

The above description is merely illustrative of the technical spirit of the present disclosure, and those skilled in the art to which the present disclosure pertains may make various modifications and variations without departing from the essential characteristics of the present disclosure. Therefore, the embodiments disclosed in the present disclosure are not intended to limit the technical spirit of the present disclosure, but to explain the scope, and the scope of the technical spirit of the present disclosure is not limited by the embodiments. The scope of protection of the present disclosure should be interpreted by the claims below, and all technical spirits within the scope equivalent thereto should be interpreted as being included in the scope of the present disclosure.

10: image processing and verification system 100: image processing device
110: image recording unit 120: image processing unit
130: memory 140: privacy-protected image generation unit
150: communication unit 200: video verification device
300: time stamp server

Claims (20)

Memory;
Communication department;
An imaging unit;
The original image photographed by the image photographing unit is stored in the memory, each of the frames included in the photographed original image is divided into a predetermined number of blocks, and each of the predetermined number of blocks is respectively An image processor configured to generate a first image by performing pixelation, and extract and store first hash data from the first image; And
Objects subject to privacy protection are respectively detected from frames included in the stored original image, each of the frames included in the stored original image is divided into the predetermined number of blocks, and the object among the predetermined number of blocks is A privacy-protected image generation unit that generates a second image by performing pixelation on an included critical block, and stores an index indicating the location of the important block and the second image in the memory;
Including,
The image processor generates the original image by encoding frames taken by the image capturing unit, and decodes the original image into the frames in parallel while storing the original image in the memory. Create,
The communication unit transmits the second image, the index and the first hash data to an image verification device,
The first hash data is compared with the second hash data in the image verification apparatus, and the integrity of the second image is determined, and the second hash data is generated by using the index in the image verification apparatus. An image processing apparatus extracted from a third image generated by performing pixelation on blocks other than the important block among each frame of the 2 image. delete According to claim 1,
An image processing apparatus in which each frame of the first image and each frame of the second image corresponding to each frame of the first image are divided into blocks having the same number and the same shape. According to claim 1,
The image processing unit extracts frame hash data from each frame included in the first image, and extracts the first hash data from the extracted frame hash data. According to claim 1,
The image processing unit transmits the first hash data to a time stamp server through the communication unit, and receives and stores time-stamped first hash data from the time stamp server. Device. According to claim 1,
An image processing apparatus in which the intensity of pixelation performed for each block of the first image and the intensity of pixelation performed for the important block of the second image are set to be the same. delete delete delete delete Storing the original image photographed by the image photographing unit;
Dividing each of the frames included in the photographed original image into a predetermined number of blocks, and performing pixelation on all of the predetermined number of blocks to generate a first image;
Extracting and storing first hash data from the first image;
Detecting an object that is a privacy protection object from the frames included in the stored original image, respectively;
Dividing each of the frames included in the stored original image into the predetermined number of blocks, and generating a second image by performing pixelation on an important block including the object among the predetermined number of blocks;
Storing the index indicating the location of the important block and the second image;
Transmitting the second image, the index and the first hash data to an image verification device;
Generating a third image by performing pixelation on blocks other than the important block, which includes an object subject to privacy protection, among the frames of the second image using the index;
Extracting second hash data from the third image; And
Determining the integrity of the second image by comparing the second hash data with the first hash data;
Including,
In the step of storing the original image and generating the first image, the original image is generated in parallel while encoding the frames captured by the image capturing unit to generate the original image and storing the original image in memory. And decoding the frames into the frames to generate the first image. delete The method of claim 11,
An image processing and verification method in which each frame of the first image and each frame of the second image corresponding to each frame of the first image are divided into blocks having the same number and the same shape. The method of claim 11,
The step of extracting and storing the first hash data extracts frame hash data from each frame included in the first image, and performs image processing and verification to extract the first hash data from the extracted frame hash data. Way. The method of claim 11,
The step of extracting and storing the first hash data is an image processing and verification method of transmitting the first hash data to a time stamp server and receiving and storing the first hash data time stamped from the time stamp server. The method of claim 11,
An image processing and verification method in which the intensity of pixelation performed for each block of the first image and the intensity of pixelation performed for the important block of the second image are set to be the same. delete delete The method of claim 11,
The step of extracting the second hash data extracts frame hash data from each frame included in the third image, and extracts the second hash data from the extracted frame hash data. The method of claim 11,
An image processing and verification method in which the intensity of pixelation performed on the important block of the second image and the intensity of pixelation performed on blocks other than the important block of the second image are set to be the same.

Images