JP2020010365A - Communication system, image generation method, and communication device - Google Patents

Abstract

To provide a communication system enabling confirmation of content of an image without restoration of the image while protecting privacy, and enabling restoration of the privacy protected image to an original image.SOLUTION: A communication system transmits an image from a first communication device 103 to a second communication device 110. The first communication device 103 includes: a low frequency image generator unit for generating a low frequency image obtained by extracting a low frequency component from a predetermined image; a differential image generator unit for generating, as a differential image, difference between the predetermined image and the low frequency image; and a transmission unit for transmitting the low frequency image and the differential image. The second communication device 110 includes: a receiver unit for receiving the low frequency image and the differential image; and an image composition unit for composing the received low frequency image with the differential image to restore to the predetermined image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a communication system such as a monitoring camera, an image generation method, and a communication device.

  Conventionally, in a surveillance camera system, while masking a part of the area with a mask to protect privacy, it is also possible to restore the part hidden by the mask in an emergency such as a crime so that security and privacy are compatible. (For example, see Patent Document 1).

JP 2008-288744 A

  However, according to the above-described conventional technology, it is not possible to check the contents of the portion hidden by the mask. For example, it is not possible to check even if there is no person in the store, such as contents that are not related to privacy unless the image is restored. I couldn't.

  Also, when processing is performed by superimposing noise on an image when concealing with a mask, an image before concealing with a mask may not be able to be restored, for example, when it is necessary to check a portion concealed by a mask in an emergency.

  Accordingly, in view of the above-described conventional problems, the present invention can confirm the contents of an image without restoring an image while protecting privacy, and restores a privacy-protected image to an original image. It is an object of the present invention to provide a communication system, an image generation method, and a communication device that can perform the communication.

  The present invention is a communication system in which a first communication device, a second communication device, and a plurality of recording devices are connected via a network, wherein the first communication device converts original image data A low-frequency image data from which a low-frequency component is extracted is generated, a difference image indicating a difference between the original image data and the low-frequency image data, and a detail image of the original image data which can be confirmed in detail as compared with the low-frequency image data Generating a plurality of secret sharing data from the difference image data; transmitting each of the low-frequency image data and the plurality of secret sharing data to any of the plurality of recording devices; The apparatus records at least one of the low-frequency image data or the secret sharing data, and the second communication device includes at least one of the plurality of secret sharing data and the low-frequency secret sharing data. Receiving image data from any of the plurality of recording devices, outputting the low-frequency image data when logging in with a first ID, and outputting the secret sharing data when logging in with a second ID. And a communication system for restoring and outputting at least the differential image data using the method.

  Further, the present invention is a communication system in which a first communication device, a second communication device, and a plurality of recording devices are connected via a network, wherein the first communication device includes an original image. Generating low-frequency image data by extracting a low-frequency component from the data, indicating a difference between the original image data and the low-frequency image data, and confirming details of the original image data in comparison with the low-frequency image data. Generating difference image data, generating a plurality of secret sharing data from the difference image data, transmitting each of the low-frequency image data and the plurality of secret sharing data to any of the plurality of recording devices, The recording device records at least one of the low-frequency image data or the secret sharing data, the second communication device, at least one of the plurality of secret sharing data and the Frequency image data, respectively, received from any of the plurality of recording devices, outputs the low-frequency image data during normal reproduction, at the time of restoration, using the secret sharing data, at least restore the difference image data And a communication system for outputting.

  Further, the present invention is an image generating method using a communication system in which a first communication device, a second communication device, and a plurality of recording devices are connected via a network, The communication device generates low-frequency image data by extracting a low-frequency component from the original image data, indicates a difference between the original image data and the low-frequency image data, and compares the original image data with the low-frequency image data. Generating difference image data capable of confirming details of the secret image data, generating a plurality of pieces of secret sharing data from the difference image data, and sending each of the low-frequency image data and the plurality of pieces of secret sharing data to any of the plurality of recording devices. And the plurality of recording devices record at least one of the low-frequency image data or the secret sharing data, and the second communication device communicates a small number of the plurality of secret sharing data. At least one and the low-frequency image data are received from any of the plurality of recording devices, and the low-frequency image data is output during normal reproduction. Provided is an image generation method for restoring and outputting image data.

  The present invention also relates to a communication device connected to a first communication device and a plurality of recording devices via a network, wherein the low-frequency image data is obtained by extracting a low-frequency component from the original image data; And the difference between the low-frequency image data, and at least one of a plurality of secret sharing data generated from difference image data capable of confirming details of the original image data compared to the low-frequency image data, A communication unit that receives from any of the plurality of recording devices, and a processor that outputs the low-frequency image data during normal reproduction, and that restores and outputs the difference image data using the secret shared data during restoration. And a communication device comprising:

  According to the communication system, the image generation method, and the communication device of the present invention, it is possible to confirm the contents of an image without restoring the image while protecting the privacy, and to convert the privacy-protected image into the original image. Can be restored.

FIG. 1 is a schematic diagram illustrating an overall configuration of a surveillance camera system according to a first embodiment of the present invention. 1 is a block diagram of a first communication device according to a first embodiment of the present invention. Flow chart of processing in the first communication device of the first embodiment of the present invention Explanatory diagram of the spatial frequency components included in the original image, low-frequency image, and difference image Illustration of smoothing filter Illustration of block averaging Illustration of low frequency image Explanatory drawing of the conversion method of the difference value of the difference image Block diagram of conventional receiver FIG. 3 is a block diagram at the time of reproduction of the second communication device according to the first embodiment of the present invention. FIG. 4 is a flowchart of a reproduction process in the second communication device according to the first embodiment of the present invention. FIG. 4 is a block diagram at the time of restoration of the second communication device according to the first embodiment of the present invention. FIG. 4 is a flowchart of a restoration process in the second communication device according to the first embodiment of the present invention. Illustration of creating a restored image by image synthesis Illustration of a method for converting a composite value of a restored image Block diagram of a first communication device according to a second embodiment of the present invention Flow chart of processing in the first communication device according to the second embodiment of the present invention Illustration of mask area setting method Illustration of low-frequency image and difference image Illustration of area size setting Illustration of block size and image quality of low frequency image FIG. 4 is a schematic diagram illustrating an overall configuration of a surveillance camera system according to a third embodiment of the present invention. Block diagram of a first communication device according to a third embodiment of the present invention FIG. 9 is a flowchart of a process in the first communication device according to the third embodiment of the present invention. Illustration of the threshold secret sharing scheme Block diagram of a second communication device according to a third embodiment of the present invention FIG. 9 is a flowchart of a process in the second communication device according to the third embodiment of the present invention. FIG. 4 is a schematic diagram illustrating an overall configuration of a surveillance camera system according to a fourth embodiment of the present invention. Block diagram of a first communication device according to a fourth embodiment of the present invention FIG. 9 is a flowchart of a process in the first communication device according to the fourth embodiment of the present invention. Illustration of the configuration of the join file Illustration of the configuration of a combined file when secret sharing is used for concealment FIG. 14 is a block diagram at the time of reproduction of the second communication device according to the fourth embodiment of the present invention. 4 is a flowchart of a reproduction process in a second communication device according to a fourth embodiment of the present invention. FIG. 14 is a block diagram at the time of restoration of the second communication device according to the fourth embodiment of the present invention. FIG. 11 is a flowchart of a restoration process in the second communication device according to the fourth embodiment of the present invention. Illustration of the configuration of a combined file using encrypted data and a decryption key Illustrative diagram of the configuration of a combined file when data is divided

  A communication system according to the present invention is a communication system for transmitting an image from a first communication device to a second communication device, wherein the first communication device converts a low-frequency image obtained by extracting a low-frequency component from a predetermined image. A low-frequency image generating unit that generates a difference image between the predetermined image and the low-frequency image as a differential image; and a transmitting unit that transmits the low-frequency image and the differential image, The communication device includes a receiving unit that receives the low-frequency image and the difference image, and an image combining unit that combines the received low-frequency image and the difference image to restore a predetermined image.

  Accordingly, the content of the image can be confirmed without protecting the privacy and without restoring the image, and the privacy-protected image can be restored to the original image. Also, the degree of privacy protection can be freely adjusted.

  In addition, the first communication device further includes an image encoding unit that encodes at least a low-frequency image, and the second communication device further includes an image decoding unit that decodes at least the encoded low-frequency image. And decodes the low-frequency image and combines it with the difference image.

  Thus, the data size of the restored image can be reduced without deterioration.

  The low-frequency image generation unit generates a low-frequency image by averaging pixels in a predetermined area.

  Thus, a low-frequency image can be easily generated.

  Further, the low-frequency image generation unit generates a low-frequency image by extracting a low-frequency component from only a part of the predetermined image, and the difference image generation unit generates a difference image having the same size as the predetermined image.

  As a result, it is possible to restore the original image without separately storing the positions of some areas while protecting only some areas related to privacy.

  In addition, the first communication device further includes an image dividing unit that generates a plurality of data concealed based on the difference image, a low-frequency image, and one of the plurality of data. A shared data generation unit that generates the shared data by combining the low frequency image with the low-frequency image, and the transmission unit transmits the shared data.

  Thus, an image in which privacy is protected can be confirmed using the shared data, and the original image can be confirmed by collecting the shared data as needed.

  Further, the image generation method of the present invention includes a step of generating a low-frequency image by extracting a low-frequency component from a predetermined image; a step of generating a difference between the predetermined image and the low-frequency image as a difference image; Combining the image and the difference image to restore the image to a predetermined image.

  Accordingly, the content of the image can be confirmed without protecting the privacy and without restoring the image, and the privacy-protected image can be restored to the original image.

  Further, the communication device of the present invention is a communication device for transmitting an image to another communication device, wherein the low-frequency image generation unit that generates a low-frequency image by extracting a low-frequency component from a predetermined image, A low frequency image and a low frequency image as a differential image, and a transmitting unit transmitting the low frequency image and the differential image.

  Accordingly, it is possible to confirm the contents of the image without restoring the image while protecting privacy, and to generate an image that can restore the privacy-protected image to the original image.

  Embodiments of a communication system, an image generation method, and a communication device according to the present invention will be described with reference to the drawings. In the present embodiment, a surveillance camera system will be described as an example, but the present invention can be applied to other systems as appropriate.

(Example 1)
The configuration of the monitoring camera system will be described with reference to FIG. FIG. 1 is a schematic diagram illustrating an overall configuration of a surveillance camera system according to Embodiment 1 of the present invention.

  In FIG. 1, reference numeral 101 denotes an imaging system. In the imaging system 101, reference numeral 102 denotes a monitoring camera, reference numeral 103 denotes a first communication device, reference numeral 104 denotes a recording device, and these are connected by an internal network (LAN or the like).

  Reference numeral 105 denotes a reproduction system. In the reproduction system 105, reference numeral 106 denotes a receiving device, 107 denotes a recording device, and 108 denotes a monitor, which are connected to an internal network different from the imaging system 101.

  Reference numeral 109 denotes a restoration system. In the restoration system 109, 110 is a second communication device, 111 is a recording device, 112 is a monitor, and the restoration system 109 is also connected to an internal network different from the imaging system 101. .

  Further, the imaging system 101, the reproduction system 105, and the restoration system 109 are externally connected via a network 113.

  The imaging system 101 creates a privacy-protected image and data for restoration from an image acquired by the surveillance camera 102, and outputs the privacy-protected image to the reproduction system 105 or the restoration system 105 via the network 113. Send to 109. Further, data for restoration is recorded in the recording device 104 connected internally.

  Here, since the imaging system 101 is built in a specific internal network, access from outside such as the reproduction system 105 can be suppressed via the network 113, and the image acquired by the monitoring camera 102 can be suppressed. Privacy can be protected.

  Note that, even if the privacy is secured outside the imaging system 101, data for restoration may be recorded in a place other than the recording device 104.

  The reproduction system 105 receives the image whose privacy is protected by the receiving device 106 and reproduces the image on the monitor 108. Since the privacy of this image data is protected, not all details of the image can be confirmed at the time of reproduction. Also, the privacy protected image is recorded in the recording device 107 connected internally as necessary. Note that this system is a system used by general users that does not require privacy information.

  The restoration system 109 receives the privacy-protected image by the second communication device 110 and reproduces the image on the monitor 112. As with the image data, not all image details can be confirmed, as described above. Also, the privacy-protected image is recorded in the recording device 111 connected internally.

  When it is necessary to check the details of the image, the second communication device 110 receives data for restoration from the imaging system 101 and outputs the privacy-protected image recorded on the recording device 111 for restoration. The original image is restored by using the data of. Since the privacy-protected portion of this image is restored, all details of the original image can be confirmed. This system is used by an administrator because the details of an image including privacy information can be checked when necessary.

  In this embodiment, the reproduction system 105 and the restoration system 109 are described separately. However, the reproduction system 105 and the restoration system 109 may be implemented by the same system. In this case, for example, use of the restoration system 109 is performed for each user using a login ID or the like. You only need to set the authority.

  Next, the first communication device 103 will be described in detail with reference to FIGS. FIG. 2 is a block diagram of a first communication device according to the first embodiment of the present invention, and FIG. 3 is a flowchart of a process in the first communication device of the first embodiment of the present invention.

  2, reference numeral 201 denotes a low-frequency image generation unit, 202 denotes a difference image generation unit, 203 denotes an image coding unit, 204 denotes a transmission unit, 205 denotes a frame image, 206 denotes low-frequency image encoded data, and 207 denotes a difference image code. Data.

  Hereinafter, the flow of processing will be described with reference to the flowchart of FIG.

  First, the surveillance camera 102 forms a frame image 205 as digital data by forming an optical signal around the camera on an image sensor using a lens and converting the image into an electric signal (S301).

  Then, the low-frequency image generation unit 201 creates a low-frequency image from the frame image 205 (S302).

  Further, the difference image generation unit 202 creates a difference image between the frame image 205 acquired by the monitoring camera 102 and the low frequency image created by the low frequency image generation unit 201 (S303).

  The image encoding unit 203 encodes the low-frequency image created by the low-frequency image generation unit 201 and the difference image created by the difference image generation unit 202 (S304).

  The transmission unit 204 transmits the encoded low-frequency image data 206 encoded by the image encoding unit 203 to the reproduction system 105 and the restoration system 109 via the network 113 (S305). The difference image encoded data 207 encoded by the image encoding unit 203 is recorded on the recording device 104 (S306).

  Here, before describing each component block, a low-frequency image and a difference image will be described. FIG. 4 is an explanatory diagram of the spatial frequency components in the original image, the low-frequency image, and the difference image.

  As shown in FIG. 4, when a certain image (original image) is transformed and the spatial frequency is plotted on the horizontal axis and the vertical axis is amplitude, the low-frequency image is a spatial frequency component contained in the original image. Is an image from which a component having a low spatial frequency is extracted.

  On the other hand, the difference image is an image including the remaining frequency components excluding the spatial frequency components included in the low-frequency image from the spatial frequency components of the original image.

  Since the low-frequency image is obtained by extracting the low-frequency component as described above, it does not include fine parts of the image such as the outline represented by the component having a high spatial frequency, and the focus is blurred or mosaiced. This is an image whose details cannot be confirmed.

  Therefore, the low-frequency image is an image in which privacy can be protected because the rough situation of the place where the image was captured by the camera can be grasped, but the details of the image cannot be confirmed.

  On the other hand, since the difference image has a high spatial frequency, the difference image is an image in which a detailed situation such as an outline can be confirmed. Therefore, the image can be used to specify the privacy information of the person, the vehicle type, the vehicle, and the like from the face, the vehicle, the outline of the license plate, and the like.

  Hereinafter, each component block will be specifically described.

  The low-frequency image generation unit 201 creates a low-frequency image from a frame image. A low-frequency image can be created by temporarily converting an image into a spatial frequency domain and extracting a component having a low spatial frequency. For example, a low-frequency image can be created by performing frequency transformation such as Fourier transformation, discrete cosine transformation, and wavelet transformation, extracting a component having a low spatial frequency, and performing inverse transformation.

  Note that the conversion to the spatial frequency domain requires a CPU having a certain processing capacity because of the heavy load on the CPU, but can be more easily performed.

  For example, if filter processing or block averaging is performed on a real space, the details of an image cannot be understood, and only a rough situation can be understood.

  In other words, averaging reduces the high spatial frequency components of the image, so it is possible to create a low-frequency image without understanding the details of the image without performing conversion to spatial frequency, and an image that can sufficiently protect privacy Can be created.

  Therefore, a low-frequency image can be created without conversion to a spatial frequency, and the load can be reduced.

  Hereinafter, the respective creation methods will be described with reference to FIGS. FIG. 5A is an explanatory diagram of a smoothing filter based on a moving average, and FIG. 5B is an explanatory diagram of a smoothing filter based on a weighted average.

  Specifically, when a 9-pixel smoothing filter as shown in FIG. 5 is used, the sum of the target pixel and the eight surrounding pixels is divided around the pixel to be smoothed, thereby obtaining the target pixel. Is performed, and this process is performed on all the pixels.

  In this case, in the case of FIG. 5A, the average of the target pixel and the surrounding eight pixels is simply taken as the pixel after smoothing, but in the case of FIG. After performing weighting in advance so that the target pixel has a greater influence on the pixel, the pixel after smoothing is calculated.

  Note that in the example of FIG. 5, smoothing is performed using nine pixels, but the number of pixels used is not limited to this. By changing the number of neighboring pixels and the coefficient value used for smoothing, the degree of blur can be adjusted. If the image data is composed of a plurality of planes such as RGB, these processes must be performed for each plane (R, G, B).

  FIG. 6 is an explanatory diagram of block averaging. In the block averaging, unlike the above-described smoothing filter, the image is divided into 4 × 4 pixel blocks and the value of each pixel is replaced with the average value of the blocks, instead of performing the processing on each pixel. A low-frequency image averaged for each image can be created.

  In the case of a block of 4 × 4 pixels at the upper left in FIG. 6, since the average value of 16 pixels included in the block is 117, the value of each pixel is replaced with 117. By repeating the same processing for each block, a low-frequency image based on block averaging can be created.

  Further, since image data has a high correlation with neighboring pixels and values of neighboring pixels are often close, it is not necessary to calculate the average value of all pixels included in a block when performing block averaging. The average value of the pixels of the section may be used as the average value of the block. In this way, a process with a lighter load can be realized.

  Also, by changing the size of the block, the intensity of the mosaic can be adjusted. The greater the block size, the greater the degree of privacy protection. When a low-frequency image is created using block averaging, if the image data is composed of a plurality of planes, it is necessary to perform these processes on each plane.

  FIG. 7 shows an example of a low-frequency image created using filter processing or block averaging. FIG. 7A is an exemplary view of a low-frequency image using a smoothing filter, and FIG. 7B is an exemplary view of a low-frequency image using block averaging. As described above, even if the filtering process or the block averaging is used in the real space, a privacy-protected low-frequency image can be created.

  The difference image generation unit 202 creates a difference image between the frame image and the low frequency image. This difference image is data including a component having a high spatial frequency for understanding details of the image, and is an image related to privacy.

  Then, the difference image becomes data for restoring the original frame image (image including privacy) by combining with the low-frequency image.

  The difference image is obtained by subtracting the value of the corresponding pixel between the frame image and the low-frequency image. Here, when the frame image acquired by the monitoring camera 102 is 8-bit data, the value of each pixel is in the range of 0 to 255. On the other hand, the value of the difference image obtained by subtracting the pixel values becomes 9-bit data of −255 to 255, and the range in which the data value can be obtained is increased.

  Therefore, the data is converted into 8-bit data using the conversion shown in FIG. In FIG. 8, the horizontal axis represents the difference value (value of the difference image) between the frame image and the low-frequency image, and the vertical axis represents the value of the difference image after the conversion. As shown in FIG. 8, an offset of 128 is added to the value of the difference image, and a value of 0 or less is clipped to 0, and a value of 255 or more is clipped to 255. Converted to bit data.

  As a method of converting to 8-bit data, in addition to such clipping, a method of linearly or non-linearly quantizing the entire data range may be used. Further, in order to increase the S / N ratio of the restored image when restoring the frame image, the difference data may be retained as 9-bit data.

  The image encoding unit 203 encodes the low-frequency image and the difference image. In the present embodiment, a moving image captured by a surveillance camera is targeted. Therefore, as a method of image encoding, MPEG or H.264, which is a compression standard, is used. H.264 / AVC, and the like. Also, JPEG or JPEG2000 can be applied to still images.

  The image encoding method is preferably a standard method from the viewpoints of processing speed, encoding efficiency, and versatility, but is not limited to the standard method. If so, a unique coding method may be used.

  In this embodiment, both the low-frequency image and the difference image are encoded. However, trade-offs such as a processing speed for encoding, a writing speed to the storage device, a communication speed in the network, and a cost for the storage device are performed. Accordingly, a configuration in which only the low-frequency image is encoded and a configuration in which neither the low-frequency image nor the difference image are encoded are conceivable. If not encoded, the image data is in an uncompressed format. In this case, the image data in the non-compressed format must be displayed and reproduced by a reproduction system or a restoration system in the surveillance camera system.

  In the present invention, a low-frequency image is used as an image for protecting privacy, and a difference image is used as data for restoration. As described with reference to FIG. 4, the low-frequency image is an image obtained by extracting a component having a low spatial frequency from the spatial frequency component included in the original image, and the difference image is a spatial frequency component included in the low-frequency image. Is an image including the remaining frequency components excluding.

  However, for the purpose of protecting privacy, it is not necessary to completely separate the low frequency component and the high frequency component from a certain spatial frequency as shown in FIG. If most of the high-frequency components are not included, the outline cannot be determined, so that privacy can be protected only by using filter processing and block averaging as described above.

  On the other hand, as a method of creating an image in which privacy is protected, other methods such as superimposing noise on a frame image may be considered in addition to the low-frequency image. Even when noise is superimposed, privacy is protected in the noise image.

  However, when such a method is used, the frequency characteristic itself changes in the noise image or the difference image between the original image and the noise image. That is, since noise includes many components having a high spatial frequency, the amplitude of the high-frequency component of the noise image or the difference image greatly differs from that of the original image.

  Here, consider the image encoding performed in a later stage. Many image coding methods, such as a standard compression method, perform frequency conversion such as discrete cosine transform and wavelet transform, and perform different processing for each frequency. Specifically, since human visual characteristics are insensitive to high frequencies, in order to reduce the amount of information, a component having a high spatial frequency is coarsely quantized as compared with a component having a low spatial frequency. Therefore, if a component having a high spatial frequency such as noise is superimposed and encoded, a quantization error becomes large, and it becomes difficult to remove the superimposed noise properly at the time of restoration.

  In addition, when a component having a high spatial frequency, such as noise, which is not included in a normal natural image, is superimposed on an image, it is necessary to retain a high-frequency component which has not originally occurred as encoded data. Therefore, there is also a problem that the coding efficiency is significantly reduced.

  Therefore, as described in the present invention, these problems can be solved by separating the image into two images in view of the spatial frequency component of the original image.

  In other words, by separating the low-frequency image and the difference image and encoding each image individually, the image quality of the restored image synthesized after decoding is not significantly deteriorated as compared with the original image. Also, the sum of the data amounts of the individually encoded data is not so large as compared with the case where the original image is encoded.

  Therefore, by adopting such a configuration, it is possible to confirm the outline of the image without restoring the image while protecting the privacy with almost the same data amount as before, and in an emergency, A surveillance camera system capable of restoring details of an image can be realized.

  Next, the receiving device 106 will be described with reference to FIG. FIG. 9 shows a block diagram of a conventional receiving apparatus.

  In FIG. 9, reference numeral 901 denotes a receiving unit, 902 denotes an image decoding unit, 206 denotes low-frequency image encoded data, and 903 denotes low-frequency image data. The receiving device 106 is for receiving and reproducing a low-frequency image in which privacy is protected. The receiving unit 901 receives the low-frequency image encoded data 206. If reproduction is necessary later, the low-frequency image encoded data 206 is recorded in the recording device 111 connected internally. The image decoding unit 902 decodes the low frequency image encoded data 206. Then, the monitor 112 displays and reproduces the obtained low-frequency image data 903. Since the low-frequency image encoded data 206 is data encoded by the image encoding unit 203 of the first communication device 103, the image decoding unit 902 needs to be able to decode this data. If the image encoding unit 203 has a standard compression format, the image decoding unit 902 only needs to be able to decode encoded data of the standard system. If the image encoding unit 203 has a unique encoding method, the image decoding unit 902 also has a corresponding unique decoding method.

  Next, the processing of the second communication device 110 will be described in detail. The second communication device can perform the same reproduction processing as that of the receiving device 106 and a restoration process that cannot be performed by the receiving device 106.

  First, the reproduction process of the second communication device 110 will be described in detail with reference to FIGS. FIG. 10 is a block diagram at the time of reproduction of the second communication device according to the first embodiment of the present invention, and FIG. 11 is a flowchart of a reproduction process at the second communication device of the first embodiment of the present invention.

  10, reference numeral 1001 denotes a receiving unit; 902, an image decoding unit; 1002, an image synthesizing unit for synthesizing a decoded low-frequency image and a difference image; 206, low-frequency image encoded data; and 903, low-frequency image data .

  Hereinafter, the flow of processing will be described with reference to the flowchart of FIG.

  First, the receiving unit 1001 receives the encoded low-frequency image data 206 from the imaging system 101 via the network 113 (S1101).

  Next, the receiving unit 1001 records the received low-frequency image encoded data 206 in the recording device 111 (S1102). At this time, information for specifying the difference image encoded data, which is necessary for restoring the image, is also recorded. Specifically, the file name of the difference image encoded data corresponding to the received low frequency image encoded data and the recorded location (file path or the like) are managed and recorded using a table or the like. .

  Then, the image decoding unit 902 decodes the low frequency image encoded data 206 and creates one frame of low frequency image data 903 (S1103).

  Finally, the monitor 112 displays the low-frequency image data 903, which is the created frame image (S1104).

  At the time of normal reproduction, only low-frequency image data is decoded. Therefore, displaying only an image in which privacy is protected does not mean that all details of the image can be confirmed. Also, the low-frequency image encoded data 206 is recorded in the recording device 111 because it is also required at the time of restoration.

  Next, the restoration process of the second communication device 110 will be described in detail with reference to FIGS. FIG. 12 is a block diagram at the time of restoration of the second communication device in the first embodiment of the present invention, and FIG. 13 is a flowchart of restoration processing in the second communication device of the first embodiment of the present invention.

  12, reference numeral 1001 denotes a receiving unit; 902, an image decoding unit; 1002, an image synthesizing unit for synthesizing a decoded low-frequency image and a differential image; 206, low-frequency image encoded data; 207, differential image encoded data; Reference numeral 1201 denotes restored image data.

  Hereinafter, the processing flow will be described with reference to the flowchart of FIG.

  First, the receiving unit 1001 acquires the low-frequency image encoded data 206 recorded in the recording device 111 (S1301).

  Next, the receiving unit 1001 receives the encoded difference image data 207 from the imaging system 101 via the network 113 (S1302). Specifically, it receives the difference image encoded data corresponding to the low frequency image encoded data 206 to be restored. The corresponding differential image encoded data can be specified from the file name and the location (file path or the like) managed and recorded in the table or the like described in the reproduction process.

  Then, the image decoding unit 902 decodes the encoded low-frequency image data 206 and the encoded differential image data 207, and creates data of a low-frequency image and a differential image for one frame (S1303).

  The image combining unit 1002 combines the low-frequency image and the difference image to create a restored image 1201 (S1304).

  Finally, the monitor 112 displays the restored image data 1201 which is the created frame image (S1305).

  Hereinafter, the image combining unit 1002 will be specifically described.

  The image combining unit 1002 combines the decoded low-frequency image and the difference image to create a restored image. FIG. 14 is an explanatory diagram of creating a restored image by image synthesis. The difference image has an offset of 128 added to the actual difference value in the difference image generation unit 202 of the first communication device 103. Therefore, a composite value of the restored image is created by subtracting the offset from the pixel value of the difference image, changing the offset to a value of -128 to 127, and adding it to the value of the corresponding low-frequency image.

  However, since the low-frequency image and the difference image are separately encoded, the combined value of the restored image added may not be within the range of 0 to 255 due to a quantization error. Therefore, the value of the restored image is converted into 8-bit data using the conversion as shown in FIG. In FIG. 15, the horizontal axis represents the value obtained by combining the low-frequency image and the difference image for restoration, and the vertical axis represents the value of the restored image after the combined value has been converted. The value of 0 or less is clipped to 0, and the value of 255 or more is clipped to 255, thereby converting the value of the restored image into 8-bit data.

  By adopting the configuration described above, it is possible to confirm the outline of the image without restoring the image while protecting the privacy with almost the same data amount as before, and in an emergency, A surveillance camera system capable of restoring details of an image can be realized. Further, since the system can be configured using the standard image encoding unit and image decoding unit, a simple, inexpensive, and highly versatile surveillance camera system can be realized.

  Although the present embodiment has been described with respect to an image input from the surveillance camera 102, the present invention is also applicable to image data acquired from a source other than the surveillance camera 102 or image data stored in advance. It goes without saying that it is possible.

  Further, the image in the present embodiment is not limited to a moving image or a still image, and the frame image received from the monitoring camera 102 does not need to be digital data, and may be analog data. In this case, the data is changed to digital data and divided into a low-frequency image and a difference image.

  In the present embodiment, the imaging system 101 is configured to connect the monitoring camera 102, the first communication device 103, and the recording device 104 via an internal network, but some or all of them are implemented in the monitoring camera 102. However, it goes without saying that the present invention is applicable.

  Further, in the present embodiment, the restoration system 109 is configured to connect the second communication device 110, the recording device 111, and the monitor 112 via an internal network. However, these may be configured by a personal computer and a monitor.

  In addition, the low-frequency component of the spatial frequency is used as the image in which the privacy is protected, but it is not necessary to include all the low-frequency components.

  Further, in this embodiment, the low frequency component of the spatial frequency is an image in which privacy is protected. However, a part of the spatial frequency component (for example, the frequency is divided into three of low frequency, middle frequency, and high frequency) If the details (details) of the original frame image are not visible even when the intermediate frequency component is extracted, the intermediate frequency component may be an image in which privacy is protected.

(Example 2)
Hereinafter, a second embodiment of the present invention will be described with reference to the drawings. The components having the same functions as those in the first embodiment are denoted by the same reference numerals, and description thereof will be omitted.

  FIG. 16 is a block diagram of a first communication device according to the second embodiment of the present invention, and FIG. 17 is a flowchart of processing in the first communication device of the second embodiment of the present invention.

  In FIG. 16, reference numeral 1601 denotes a mask area setting unit that sets an area for which a low-frequency image is to be created as a mask area; 1602, an area size setting unit that sets the size of a predetermined area used for low-frequency image generation; A low-frequency image generation unit that creates a low-frequency image using an average of pixels in the region, 1603 is coded low-frequency image data, and 1604 is coded difference image data.

  Hereinafter, the flow of processing will be described with reference to the flowchart of FIG.

  First, the monitoring camera 102 obtains a frame image 205 as digital data by forming an optical signal around the camera on an image sensor using a lens and converting the image into an electric signal (S1701).

  Next, the mask area setting unit 1601 sets a mask area as an area for protecting privacy (S1702). Since only the mask area is converted into a low-frequency image, image details of the mask area cannot be confirmed.

  Furthermore, the area size setting unit 1602 sets the size of a predetermined area (area size) used when creating a low-frequency image (S1703).

  Then, the low-frequency image generation unit 201 'creates a low-frequency image from the frame image using the set area size for the set mask area (S1704). The low-frequency image is created using block averaging, which calculates an average value for each pixel within the set area size.

  Further, the difference image generation unit 202 creates a difference image between the frame image acquired by the monitoring camera 102 and the low frequency image created by the low frequency image generation unit 201 '(S1705).

  The image encoding unit 203 encodes the low-frequency image created by the low-frequency image generation unit 201 'and the difference image created by the difference image generation unit 202 (S1706).

  The transmission unit 204 transmits the low-frequency image encoded data 1603 encoded by the image encoding unit 203 to the reproduction system 105 and the restoration system 109 via the network 113 (S1707). The difference image encoded data 1604 encoded by the image encoding unit 203 is recorded in the recording device 104 (S1708).

  Hereinafter, each component block will be specifically described.

  The low-frequency image generation unit 201 'creates a low-frequency image using an average of pixels in a predetermined area. Although the basic function is the same as that of the low-frequency image generation unit 201 described in the first embodiment when using block averaging, the low-frequency image generation unit 201 ′ in the present embodiment uses a mask area setting unit 1601 Only the set area is converted to a low-frequency image. Further, a function for calculating a block average with the block size set by the area size setting unit 1602 is added. As will be described in detail later, the low-frequency image created by the low-frequency image generation unit 201 'becomes an image as shown in FIG.

  The mask area setting unit 1601 sets a mask area as an area for protecting privacy so that details of an image area related to privacy cannot be confirmed. The low-frequency image generation unit converts only the set mask area into a low-frequency image, and creates a low-frequency image without converting other areas as captured frame images. FIG. 18 is an explanatory diagram of mask area setting.

  In FIG. 18, reference numeral 1801 denotes a general frame image, and reference numerals 1802 and 1803 denote mask areas. FIG. 18 shows an example in which the detected face area is set as a mask area. A mask area 1802 is set for each pixel, and a mask area 1803 is set for each block. Face, person, license plate, moving object, and the like can be considered as targets of the mask area related to privacy.

  As a method for detecting the target region, a method using object detection by machine learning, a method using object tracking, a method using optical flow or background difference, a method combining these, and the like are conceivable. With these methods, a mask area can be dynamically set for each frame.

  In addition, a fixed surveillance camera often knows in advance a position where an object related to privacy such as a person is likely to be reflected. In such a case, a method in which the user manually sets a mask area in advance may be considered. Further, the number of mask regions is not limited to one. If there are a plurality of privacy-related areas in the image, a plurality of mask areas can be set. FIG. 19 is an exemplary diagram of a low-frequency image and a difference image.

  FIG. 19A is an exemplary diagram of a low-frequency image in a case where a person's face is set as a mask area. FIG. 19B is an illustration of the difference image. In the low-frequency image, areas other than the mask area are not converted as captured frame images. Therefore, when a difference image is created, no difference occurs between the low-frequency image and the frame image outside the mask area.

  That is, an area other than the mask area in the difference image is an image of a fixed level of the pixel value 128 obtained by adding the offset 128 to the difference value 0. Therefore, at the time of restoration, the original image can be restored only by synthesizing the low-frequency image and the difference image as they are, without storing the set mask area position information or the number information. That is, by creating a difference image as in the present embodiment, even when a mask area is set, the difference image can be handled in the same manner as in the first embodiment in which the entire frame image is converted into a low-frequency image. it can.

  The region size setting unit 1602 sets the size of a predetermined region for calculating the average of pixels when creating a low-frequency image. FIG. 20 is an explanatory diagram of the setting of the predetermined area size. FIG. 20A shows an example in which the predetermined area size is set to 4 × 4 pixels. In this case, the value of each pixel in the block of 4 × 4 pixels is set as the average value of the pixels in the block. Create a low-frequency image. Similarly, FIG. 20B is an exemplary diagram in the case of 6 × 6 pixels, and FIG. 20C is an exemplary diagram in the case of 8 × 8 pixels.

  Increasing the size of a block is equivalent to converting the image into a low-resolution image, so that the information amount of the image decreases. Accordingly, when the degree of privacy protection is desired to be increased, the size of the predetermined area is increased, and when the degree of privacy protection is desired to be reduced, the size of the predetermined area is reduced, so that the privacy protection strength of the low-frequency image can be freely adjusted. . Even if the size of the block is adjusted arbitrarily, of the spatial frequency components included in the frame image, the components with low spatial frequencies are separated into low-frequency images, and the components with high spatial frequencies are separated into difference images. There is no. Therefore, the coding efficiency of the image coding unit performed in the subsequent stage does not significantly decrease.

  In addition, an object to be protected, such as a face, is acquired as an image enlarged as the camera approaches the camera. In the enlarged image, even if a low-frequency image is created with the same block size, the effect of privacy protection is not the same. FIG. 21 is an exemplary diagram of the block size and the image quality of the low-frequency image when the low-frequency image is created using the block average.

  FIGS. 21A and 21B show an example in which a low-frequency image is created in a block of 8 × 8 pixels, and FIG. 21C shows an example in which a low-frequency image is created in a block of 16 × 16 pixels.

  When comparing FIG. 21 (a) and FIG. 21 (b), since the face region is enlarged in FIG. 21 (b), if a low frequency image is created with the same block size, the effect of privacy protection is lower. You can see that. In the case of such an enlarged image, it is more effective to create a low-frequency image by increasing the block size as shown in FIG.

  Therefore, privacy is protected by linking the size of the mask area with the size of the predetermined area so that the size of the predetermined area is large when the mask area is large and small when the mask area is small. In addition, an image whose outline can be easily checked can be obtained. When dynamically setting the mask area using face detection, etc., if the size of the block is also dynamically set according to the size of the detected mask area, the privacy of the image can be roughly protected while protecting the privacy. It becomes possible to create an appropriate low-frequency image in which the situation can be easily checked.

  Note that, in the present embodiment, the case where block averaging is used in the low-frequency image generation unit has been described as an example, but a smoothing filter may be used. When a smoothing filter is used in the low-frequency image generation unit, a similar effect can be realized by providing a smoothing filter setting unit instead of the area size setting unit 1602.

  Specifically, the smoothing filter setting unit sets the size of the neighboring pixels used for the smoothing filter and the filter coefficient, and if it is desired to increase the degree of privacy protection, increase the size of the neighboring pixels used for smoothing. If it is desired to decrease the privacy, the size of the neighboring pixels is reduced, so that the privacy protection strength of the low-frequency image can be freely adjusted. By dynamically setting the size of neighboring pixels used for smoothing in conjunction with the size of the mask area, it is possible to confirm the general situation of the shooting location while protecting privacy, as in the case of using block averaging. An easy and appropriate low-frequency image can be created.

  In this embodiment, the mask area is determined by a person detection or the like, and the area size for creating a low-frequency image (averaging pixels) is determined in accordance with the mask area. .

  Specifically, the size of the person is detected, the mask area is determined, and the area size that can protect privacy is determined according to the size of the person. It is possible to determine the size.

  With the configuration as described above, it is possible to prevent the image details from being confirmed only in the area related to privacy. Further, there is no need to store the number and position information of privacy-related areas, and restoration can be performed with a simple configuration. Therefore, a surveillance camera that is more convenient for the user can be provided. In addition, since the size of the predetermined area for creating the low-frequency image can be changed arbitrarily, the degree of privacy protection can be freely adjusted while minimizing the decrease in encoding efficiency, and the use of the camera and the installation position can be freely adjusted. It is possible to provide a surveillance camera system whose degree is significantly improved.

  In the second embodiment, the first communication device 103 is as described above, but the other configuration is the same as that of the first embodiment.

  Note that, as in the present embodiment, the mask area and the area size can be changed by focusing on the spatial frequency of the image, dividing the image into low-frequency images and differential images while maintaining the spatial frequency components of the image. This is because it is transmitted to the unit 203.

  In general, when creating an image in which privacy is protected, an increase in the amount of data and a deterioration in image quality pose a problem. Specifically, it is realized by exchanging or converting variable-length code data in the image encoding unit in code units, block units, or the like. Therefore, the degree of freedom of the replacement method and the conversion method is restricted by the configuration of the image encoding unit, and privacy protection in which a mask area and an area size are freely set cannot be realized.

  Further, in order to create a privacy protected image inside the image encoding unit, an image encoding unit having a special configuration is required instead of the standard system. Therefore, the image decoding unit for image restoration also needs a special configuration, and the target devices are limited.

  On the other hand, if an image in which privacy is protected can be created in front of the image encoding unit, the image encoding unit only needs to use a conventionally used image encoding unit and can be easily realized.

  However, as described in the first embodiment, if noise is used to protect privacy, the spatial frequency component of the original image changes, and the original frame image cannot be restored or the data amount increases. Or

  Therefore, in this embodiment, the mask region and the region size can be freely set, and the image is divided into a low-frequency image and a difference image so that the original frame image can be restored.

(Example 3)
FIG. 22 is a schematic diagram illustrating the overall configuration of the surveillance camera system according to the third embodiment of the present invention. The components having the same functions as those of the first and second embodiments are denoted by the same reference numerals, and description thereof is omitted.

  In FIG. 22, reference numeral 101 'denotes an imaging system. In the imaging system 101', reference numeral 102 denotes a monitoring camera, reference numeral 103 '' denotes a first communication device, and these are connected via an internal network.

  Reference numeral 105 'denotes a reproduction system. In the reproduction system 105', reference numeral 106 'denotes a receiving device, reference numeral 108 denotes a monitor, and these are also connected by an internal network different from the imaging system 101'.

  Reference numeral 109 'denotes a restoration system. In the restoration system 109', reference numeral 110 'denotes a second communication device, which is connected to the monitor 112 via a different internal network from the imaging system 101'.

  Reference numerals 2201, 2202, and 2203 denote recording devices installed outside the network such as a cloud. The imaging system 101 ′, the reproduction system 105 ′, and the restoration system 109 ′ perform recording devices 2201, 2202, and 2203 via the network 113. And is externally connected.

  Here, the first communication device 103 '' in the imaging system 101 'includes means for concealing data. In addition, the second communication device 110 'in the restoration system 109' has means for restoring concealed data. By providing such means, privacy-protected low-frequency image data and differential image data for restoration, rather than a recording device of the internal network, to a recording device connected to the outside of the network such as a cloud, Be able to record safely.

  Next, the first communication device 103 '' will be described in detail with reference to FIGS. FIG. 23 is a block diagram of a first communication device according to the third embodiment of the present invention, and FIG. 24 is a flowchart of processing in the first communication device of the third embodiment of the present invention.

  In FIG. 23, 201 is a low-frequency image generation unit, 202 is a difference image generation unit, 203 is an image encoding unit, 2301 is a data concealment unit, 2302 is a transmission unit, 206 is low-frequency image encoded data, and 2303 is a difference image Confidential data.

  Hereinafter, the flow of processing will be described with reference to the flowchart of FIG.

  First, the monitoring camera 102 obtains a frame image as digital data by forming an optical signal around the camera on an image sensor using a lens and converting the image into an electric signal (S2401).

  Then, the low-frequency image generation unit 201 creates a low-frequency image from the frame image (S2402).

  Further, the difference image generation unit 202 creates a difference image between the frame image acquired by the monitoring camera 102 and the low frequency image created by the low frequency image generation unit 201 (S2403).

  The image encoding unit 203 encodes the low-frequency image created by the low-frequency image generation unit 201 and the difference image created by the difference image generation unit 202 (S2404).

  The data concealment unit 2301 conceals the difference image encoded data encoded by the image encoding unit 203 using encryption or secret sharing (S2405).

  The transmitting unit 2302 transmits the low-frequency image encoded data 206 encoded by the image encoding unit 203 via the network 113 (S2406). The concealment data 2303 of the difference image concealed by the data concealment unit 2301 is transmitted via the network 113 (S2407). The transmitted data is respectively recorded in any of the recording devices 2201, 2202, 2203 installed outside the cloud or the like.

  Hereinafter, the data concealing unit 2301 will be specifically described.

  The data concealing unit 2301 conceals the coded difference image data using encryption or secret sharing. This is because the difference image contains information related to privacy, such as a human face, and therefore it is preferable to encrypt or disperse the data to conceal the data.

  As a method of concealment, a method using encryption is conceivable. As encryption, there are block ciphers and stream ciphers, which are common key ciphers. Triple DES and AES can be applied as block ciphers, and RC4 and the like can be applied as stream ciphers. By using encryption, differential image encoded data including privacy information can be rewritten to information that cannot be understood by a third party, and computational security can be obtained.

  It should be noted that the encryption method is not limited to these, and other methods such as public key encryption can achieve the purpose. When encryption is used, key management must be performed separately.

  As a method of concealment other than encryption, a method using secret sharing may be considered. As the secret sharing, there is a threshold secret sharing method. FIG. 25 is an explanatory diagram of the threshold secret sharing scheme. The threshold secret sharing means that data s to be kept secret is distributed and stored in n distributed shares, and k (k ≦ n) distributed shares are collected and restored. It is obtained. Since the distributed share is converted into information that cannot be understood by a third party and cannot be decrypted by calculation, information-theoretic security can be obtained. The threshold secret sharing described here is called a (k, n) threshold secret sharing method with a threshold k and the number of shares n.

  In addition to the (k, n) threshold secret sharing, the secret sharing has a (k, L, n) ramp-type secret sharing with a threshold k, the number of divisions L, and the number of shares n, and has no threshold. There are several variations such as secret sharing, polynomial secret sharing, and secret sharing speeded up using exclusive OR.

  Here, the threshold secret sharing is described as an example, but the purpose of the present invention can be achieved by using other secret sharing techniques. By performing distributed management with a plurality of shares, redundancy can be provided, so that restoration is possible even when part of stored data is lost. Therefore, by using secret sharing, the reliability of the system can be improved.

  In addition, if secret sharing is used, it is not necessary to manage keys required for encryption. The use of the ramp type secret sharing also has the effect of reducing the data amount of the confidential data of the difference image to 1 / L. When a secret sharing scheme is used, a plurality of shared shares are generated, so that the encoded data of the difference image is stored in a distributed manner. Specifically, if the number of variances is n, n pieces of data relating to the difference image are output.

  Also, in FIG. 23, the concealment data 2303 of the differential image, which is the output of the data concealment unit 2301, is described as having a plurality of outputs. However, when encryption is used for the concealment method, the output is 1 Individual. When secret sharing is used as a concealment method, if the number of secret sharing shares is n, the output is n.

  Next, the restoration process of the second communication device 110 'will be described in detail with reference to FIGS. FIG. 26 is a block diagram of a second communication device according to the third embodiment of the present invention, and FIG. 27 is a flowchart of processing in the second communication device of the third embodiment of the present invention.

  26, reference numeral 2601 denotes a receiving unit, 2602 denotes a data restoring unit, 902 denotes an image decoding unit, 1002 denotes an image synthesizing unit that synthesizes a decoded low-frequency image and a difference image, 206 denotes low-frequency image encoded data, and 2303 denotes Reference numeral 2603 denotes secret image data of the difference image.

  Hereinafter, the flow of processing will be described with reference to the flowchart of FIG.

  First, the receiving unit 2601 receives the low-frequency image encoded data 206 from one of the recording devices 2201, 2202, and 2203 installed outside the cloud or the like via the network 113 (S2701).

  Next, the receiving unit 2601 receives the confidential data 2303 of the difference image from one of the recording devices 2201, 2202, and 2203 similarly installed outside via the network 113 (S2702).

  Then, the data restoration unit 2602 restores the secret data 2303 of the difference image (S2703). By restoring the secret data, encoded difference image data 2603 can be obtained.

  Further, the image decoding unit 902 decodes the encoded low-frequency image data 206 and the encoded differential image data 2603, and creates data of a low-frequency image and a differential image for one frame (S2704).

  The image combining unit 1002 combines the low-frequency image and the difference image to create a restored image (S2705).

  Finally, the monitor 112 displays the restored image data, which is the created frame image (S2706).

  Hereinafter, the data restoration unit 2602 will be specifically described.

  The data restoration unit 2602 restores the confidential data 2303 of the difference image, and creates difference image encoded data 2603. Since the concealment of the encoded difference image data is performed by the data concealment unit 2301 of the first communication device 103 ″, the data restoration unit 2602 can restore the concealment method applied by the data concealment unit 2301. Must be something. Since a method using encryption or a method using secret sharing for concealment is conceivable, it is necessary to prepare a restoration method corresponding to these methods. When the secret sharing method is used for concealment, a plurality of variance shares are generated, and thus confidential data of the difference image is stored in a plurality of pieces. If the number of shares (threshold) required for restoration is k, k pieces of data relating to the difference image must be collected.

  Also, in FIG. 26, the concealment data 2303 of the difference image which is an input of the reception unit 2601 and the data restoration unit 2602 is described as having a plurality of inputs. , One input. When secret sharing is used as a concealment method, if the number of shares (threshold) required for restoring secret sharing is k, the number of inputs is k.

  With the above-described configuration, the data regarding the difference image including the privacy information can be concealed, and a highly confidential surveillance camera system can be provided. In other words, the low-frequency image is an image for which image details cannot be confirmed, and all information related to privacy of the difference image is concealed, so that data can be more securely recorded on an external server such as a recording device on the cloud. Become like In addition, since encoding and concealment are performed after conversion into a low-frequency image and a difference image, it is not necessary to use a special encoder, and a system can be constructed with a simple configuration using a general-purpose module. That is, MPEG, H.264, etc. By using a standard encoding module such as H.264 / AVC or a standard concealment module such as Triple DES or AES, it becomes possible to realize a highly confidential surveillance camera system with almost the same data amount as in the past.

  Note that the reproduction process in the second communication device of the third embodiment can be realized by performing the same process as the reproduction process described in the second communication device of the first embodiment.

  Further, in the present embodiment, the concealment is performed on the difference image coded data, but the difference image data may be concealed without being coded. The present invention can be applied to a configuration in which a difference image is not encoded due to trade-offs such as a processing speed of encoding, a writing speed to a storage device, a communication speed in a network, and a cost of a storage device. Needless to say.

  In the present embodiment, the imaging system 101 ′ is configured to connect the monitoring camera 102 and the first communication device 103 ″ via an internal network. It goes without saying that it is applicable.

  Further, it is needless to say that the object of the present invention can be achieved even when the low-frequency image or the difference image is recorded in a place other than the recording apparatus described in the present embodiment.

  Further, in the present embodiment, only data relating to the difference image is concealed using encryption or secret sharing, but data relating to the low-frequency image is not concealed. This is because the low-frequency image does not need to be concealed because the image itself is an image protecting privacy, and if concealed, decoding and reproduction cannot be performed by a general-purpose reproduction system. With the configuration as in the present embodiment, an image whose outline can be confirmed while protecting privacy can be reproduced by a general-purpose reproduction system, and when necessary, image details can be confirmed by restoring concealment. Become like

(Example 4)
FIG. 28 is a schematic diagram illustrating an overall configuration of a surveillance camera system according to Embodiment 4 of the present invention. The components having the same functions as those of the first, second, and third embodiments are denoted by the same reference numerals, and description thereof is omitted.

  In FIG. 28, 101 ″ is an imaging system, 102 is a surveillance camera, 103 ″ is a first communication device, and these are connected by an internal network. 109 ″ is a restoration system, 110 ″ is a second communication device, 111 is a recording device, 112 is a monitor, and these are also connected by an internal network. Further, the imaging system 101 ″ and a plurality of restoration systems 109 ″ are externally connected via a network 113.

  The first communication device 103 "" in the imaging system 101 "includes a unit that combines data of the low-frequency image and the difference image into one file. Further, the second communication device 110 '' in the restoration system 109 '' includes means for separating and extracting the data of the low-frequency image and the difference image from the combined file. By providing such means, the low-frequency image and the difference image can be handled as one file without separately managing them.

  Hereinafter, in the present embodiment, the first communication device 103 ′ ″ and the second communication device 110 ″ use a secret sharing method with the number of shares n and the threshold k for data concealment and data restoration. The following describes an example of a case in which there is an error.

  First, the first communication device 103 "" will be described in detail with reference to FIGS. FIG. 29 is a block diagram of a first communication device according to the fourth embodiment of the present invention, and FIG. 30 is a flowchart of processing in the first communication device of the fourth embodiment of the present invention.

  In FIG. 29, 201 is a low-frequency image generation unit, 202 is a difference image generation unit, 203 is an image encoding unit, 2301 is a data concealment unit, 2901 is a data combination unit, 2902 is a transmission unit, and 206 is low-frequency image encoding. Data 2303 is confidential data of the difference image, and 2903 is a data file in which coded low-frequency image data and confidential data of the difference image are combined.

  Hereinafter, the processing flow will be described with reference to the flowchart of FIG.

  First, the surveillance camera 102 forms a frame image as digital data by forming an optical signal around the camera on an image sensor using a lens and converting the image into an electric signal (S3001).

  Then, the low-frequency image generation unit 201 creates a low-frequency image from the frame image (S3002).

  Further, the difference image generation unit 202 creates a difference image between the frame image acquired by the monitoring camera 102 and the low frequency image created by the low frequency image generation unit 201 (S3003).

  The image encoding unit 203 encodes the low-frequency image created by the low-frequency image generation unit 201 and the difference image created by the difference image generation unit 202 (S3004).

  The data concealment unit 2301 conceals the difference image encoded data encoded by the image encoding unit 203 using secret sharing (S3005).

  Then, the low-frequency image encoded data 206 encoded by the image encoding unit 203 and the concealment data 2303 of the difference image created by the data concealment unit 2301 are output as a temporary file (S3006). Since secret sharing is used for concealment, if the number of shares is n, n pieces of confidential data of the differential image will be created.

  The data combining unit 2901 combines the low-frequency image encoded data 206 output as a file and the confidential data 2303 of the difference image to create a data file 2903 (S3007). Since n pieces of confidential data of the difference image are created, n data files 2903 are also created.

  The transmitting unit 2902 transmits the data file 2903 combined by the data combining unit 2901 to the restoration system 109 '' via the network 113 (S3008). Since the n data files 2903 are distributedly managed, the data files 2903 are distributed and transmitted to a plurality of restoration systems 109 ''.

  Hereinafter, the data combining unit 2901 will be specifically described.

  The data combining unit 2901 combines the low-frequency image encoded data output as a temporary file and the file of the confidential data of the difference image. FIG. 31 is a view showing an example of the combined file structure. In FIG. 31, the combined file stores low-frequency image encoded data from the beginning. Following the low-frequency image encoded data, a head marker of the difference image data is stored, and subsequently, a concealment parameter is stored. The concealment parameter is a parameter necessary for restoring the confidential data of the difference image.

  When concealment is performed using the threshold secret sharing method, for example, the number of shares, thresholds, data sizes before and after secret sharing processing, and the like are stored as concealment parameters. Subsequently, the secret data of the difference image is stored. Finally, the end marker of the difference image data is stored, and subsequently, the offset from the end of the file to the top marker of the difference image data is stored.

  FIG. 32 is an illustration of an example of a combined file configuration when the data concealing unit 2301 uses secret sharing. When the secret sharing of the sharing number n is applied, n pieces of confidential data of the difference image are created. Therefore, n combined files are also created. The low-frequency image encoded data stored at the head of each file is all the same, but the confidential data of the difference image stored thereafter is n different pieces of distributed data.

  Here, an example in which the low-frequency image encoded data and the confidential data of the difference image are combined has been described, but it goes without saying that the present invention is applicable regardless of whether or not the low-frequency image is encoded. No. Further, it goes without saying that the present invention is applicable regardless of whether the difference image is encoded or not. It is possible to achieve the object of the present invention by creating a data file by combining the respective files.

  Further, the configuration of the combined file is not limited to the examples shown in FIGS. Since the combined file must be able to be decoded and reproduced as a low-frequency image by a general-purpose reproduction system, it is necessary to store data relating to the low-frequency image at the top. However, the object of the present invention can be achieved even if the data relating to the difference image is not a configuration shown in FIGS.

  In FIG. 29, the secret data 2303 of the difference image and the combined file 2903 are described so as to have a plurality of outputs. This is because, when the secret sharing is used as the concealment method, if the number of shares of the secret sharing is n, the output is n. If encryption is used for the concealment method, these outputs are one.

  Next, the processing of the second communication device 110 '' will be described in detail. The second communication device can perform the reproduction process and the restoration process in the same manner as described in the first embodiment.

  First, the reproduction process of the second communication device 110 '' will be described in detail with reference to FIGS. FIG. 33 is a block diagram at the time of reproduction of the second communication device according to the fourth embodiment of the present invention, and FIG. 34 is a flowchart of a reproduction process in the second communication device of the fourth embodiment of the present invention.

  In FIG. 33, reference numeral 3301 denotes a receiving unit, 3302 denotes a data separating unit, 2602 denotes a data restoring unit, 902 denotes an image decoding unit, 1002 denotes an image synthesizing unit, 3303 denotes low-frequency image encoded data and secret data of a difference image are combined. The data file 3304 is low frequency image encoded data, and 3305 is low frequency image data.

  Hereinafter, the processing flow will be described with reference to the flowchart of FIG.

  First, the receiving unit 3301 receives, from the imaging system 101, the data file 3303 in which the low-frequency image encoded data and the confidential data of the difference image are combined via the network 113 (S3401).

  Next, the receiving unit 3301 records the received combined file 3303 in the recording device 111 (S3402).

  Further, the data separation unit 3302 acquires the low-frequency image encoded data 3304 from the combined file (S3403). However, since the low-frequency image encoded data is stored at the head of the combined file, it can be obtained by reading the data from the head of the file without any special processing.

  Then, the image decoding unit 902 decodes the low-frequency image encoded data 3304 and creates one frame of low-frequency image data 3305 (S3404).

  Finally, the monitor 112 displays the low-frequency image data 3305, which is the created frame image (S3505).

  At the time of normal reproduction, only low-frequency image data is decoded. Therefore, only an image in which privacy is protected is displayed, and not all image details can be confirmed. Also, the combined file 3303 is necessary for restoration, and is recorded in the recording device 111.

  Further, even when the combined file 3303 is received by a system having a configuration as shown in the playback system 105 of the first embodiment, it can be played back as an image in which privacy is protected.

  Next, the restoration processing of the second communication device 110 '' will be described in detail with reference to FIGS. FIG. 35 is a block diagram at the time of restoration of the second communication device according to the fourth embodiment of the present invention, and FIG. 36 is a flowchart of restoration processing in the second communication device of the fourth embodiment of the present invention.

  In FIG. 35, 3301 is a receiving unit, 3302 is a data separating unit, 2602 is a data restoring unit, 902 is an image decoding unit, 1002 is an image synthesizing unit, 3303 and 3501 are low-frequency image encoded data and secret data of a differential image. The combined data file, 3304 is coded low-frequency image data, 3502 is confidential data of a difference image, 3503 is coded difference image data, and 3504 is a restored image.

  Hereinafter, the processing flow will be described with reference to the flowchart of FIG.

  First, the receiving unit 3301 acquires the data file 3303 in which the low-frequency image encoded data recorded in the recording device 111 and the confidential data of the difference image are combined. Restoration of secret sharing requires data files that are sharing shares by the number of thresholds k. Therefore, via the network 113, k-1 data files 3501 recorded in the restoring system 109 '' are received and acquired (S3601).

  Next, the data separation unit 3302 separates the data files 3303 and 3501 into low-frequency image encoded data and confidential data of a difference image, and acquires low-frequency image encoded data 3304 (S3602). Specifically, the head address of the low frequency image encoded data is acquired from the data file 3303. In this embodiment, the data file has a configuration as shown in FIG. Therefore, the head address of the low frequency image encoded data is the head of the data file 3303.

  Further, secret data 3502 of the difference image is obtained from the data separated by the data separation unit 3302 (S3603). Specifically, the head address of the confidential data of the difference image is obtained from the data files 3303 and 3501. At the end of the data files 3303 and 3501, the offset from the end of the file to the head marker of the difference image data is stored. Therefore, the head address of the confidential data of the difference image can be obtained by using this value to move from the end of the file by the offset. Since the secret sharing of the threshold value k is used for data concealment, there are k pieces of concealment data of the difference image necessary for restoration. Therefore, the head addresses of the confidential data of the k differential images are obtained from the k data files.

  Further, the data restoration unit 2602 restores the secret data 3502 of the difference image (S3604). By restoring the secret data, difference image encoded data 3503 can be obtained.

  Then, the image decoding unit 902 decodes the encoded low-frequency image data 3304 and the encoded differential image data 3503, and creates data of the low-frequency image and the differential image for one frame (S3605).

  The image combining unit 1002 combines the low-frequency image and the difference image to create a restored image 3504 (S3606).

  Finally, the monitor 112 displays the restored image data 3504, which is the created frame image (S3607).

  In FIG. 35, the combined file 3501 that is an input of the receiving unit 3301 and the data separating unit 3302 is described as having a plurality of inputs. This is because, when the secret sharing is used as the concealing method, if the number of shares (threshold) required for restoring the secret sharing is k, the necessary concealment data 3502 of the differential image is k. If encryption is used for the concealment method, it is possible to restore only the combined file 3303 recorded in the recording device 111, so there is no need to receive the combined file 3501 and the data restoration unit 2602 requires only one input. is there.

  Further, in the present embodiment, the case where the encryption or the secret sharing is used for concealing the difference image has been described as an example. However, there are some variations in the method of creating the combined file. For example, as shown in FIG. 37, it is conceivable to separate the difference image into encrypted data and a decryption key of the difference image, combine them with the low-frequency image encoded data, and record them as different combined files. In this case, instead of storing the concealment parameter, the file storing the decryption key may store an identifier indicating that it is the decryption key.

  Alternatively, as shown in FIG. 38, it is conceivable to divide the difference image encoded data into n files, combine each fragment with the low-frequency image encoded data, and record it. Alternatively, the difference image data may be divided into n files as they are, and each fragment may be combined with the low-frequency image encoded data. Alternatively, the difference image encoded data may be encrypted and then divided into n files. Then, each fragment may be combined with the low frequency image encoded data.

  Simply dividing data reduces the strength of concealment compared to encryption or secret sharing, but it can reduce the load on the CPU and protect data while protecting privacy. Since the object of the invention can be achieved, it is possible to appropriately use the terminal according to the terminal to be used and the importance of privacy protection.

  With the above-described configuration, low-frequency image data in which privacy is protected and confidential data of a difference image required for restoration can be recorded as one file. This file can be reproduced as an image with privacy protected by a normal receiving device, and can be restored in an emergency such as a crime. When encryption is used for concealment, it can be restored with a key. In the case where secret sharing is used for concealment, restoration can be performed by collecting a required number of shared files.

  Therefore, it is not necessary to separately record the low-frequency image and the difference image, and the file management becomes easy. Specifically, it is not necessary to manage and record the file name of the difference image data and the recorded location (file path or the like) corresponding to the low-frequency image data using a table or the like.

  From the user's point of view, it can be handled in the same way as encrypted or secret-shared binary data. More specifically, for example, when data is encrypted, the encrypted data is meaningless binary data, but when decrypted using a key, the data becomes a meaningful data file. If the data is secretly shared, the shared data is meaningless binary data, but if the required number of shared shares for restoration are collected and restored, the data becomes a meaningful data file.

  Similarly, when the encryption of the present invention is applied to image data, the encrypted data can be reproduced only as a privacy-protected image, but when decrypted using a key, the image can be reproduced as an image in which all details can be confirmed. Become like In addition, when the secret sharing of the present invention is applied to image data, the shared data can be reproduced only as a privacy-protected image, but if the required number of shared shares for restoration are collected and restored, all details can be confirmed. It can be reproduced as an image. In other words, from the user's point of view, files can be handled with the same interface as ordinary encryption and secret sharing, restored, can be played back as the original image, and privacy protected images without recovery The data can be reproduced while checking the outline.

  In addition, if secret sharing is used for concealment, not only high confidentiality but also a disaster-resistant and fault-tolerant system can be constructed as a backup, realizing a surveillance camera system that is highly convenient for users. become able to.

  INDUSTRIAL APPLICABILITY The communication system, image generation method, and communication device of the present invention are useful for a system that wants to confirm details of an image when necessary while protecting privacy, and is useful for, for example, a surveillance camera system.

Reference Signs List 101 imaging system 102 surveillance camera 103 first communication device 104 recording device 105 reproduction system 106 receiving device 107 recording device 108 monitor 109 restoration system 110 second communication device 111 recording device 112 monitor 113 network 201 low frequency image generation unit 202 Difference image generation unit 203 Image encoding unit 204 Transmission unit 901 Receiving unit 902 Image decoding unit 1001 Receiving unit 1002 Image synthesizing unit 1601 Mask area setting unit 1602 Area size setting unit 2201 Recording device 2202 Recording device 2203 Recording device 2301 Data concealment unit 2302 Transmitting section 2601 receiving section 2602 data restoring section 2901 data combining section 2902 transmitting section 3301 receiving section 3302 data separating section

Claims (9)

A communication system in which a first communication device, a second communication device, and a plurality of recording devices are connected via a network,
The first communication device includes:
Generate low frequency image data by extracting low frequency components from the original image data,
Shows the difference between the original image data and the low-frequency image data, generates difference image data capable of confirming details of the original image data compared to the low-frequency image data,
Generating a plurality of secret sharing data from the difference image data;
Transmit each of the low-frequency image data and the plurality of secret sharing data to any of the plurality of recording devices,
The plurality of recording devices,
Record at least one of the low-frequency image data or the secret sharing data,
The second communication device includes:
Receiving at least one of the plurality of secret sharing data and the low-frequency image data from any of the plurality of recording devices,
When logging in with the first ID, the low-frequency image data is output. When logging in with the second ID, at least the difference image data is restored and output using the secret sharing data.
Communications system. A communication system in which a first communication device, a second communication device, and a plurality of recording devices are connected via a network,
The first communication device includes:
Generate low frequency image data by extracting low frequency components from the original image data,
Shows the difference between the original image data and the low-frequency image data, generates difference image data capable of confirming details of the original image data compared to the low-frequency image data,
Generating a plurality of secret sharing data from the difference image data,
Transmitting each of the low-frequency image data and the plurality of secret sharing data to any of the plurality of recording devices,
The plurality of recording devices,
Record at least one of the low-frequency image data or the secret sharing data,
The second communication device includes:
Receiving at least one of the plurality of secret sharing data and the low-frequency image data from any of the plurality of recording devices,
At the time of normal reproduction, the low-frequency image data is output, and at the time of restoration, at least the difference image data is restored and output using the secret shared data,
Communications system. The first communication device includes:
Encoding the generated low-frequency image data and the difference image data,
Generating a plurality of secret sharing data from the encoded difference image data,
The communication system according to claim 2. The first communication device includes:
Encoding the generated low-frequency image data and the difference image data and transmitting each to any of the plurality of recording devices,
The communication system according to claim 2. The first communication device includes:
Generating the low-frequency image data by taking an average of pixels in a predetermined area,
The communication system according to any one of claims 2 to 4. The first communication device includes:
Generate the low-frequency image data by extracting low-frequency components only in a part of the original image data,
Generate the difference image data of the same size as the low-frequency image data,
The communication system according to claim 2. An image generation method using a communication system in which a first communication device, a second communication device, and a plurality of recording devices are connected via a network,
The first communication device includes:
Generate low frequency image data by extracting low frequency components from the original image data,
Shows the difference between the original image data and the low-frequency image data, generates difference image data capable of confirming details of the original image data compared to the low-frequency image data,
Generating a plurality of secret sharing data from the difference image data;
Transmit each of the low-frequency image data and the plurality of secret sharing data to any of the plurality of recording devices,
The plurality of recording devices,
Record at least one of the low-frequency image data or the secret sharing data,
The second communication device includes:
At least one of the plurality of secret sharing data and the low-frequency image data, received from any of the plurality of recording devices,
During normal reproduction, the low-frequency image data is output,
At the time of restoration, using the secret sharing data, the difference image data is restored and output,
Image generation method. A communication device connected to the first communication device and the plurality of recording devices via a network,
The low-frequency image data obtained by extracting a low-frequency component from the original image data, and a difference between the original image data and the low-frequency image data are shown, and the details of the original image data can be confirmed in comparison with the low-frequency image data. A communication unit that receives at least one of the plurality of secret sharing data generated from the difference image data from any of the plurality of recording devices;
A processor that outputs the low-frequency image data during normal reproduction, and uses the secret shared data during restoration to restore and output the difference image data.
Communication device. The data file received by the communication unit, the data file having the low-frequency image data and at least one of the plurality of pieces of secret sharing data addressed to the communication device includes the low-frequency image data stored at the top of the data file, Between the frequency image data and at least one of the plurality of secret sharing data addressed to the communication device, at least one leading marker of the plurality of secret sharing data addressed to the communication device is stored.
The communication device according to claim 8.