JP2021077996A - Image processing device and image processing method - Google Patents

Abstract

To provide a technique capable of proper concealment for protecting personal information, which is to be applied to a photographic image.SOLUTION: An image processing device comprises: a detection unit provided to be capable of detecting out of an acquired photographic image an object including a specific part needing concealment for protecting personal information and the specific part; and a concealment unit which performs concealment processing for the concealment on the photographic image based on a result of the detection by the detection unit. The concealment processing is different between in a case of having detected both the object and the specific part and in a case of having detected only the object out of the object and the specific part.SELECTED DRAWING: Figure 6

Description

The present invention relates to an image processing apparatus and an image processing method.

Conventionally, a technique for protecting the privacy of a subject reflected in a photographed image taken by a camera is known. For example, Patent Document 1 discloses an image processing device including an image acquisition unit, a number of people / person flow estimation unit, a privacy level setting unit, and an image processing unit.

In the image processing apparatus of Patent Document 1, the image acquisition unit acquires an image. The number / person flow estimation unit takes the image acquired by the image acquisition unit as an input and estimates the density distribution corresponding to the head position of the crowd included in the image and the number of people constituting the crowd based on the density distribution. .. The privacy level setting unit sets a privacy level indicating the degree of image processing performed to protect the privacy of a plurality of subjects. The image processing unit receives the image acquired by the image acquisition unit as an input, and performs processing processing according to the privacy level based on the estimation result by the number of people / person flow estimation unit. According to the image processing apparatus of Patent Document 1, since the result of estimation by the number of people / person flow estimation unit is used, there is little omission of detection of the head even in a crowded situation, and the subject can be appropriately concealed.

JP-A-2019-71050

By the way, when the feature of the object cannot be detected, for example, the size of the object to be detected is small in the captured image, the detection of the object to be detected may fail regardless of the congestion situation or the like. If the detection of an object that can identify an individual such as a face fails, there is a possibility that the process of concealing the face or the like may be leaked, which causes a problem.

Further, if the range of concealment in the captured image becomes too wide, the information that can be extracted from the captured image decreases. For this reason, it is desirable to avoid excessive concealment of captured images.

In view of the above problems, an object of the present invention is to provide a technique capable of appropriately concealing a photographed image for protection of personal information.

In order to achieve the above object, the image processing apparatus of the present invention includes an object including a specific part that needs to be concealed to protect personal information and a detection unit provided so as to be able to detect the specific part from the acquired captured image. Based on the detection result of the detection unit, the captured image is provided with a concealment unit that performs concealment processing for concealment, and the concealment processing detects both the object and the specific portion. The configuration is different (first configuration) depending on whether the object is detected or only the object is detected among the object and the specific portion.

Further, in the image processing apparatus having the first configuration, when the concealment process detects both the object and the specific portion, the concealment process performs the concealment targeting the specific portion. Therefore, when only the object is detected among the object and the specific portion, a configuration (second configuration) that is a second concealment process for concealing the object is preferable.

Further, the image processing apparatus having the second configuration further includes an image segmentation unit that performs image segmentation for labeling the meaning of each pixel of the acquired image, and the second concealment process is the image. It may be a configuration (third configuration) in which the region of the object obtained by segmentation is processed into an image in which the characteristics of the object are reduced.

Further, in the image processing apparatus having the third configuration, it is preferable that the image generated during the image segmentation is applied to the image in which the characteristics of the object are reduced (fourth configuration). ..

Further, in the image processing apparatus having any of the first to fourth configurations, the concealment unit is obtained from the photographed image with respect to at least one of the object and the specific portion obtained from the photographed image. It is preferable that the configuration (fifth configuration) is such that the concealment processing is performed while reflecting the characteristics.

Further, in the image processing apparatus having any of the first to fourth configurations, the concealment unit decryptively encrypts at least one of the object and the specific portion in the captured image. It may be the configuration (sixth configuration) for concealment.

Further, in order to achieve the above object, the image processing method of the present invention includes a detection step of attempting to detect an object including a specific part that needs to be concealed to protect personal information and the specific part from the acquired photographed image. Based on the detection result of the detection step, the captured image is provided with a concealment process for performing the concealment process for concealment, and the concealment process detects both the object and the specific part. The configuration is different (seventh configuration) depending on whether the object is detected or only the object is detected among the object and the specific portion.

According to the present invention, it is possible to appropriately conceal the photographed image for the protection of personal information.

Diagram showing the configuration of the personal information protection system The figure which shows the structure of the image processing apparatus which concerns on 1st Embodiment A flowchart showing an operation example of the image processing apparatus according to the first embodiment. The figure which shows an example of the photographed image acquired by the acquisition part The figure which illustrates the detection result of the detection part with respect to the photographed image shown in FIG. The figure which illustrates the result of the concealment processing executed according to the detection result shown in FIG. Diagram for explaining a modified example of the concealment process performed by the concealment unit The figure which shows the structure of the image processing apparatus which concerns on 2nd Embodiment A flowchart showing an operation example of the image processing apparatus according to the second embodiment. The figure which illustrates the result of the image segmentation with respect to the photographed image shown in FIG. The figure which illustrates the concealment processing result when the photographed image shown in FIG. 4 is input in the image processing apparatus of 2nd Embodiment.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following, the case where the present invention is used for a vehicle will be described as an example, but the present invention can be widely applied to other than vehicles.

<1. Personal information protection system ＞
FIG. 1 is a diagram showing a configuration of a personal information protection system 100 according to an embodiment of the present invention. The personal information protection system 100 includes a vehicle device 1 and a data collection device 2.

The vehicle device 1 is a terminal device mounted on a vehicle and used. The vehicle is, for example, a vehicle having wheels such as an automobile, a motorcycle, a train, and an automatic guided vehicle. The vehicle is equipped with various sensors including a camera. The vehicle device 1 acquires various data from various sensors. The various data may include, for example, the speed of the vehicle, the position of the vehicle, the passing time of a specific position of the vehicle, and the like, in addition to the captured image taken by the camera.

The vehicle device 1 may also be used by, for example, a drive recorder or a navigation device. However, the vehicle device 1 may be a device different from the drive recorder and the navigation device. Further, the vehicle device 1 may be an in-vehicle device or a portable terminal that can be carried by a vehicle occupant. When the vehicle device 1 is a mobile terminal, the vehicle device 1 may be, for example, a smartphone, a tablet, or the like.

The data collection device 2 is provided so as to be able to communicate with the vehicle device 1 via a network (not shown) such as the Internet or a mobile phone line network. The data collection device 2 is configured as, for example, a cloud server that provides a cloud service via a network. The data collection device 2 receives a data collection request from the data user (not shown), and provides the data user with the data collected from the vehicle device 1 based on the received collection request. The data collection device 2 is usually provided so as to be able to communicate with a plurality of vehicle devices 1 and collect data from each vehicle device 1.

As shown in FIG. 1, the vehicle device 1 includes a personal information protection processing unit 11. The personal information protection processing unit 11 performs protection processing for personal information included in data obtained from various sensors mounted on a vehicle so that an individual cannot be identified. The vehicle device 1 appropriately performs protection processing for protecting personal information on various types of data, and transmits the various data to the data collection device 2.

In principle, personal information is information that can identify a specific individual, but in this specification, personal information may be used synonymously with privacy. Personal information includes, for example, a person's face included in a photographed image taken by a camera, a vehicle license plate, a point name sign that can identify a place where the vehicle has passed, and the like. Further, the personal information is not limited to the information included in the photographed image, and may include a vehicle identification number (VIN: Vehicle Identification Number), various times, and the like. The personal information includes personal information about the occupants of the vehicle on which the vehicle device 1 is mounted, personal information about a person who exists outside the vehicle on which the vehicle device 1 is mounted, or only one of them. You can do it.

As a method of protecting personal information by the personal information protection processing unit 11, for example, a process of lowering the resolution of the photographed image, a process of encrypting the whole or a part of the photographed image, a process of encrypting character or numerical data, and a process of photographing. Examples include processing for masking all unnecessary information (including personal information) in the image. When performing encryption, it is preferable that the data collection device 2 has a configuration in which decryption can be performed.

In the present embodiment, the personal information protection processing unit 11 protects the acquired captured image by the image processing devices 3 and 3A (see FIGS. 2 and 8) described later included in the vehicle device 1. I do. The personal information protection processing unit 11 encrypts the personal information detected from other than the captured image such as the vehicle identification number by the encryption means. The vehicle device 1 transmits the data protected by the personal information protection processing unit 11 to the data collection device 2 via the transmission unit 12.

As shown in FIG. 1, the data collection device 2 includes a personal information management unit 21. The personal information management unit 21 acquires various data via the receiving unit 22 that receives data from the transmitting unit 12 of the vehicle device 1, and manages the personal information of the various data. The personal information management unit 21 processes the acquired data so that the individual is not identified and stores it in the storage unit 23. The data stored in the storage unit 23 is used by the above-mentioned data user.

In the present embodiment, the vehicle device 1 is subjected to the protection process for protecting personal information, but the present invention is not limited to this, and the data collection device 2 may perform the protection process for protecting the personal information. .. In this case, the image processing devices 3 and 3A described later are included in the data collection device 2 side. However, it is preferable that the image processing devices 3 and 3A that perform processing for protecting personal information are included not in the data collecting device 2 configured as a server but in the vehicle device 1 side that transmits data. Further, the device that transmits data to the data collection device 2 may not be the vehicle device 1, but may be, for example, a terminal device installed in a commercial facility equipped with a surveillance camera, a station, a parking lot, or the like.

<2. Image processing device>
(2-1. First Embodiment)
[2-1-1. Image processing device configuration]
FIG. 2 is a diagram showing a configuration of an image processing device 3 according to the first embodiment of the present invention. Note that FIG. 2 shows only the components necessary for explaining the features of the image processing apparatus 3 of the first embodiment, and the description of general components is omitted. Further, FIG. 2 also shows a camera 4, which is a component different from the image processing device 3 for easy understanding. As shown in FIG. 2, the image processing device 3 includes an acquisition unit 31, a control unit 32, and a storage unit 33.

The acquisition unit 31 acquires a captured image. In the present embodiment, the acquisition unit 31 continuously acquires analog or digital captured images from the camera 4 mounted on the vehicle at a predetermined cycle (for example, 1/30 second cycle). A collection of captured images acquired by the acquisition unit 31 is a moving image captured by the camera 4. When the acquired captured image is analog, the acquisition unit 31 converts the analog captured image into a digital captured image (A / D conversion). The acquisition unit 31 outputs the acquired captured image (or the converted image if A / D conversion is performed) to the control unit 32.

The camera 4 is a camera that monitors the surroundings of the vehicle, such as the front and the rear of the vehicle. However, the camera 4 may be, for example, a camera that photographs the interior of the vehicle.

The control unit 32 is a controller that comprehensively controls the entire image processing device 3. The control unit 32 is configured as, for example, a computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like, which are hardware processors.

The storage unit 33 is composed of, for example, a semiconductor memory element such as a RAM or a flash memory, a hard disk, or a storage device using a portable recording medium such as an optical disk. The storage unit 33 stores a program as firmware and various data. In the present embodiment, the storage unit 33 stores the learned model 331 that enables the detection of an object. The trained model 331 is obtained by learning by a machine learning algorithm such as a deep neural network (DNN). The trained model 331 may be obtained, for example, by supervised learning.

The detection unit 321 and the concealment unit 322 shown in FIG. 2 are functions of the control unit 32 realized by the CPU of the control unit 32 executing arithmetic processing according to a program stored in the storage unit 33. In other words, the image processing device 3 includes a detection unit 321 and a concealment unit 322.

At least one of the detection unit 321 and the concealment unit 322 of the control unit 32 is composed of hardware such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), and a GPU (Graphics Processing Unit). May be done. Further, the detection unit 321 and the concealment unit 322 are conceptual components. The functions executed by one component may be distributed to a plurality of components, or the functions of the plurality of components may be integrated into one component. Further, the acquisition unit 31 may be configured to be realized by the CPU of the control unit 32 performing arithmetic processing according to a program. Further, with respect to the specific hardware configuration of the image processing device 3, the components may be omitted, replaced, or added as appropriate according to the embodiment. For example, the control unit 32 may include a plurality of hardware processors.

The detection unit 321 is provided so as to be able to detect an object including a specific part that needs to be concealed to protect personal information and the above-mentioned specific part from the acquired captured image. In the present embodiment, the detection unit 321 uses the learned model 331 stored in the storage unit 33 to capture a predetermined object including a specific part that needs to be concealed and a specific part of the predetermined object. Detect from. It is preferable that the detection unit 321 detects an object by the trained model 331 using a low-resolution image obtained by reducing the resolution of the captured image acquired from the acquisition unit 31. As a result, the processing load of the object detection process in the detection unit 321 can be reduced.

Examples of the predetermined object include a person, a vehicle such as an automobile, a point name sign for specifying a place provided in the vicinity of a traffic light, and the like. A specific part is a part that makes it possible to identify a specific individual and estimate the privacy of an individual. When a predetermined object is a person, the specific part is, for example, a face. When the predetermined object is a vehicle, the specific part is, for example, a license plate. When a predetermined object is a point name sign, the specific part is, for example, a character part of the sign. In addition, a predetermined object may include a plurality of specific parts. Further, the predetermined object may include a plurality of types of objects. It is preferable that the predetermined object detectably provided in the detection unit 321 includes at least one of a person and a vehicle.

The trained model 331 is preferably configured to detect an object and a specific part using DNN. However, the trained model 331 may be configured to detect an object and a specific part by another machine learning algorithm such as SVM (Support Vector Machine) using HOG (Histogram Of Gradient) features. Further, the detection unit 321 may be configured to detect an object and a specific part by using, for example, template matching, without using the trained model 331 that has undergone machine learning.

The concealment unit 322 performs a concealment process for concealing the captured image based on the detection result of the detection unit 321. The concealment process is a process of processing a photographed image (data) in order to protect personal information. The concealment process differs between the case where both the object (including the specific part) and the specific part are detected and the case where only the object among the object (including the specific part) and the specific part is detected. That is, the concealment unit 322 changes the concealment method depending on whether it detects both the object and the specific part and the case where only the object is detected among the object and the specific part. For example, the method of concealment processing is changed depending on whether both the vehicle and the license plate are detected and the license plate cannot be detected and only the vehicle is detected.

According to this configuration, when a specific part for which personal information needs to be protected can be detected, it is possible to prevent the range of concealment from becoming excessive and conceal the specific part. In addition, even if a specific part cannot be detected, if an object containing the specific part can be detected, the concealment process can be performed by a method different from the case where the specific part can be detected. It is possible to reduce the possibility of omission in the concealment process for protection.

The specific part is a part of the object, and when the specific part can be detected, the object including the specific part can usually be detected. That is, it is unlikely that an object including a specific part and a state in which only a specific part is detected among the specific parts will occur. If such a state occurs, the same concealment processing as in the case where both the object and the specific part can be detected may be performed. As a result, personal information can be protected. In addition, if both the object and the specific part cannot be detected, there is no object to protect personal information and the concealment process is not performed.

Specifically, the concealment process performed by the concealment unit 322 is a first concealment process for concealing a specific part when both an object and a specific part are detected. The first concealment process is a process of concealing a specific part of the entire object in the captured image. For example, the first concealment process is a process of concealing only the license plate of the entire vehicle in the captured image. The first concealment process may be, for example, a process of painting a specific portion with one color. Further, the first concealment process may be, for example, a process of blurring a specific portion or applying a mosaic to the specific portion.

The range of concealment performed by the first concealment process may or may not completely match the detection range of the specific portion. It suffices to protect personal information by concealing a specific part. It is preferable that the range of concealment performed by the first concealment process is wider than the detection range of a specific part. As a result, personal information can be appropriately protected in consideration of an error in the detection range of a specific part.

The concealment process performed by the concealment unit 322 is a second concealment process for concealing an object when only an object is detected among the object and a specific part. The second concealment process is a process of concealing the entire object in the captured image. For example, the second concealment process is a process of concealing the entire vehicle in the photographed image when the vehicle can be detected from the photographed image but the license plate cannot be detected. The second concealment process may be, for example, a process of painting the entire object in the captured image with one color. Further, the second concealment process may be, for example, a process of blurring the entire object in the captured image or applying a mosaic to the entire object.

The range of concealment performed by the second concealment process may or may not completely match the detection range of the object. It suffices to protect personal information by concealing the object. It is preferable that the range of concealment performed by the second concealment process is wider than the detection range of the object. As a result, personal information can be appropriately protected in consideration of an error in the detection range of the object.

In the present embodiment, when the image processing device 3 can detect a specific part for which personal information needs to be protected, the image processing device 3 focuses on the specific part and conceals it. Therefore, it is possible to suppress excessive processing of the captured image and prevent excessive reduction of information about the object. On the other hand, according to the present embodiment, even if a specific part cannot be detected, if an object including the specific part can be detected, the specific part is concealed in order to conceal the entire object. The possibility of leakage can be reduced.

[2-1-2. Image processing device operation example]
FIG. 3 is a flowchart showing an operation example of the image processing device 3 according to the first embodiment of the present invention. The image processing device 3 operates the flowchart shown in FIG. 3 every time a captured image is acquired by, for example, the acquisition unit 31.

In step S1, the acquisition unit 31 acquires a captured image from the camera 4. The acquisition unit 31 acquires, for example, a captured image as shown in FIG. The photographed image shown in FIG. 4 shows two people H1 and H2 walking along the wall W arranged on the side of the road R. When the acquisition unit 31 acquires the captured image, the process proceeds to the next step S2.

In step S2, the detection unit 321 performs the object detection process. Specifically, the detection unit 321 attempts to detect a predetermined object and a specific portion included in the predetermined object by using the trained model 331. For example, when a predetermined object is a person, the detection unit 321 attempts to detect the person and the face included in the captured image. When the object detection process is completed, the process proceeds to step S3.

FIG. 5 is a diagram illustrating the detection result of the detection unit 321 for the captured image shown in FIG. In FIG. 5, the thick frames B1 to B3 are bounding boxes. The detection unit 321 has completed the classification of the objects in the bounding boxes B1 to B3. In the example shown in FIG. 5, the detection unit 321 can detect the person H1 (bounding box B1) in the captured image, but cannot detect the face of the person H1. In this example, it is probable that the facial features were not recognized and the face of the person H1 could not be detected because, for example, the person H1 was facing downward. The detection unit 321 can detect a person (bounding box B2) and a face (bounding box B3) for the person H2 in the captured image.

In step S3, the concealment unit 322 determines whether or not the detection unit 321 has detected a predetermined object. For example, when a predetermined object is a person, it is determined whether or not the person is detected from the captured image. If it is determined that a predetermined object has been detected (Yes in step S3), the process proceeds to step S4. On the other hand, when it is determined that the predetermined object has not been detected (No in step S3), the process shown in the flowchart in FIG. 3 is temporarily terminated. In the example shown in FIG. 3, it is premised that the situation where only a specific part is detected without detecting a predetermined object does not occur.

In the example shown in FIG. 5, a person can be detected for both the person H1 and the person H2. Therefore, in the example shown in FIG. 5, the process proceeds to step S4.

In step S4, the concealment unit 322 determines whether or not the detection unit 321 has detected a specific portion included in the predetermined object. For example, when the object detected by the detection unit 321 is a person, it is determined whether or not the face, which is a specific part, is detected from the captured image. When it is determined that the specific portion has been detected (Yes in step S4), the process proceeds to step S5. On the other hand, if it is determined that the specific portion has not been detected (No in step S4), the process proceeds to step S6.

In step S4, when a plurality of predetermined objects are detected in step S2, it is determined whether or not a specific portion is detected for each predetermined object. That is, it is determined which of step S5 and step S6 should be performed for each of the detected objects.

In the example shown in FIG. 5, since the person H1 and the person H2 are detected, it is determined whether or not the face, which is a specific part, is detected for each person H1 and H2. As for the person H1, since the face is not detected, it is determined that the process proceeds to step S6. Since the face of the person H2 is detected, it is determined that the process proceeds to step S5.

In step S5, the concealment unit 322 performs the first concealment process targeting a specific part. For example, when the object detected by the detection unit 321 is a person, the first concealment process is performed on the detected face. When the first concealment process is completed, the process shown in the flowchart in FIG. 3 is once terminated.

In step S6, the concealment unit 322 performs a second concealment process targeting a predetermined object. For example, when the object detected by the detection unit 321 is a person, the second concealment process is performed on the entire detected person. When the second concealment process is completed, the process shown in the flowchart in FIG. 3 is temporarily terminated.

FIG. 6 is a diagram illustrating the result of the concealment processing executed according to the detection result shown in FIG. Since the face, which is a specific part, has not been detected for the person H1, the process proceeds to step S6, and concealment is performed for the entire detected person. In the example shown in FIG. 6, a mosaic for concealment is applied to the entire area surrounded by the bounding box B1 indicating the detection of the person H1. On the other hand, since the face which is a specific part is detected for the person H2, the process is advanced to step S5, and the detected face is concealed. In the example shown in FIG. 6, a mosaic for concealment is applied to the entire area surrounded by the bounding box B3 indicating the detection of the face of the person H2.

In addition, instead of the mosaic process for concealment, for example, a blurring process or a filling process may be performed. Further, the process for concealment may be performed in a slightly wider range than the area surrounded by the bounding boxes B1 and B3, for example.

[2-1-3. Modification example]
FIG. 7 is a diagram for explaining a modification of the concealment process performed by the concealment unit 322. FIG. 7 is a diagram showing a photographed image that has undergone concealment processing. In the example shown in FIG. 7, similarly to FIG. 6 described above, the person H1 is subjected to a concealment process (second concealment process) for the entire detected person, and the person H2 is subjected to a specific part. A concealment process (first concealment process) for the (face) is being performed.

In performing the concealment processing, the concealment unit 322 may replace at least one of the object (including the specific portion) obtained from the captured image and the specific portion with CG (Computer Graphics). In the example shown in FIG. 7, only a specific part (face of human H2) obtained from the captured image is replaced with CG. The object (human H1) obtained from the photographed image is mosaiced instead of being replaced with CG for concealment. However, the object (human H1) obtained from the captured image may also be replaced with CG. Further, the object (human H1) obtained from the photographed image may be replaced with CG, and the specific portion (face of human H2) may be mosaiced or the like instead of being replaced with CG.

In addition, the concealment unit 322 may perform concealment processing on at least one of the object (including the specific portion) obtained from the captured image and the specific portion while reflecting the features obtained from the captured image. As a result, it is possible to prevent the amount of information obtained from the captured image from being reduced while protecting the personal information.

In the example shown in FIG. 7, the concealment processing is performed only on the specific portion (face of the person H2) obtained from the photographed image while reflecting the features obtained from the photographed image. The object (human H1) obtained from the photographed image is simply mosaicked for concealment. However, the object (person H1) obtained from the photographed image may also be concealed while reflecting the features obtained from the photographed image. Further, the object (human H1) obtained from the photographed image is concealed while reflecting the features obtained from the photographed image, and the specific part (face of the person H2) is simply mosaicked or the like. May be.

The features obtained from the captured image are, for example, the orientation of the face, the line of sight, the facial expression, the age group, the gender, etc. when the specific part is the face. The features obtained from the captured image are, for example, age, gender, etc. when the object including the specific part is a person. The features obtained from the captured image are, for example, when the object including a specific part is a vehicle, the type of vehicle (automobile, motorcycle, etc.), the type according to the purpose of the automobile (passenger car, truck, bus, taxi, etc.), Body type (sedans, minivans, station wagons, etc.).

In the example shown in FIG. 7, CG reflecting the facial expression (smile) obtained from the captured image is used. In addition to the display, for example, when it is detected that the person H1 is a girl, the face of the girl in the captured image may be replaced with the CG of the smiling girl. Machine learning algorithms such as DNN, template matching, and the like may be used to detect features obtained from captured images. Further, the features obtained from the captured image may be detected by adding information obtained from a sensor other than the camera such as a biological sensor.

Further, the concealment unit 322 may conceal at least one of the object (including the specific portion) and the specific portion in the captured image by decryptively encrypting the object (including the specific portion). According to this, when a special situation occurs, for example, it is necessary to confirm the personal information in the captured image in order to solve the case, the encrypted captured image is decrypted. It becomes possible to confirm personal information.

For example, the vehicle device 1 performs encryption, and the data collection device 2 performs decryption when a predetermined condition is satisfied. The predetermined condition is, for example, when permission is obtained from the occupant of the vehicle device 1. As the encryption method, for example, a known common key cryptography, public key cryptography, or a hybrid method of common key cryptography and public key cryptography may be used. In the hybrid method, for example, public key cryptography that uses a public key and a private key when delivering a common key is used, and a common key is used for encrypting image data. The vehicle device 1 may store the data before encryption or may store the data after encryption.

(2-2. Second embodiment)
Next, the image processing apparatus according to the second embodiment will be described. In the description of the image processing apparatus of the second embodiment, the description of the portion overlapping with the first embodiment will be omitted unless it is particularly necessary to explain.

[2-2-1. Image processing device configuration]
FIG. 8 is a diagram showing a configuration of an image processing device 3A according to a second embodiment of the present invention. Note that FIG. 8 shows only the components necessary for explaining the features of the image processing apparatus 3A of the second embodiment, and the description of general components is omitted. Further, FIG. 8 also shows a camera 4, which is a component different from the image processing device 3A, in order to facilitate understanding. As shown in FIG. 8, the image processing device 3A includes an acquisition unit 31, a control unit 32A, and a storage unit 33A.

Since the acquisition unit 31 is the same as that of the first embodiment, the description thereof will be omitted.

The control unit 32A is a controller that comprehensively controls the entire image processing device 3A as in the first embodiment. The control unit 32A is configured as a computer including, for example, a CPU, a RAM, a ROM, and the like. However, the control unit 32A has a function different from that of the first embodiment. This difference will be described later.

The storage unit 33A also has the same configuration as that of the first embodiment, but is different from the first embodiment in that the first trained model 331 and the second trained model 332 are stored. The first trained model 331 is the same as the trained model 331 of the first embodiment, and is used for object detection. The second trained model 332 is used for image segmentation described later. The second trained model 332 can be obtained by learning by a machine learning algorithm such as DNN.

The detection unit 321 and the concealment unit 322 and the image segmentation unit 323 shown in FIG. 8 are realized by the CPU of the control unit 32A executing arithmetic processing according to a program stored in the storage unit 33A. It is a function of. In other words, the image processing device 3A includes a detection unit 321, a concealment unit 322A, and an image segmentation unit 323.

In addition, at least one of each unit 321, 322A, 323 of the control unit 32A may be configured by hardware such as ASIC, FPGA, and GPU. Further, each part 321, 322A, 323 is a conceptual component. The functions executed by one component may be distributed to a plurality of components, or the functions of the plurality of components may be integrated into one component.

Since the configuration of the detection unit 321 is the same as that of the first embodiment, the description thereof will be omitted. The detection unit 321 uses the first trained model 331 to detect a predetermined object including a specific portion that needs to be concealed and a specific portion of the predetermined object from the captured image.

The image segmentation unit 323 performs image segmentation that labels the meaning of each pixel of the captured image acquired by the acquisition unit 31. The image segmentation unit 323 performs image segmentation on the captured image using the second trained model 332 stored in the storage unit 33A. Image segmentation classifies a plurality of elements contained in a captured image.

In the present embodiment, as an example, by image segmentation, segmentation is performed at a level of segmenting an object containing a specific part with respect to elements such as other objects, and at a level of classifying each element constituting the object including the specific part. No segmentation is done. By image segmentation, it is possible to extract the area occupied by an object including a specific part in the image.

In the present embodiment, the object detection by the detection unit 321 and the image segmentation by the image segmentation unit 323 are performed in parallel. Further, it is preferable that the image segmentation unit 323 performs image segmentation by the second trained model 332 using a low-resolution image obtained by reducing the resolution of the captured image acquired from the acquisition unit 31. As a result, the processing load of image segmentation in the image segmentation unit 323 can be reduced. The low-resolution image used for object detection by the detection unit 321 and the low-resolution image used for image segmentation may have the same resolution, but may be different from each other. For example, a low resolution image used for image segmentation may have a lower resolution than a low resolution image used for object detection.

Similar to the first embodiment, the concealment unit 322A has a first concealment process targeting a specific part and a second concealment process targeting an object including the specific part based on the detection result of the detection unit 321. Switch between and. The first concealment process performed by the concealment unit 322A is the same as that of the first embodiment. However, the second concealment process performed by the concealment unit 322A is different from that of the first embodiment.

In the second embodiment, the second concealment process is a process of converting a region of an object (including a specific part) obtained by image segmentation into an image in which the characteristics of the object are reduced. The image in which the characteristics of the object are reduced is, for example, a filled image, a blurred image, a mosaic image, or the like. According to such a configuration, when concealing an object including a specific part, it is possible to conceal the object while leaving the shape of the object as much as possible. That is, it is possible to appropriately conceal personal information while reducing the amount of information reduced from the captured image by the concealment process. In the present embodiment, it can be interpreted that the second concealment process performs the concealment process while reflecting the characteristics of the object (including the specific part) obtained from the captured image.

The range of concealment performed by the second concealment process may or may not completely match the area of the object (including a specific part) obtained by image segmentation. .. The range of concealment performed by the second concealment process may be slightly wider than the area of the object obtained by, for example, image segmentation.

[2-2-2. Image processing device operation example]
FIG. 9 is a flowchart showing an operation example of the image processing device 3A according to the second embodiment of the present invention. Since the operation of the image processing device 3A of the second embodiment is substantially the same as the operation of the image processing device 3 of the first embodiment, the differences will be mainly described.

First, the acquisition unit 31 acquires a captured image from the camera 4 (step S1). Using the acquired captured image, the object detection process (detection of a predetermined object and a specific part) by the detection unit 321 and the image segmentation by the image segmentation unit 323 are performed in parallel (step S2A).

FIG. 10 is a diagram illustrating the result of image segmentation for the captured image shown in FIG. Road R, wall W, and person H are classified by image segmentation. In the example shown in FIG. 10, each of the classified elements R, W, and H is distinguished by changing the fill color. The classification of each element may be distinguished by, for example, changing the type of hatching, instead of changing the color of the fill.

When the object detection by the detection unit 321 and the image segmentation by the image segmentation unit 323 are completed, it is determined whether or not a predetermined object has been detected by the detection unit 321 (step S3). When a predetermined object is not detected (No in step S3), the process shown in the flowchart in FIG. 3 is temporarily terminated. When a predetermined object is detected (Yes in step S3), it is determined whether or not a specific portion included in the predetermined object is detected by the detection unit 321 (step S4). In step S4, when a plurality of predetermined objects are detected in step S2, it is determined whether or not a specific portion is detected for each predetermined object.

When a specific part is detected (Yes in step S4), the first concealment process is performed on the specific part (step S5). When the specific part is not detected (No in step S4), the second concealment process is performed on the object including the specific part (step S6). The second concealment process is executed based on the result of image segmentation.

FIG. 11 is a diagram illustrating the result of concealment processing when the captured image shown in FIG. 4 is input in the image processing device 3A of the second embodiment. In this example, the detection result of the detection unit 321 is the same as the result shown in FIG. 5 of the first embodiment. That is, the person H1 is detected as a person, but the face, which is a specific part, cannot be detected. For the person H2, the person and the face can be detected.

In FIG. 11, the second concealment process is performed on the person H1. That is, with respect to the captured image acquired by the acquisition unit 31, the region of the person H1 obtained by image segmentation is made into an image (filled image in this example) in which the characteristics of the person are reduced. As a result, the personal information of the person H1 is kept secret while leaving the information such as the size and posture of the person H1.

In this example, in detail, an image generated during image segmentation (see FIG. 10) is applied to an image in which human features are reduced. That is, the image generated at the time of image segmentation is applied to the image in which the characteristics of the object (including a specific part) are reduced. According to this, it is possible to obtain an image in which personal information is concealed by using an image composition method such as alpha blending.

In FIG. 11, the first concealment process is performed on the person H2. In the example shown in FIG. 11, the region of the specific portion (face) detected by the detection unit 321 is replaced with CG. However, for concealment of a specific part, for example, painting, blurring, mosaic, etc. may be used.

<3. Points to note>
The various technical features disclosed herein can be modified in addition to the above embodiments without departing from the gist of the technical creation. That is, it should be considered that the above embodiments are exemplary in all respects and are not restrictive, and the technical scope of the present invention is not the description of the above embodiments but the claims. It is shown and should be understood to include all modifications that fall within the meaning and scope of the claims. In addition, a plurality of embodiments and modifications shown in the present specification may be appropriately combined and implemented to the extent possible.

3, 3A ... Image processing device 321 ... Detection unit 322, 322A ... Concealment unit 323 ... Image segmentation unit

Claims (7)

An object including a specific part that needs to be concealed to protect personal information and a detection unit provided so as to be able to detect the specific part from the acquired captured image.
Based on the detection result of the detection unit, the concealment unit that performs the concealment processing for concealing the captured image, and the concealment unit.
With
The image processing device is different in the case where both the object and the specific portion are detected and the case where only the object is detected among the object and the specific portion. The concealment process is
When both the object and the specific part are detected, this is the first concealment process for concealing the specific part.
The image processing apparatus according to claim 1, which is a second concealment process for concealing the object when only the object is detected among the object and the specific portion. Further, an image segmentation unit for performing image segmentation for labeling the meaning of each pixel of the acquired captured image is provided.
The image processing apparatus according to claim 2, wherein the second concealment process is a process of converting a region of the object obtained by the image segmentation into an image in which the characteristics of the object are reduced. The image processing apparatus according to claim 3, wherein the image generated at the time of the image segmentation is applied to the image in which the characteristics of the object are reduced. Any of claims 1 to 4, wherein the concealment unit performs the concealment processing on at least one of the object and the specific portion obtained from the photographed image while reflecting the features obtained from the photographed image. The image processing apparatus according to item 1. The concealment unit according to any one of claims 1 to 4, wherein the concealment unit performs the concealment by decryptively encrypting at least one of the object and the specific portion in the captured image. Image processing device. A detection process that attempts to detect an object containing a specific part that needs to be concealed to protect personal information and the specific part from the acquired captured image, and
Based on the detection result of the detection step, the concealment step of performing the concealment processing for concealment of the captured image, and
With
The image processing method is different from the case where both the object and the specific portion are detected and the case where only the object is detected among the object and the specific portion.

Images