JP6206542B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  Conventionally, there is a case where it is desired to publish or record an image in which it is difficult to identify a person even if a person is seen but the person is seen. Therefore, a technique for performing mosaic processing or mask processing on a person's face so as not to impair privacy is known (see, for example, Patent Document 1).

JP 2000-322660 A

  However, in the case of the above-mentioned Patent Document 1 and the like, not only image processing such as mosaic processing or mask processing is required for an image to be disclosed or recorded, but also an unnatural image due to locally applied mosaic or mask. There is a problem that the appearance is deteriorated.

  The present invention has been made in view of such problems, and an object of the present invention is to provide an image processing apparatus capable of executing processing related to an image according to a desired privacy level without being unnatural. An image processing method and program are provided.

One embodiment of the present invention for solving the above problems is as follows.
An image processing apparatus,
Calculating means for calculating a privacy level indicating a degree of difficulty of identifying that a person's face included in the image is a face of a specific person;
Determination means for determining whether or not the degree of privacy calculated by the calculation means is higher than a predetermined determination value ;
Control means for controlling execution of predetermined processing relating to an image when the determination means determines that the degree of privacy is higher than a predetermined determination value ;
It is characterized by having.

  According to the present invention, it is possible to execute a process related to an image according to a desired privacy level without being unnatural.

It is a block diagram which shows schematic structure of the image processing apparatus of Embodiment 1 to which this invention is applied. 3 is a flowchart illustrating an example of an operation related to automatic photographing processing by the image processing apparatus of FIG. 1. 3 is a flowchart illustrating an example of an operation related to privacy level calculation processing by the image processing apparatus of FIG. 1. It is a flowchart which shows the continuation of the privacy level calculation process of FIG. 2 is a flowchart illustrating an example of an operation related to manual imaging processing by the image processing apparatus in FIG. 1. It is a figure for demonstrating a privacy degree calculation process. It is a figure for demonstrating a privacy degree calculation process. It is a figure for demonstrating a privacy degree calculation process. It is a figure for demonstrating a privacy degree calculation process. It is a figure which shows typically an example of the display mode of the image by the image processing apparatus of FIG. 3 is a flowchart illustrating an example of an operation related to a determination value setting process by the image processing apparatus in FIG. 1. It is a block diagram which shows schematic structure of the image processing apparatus of Embodiment 2 to which this invention is applied. 13 is a flowchart illustrating an example of an operation related to an image acquisition process by the image processing apparatus of FIG. 13 is a flowchart illustrating an example of an operation related to privacy level calculation processing by the image processing apparatus in FIG. 12. It is a flowchart which shows the continuation of the privacy level calculation process of FIG.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

[Embodiment 1]
FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus 100 according to the first embodiment to which the present invention is applied.
As shown in FIG. 1, the image processing apparatus 100 according to the first embodiment specifically includes a central control unit 1, a memory 2, an imaging unit 3, a signal processing unit 4, a motion detection unit 5, and an operation. A control unit 6, an image processing unit 7, an image recording unit 8, a display unit 9, a communication control unit 10, and an operation input unit 11 are provided.
The central control unit 1, the memory 2, the imaging unit 3, the signal processing unit 4, the motion detection unit 5, the operation control unit 6, the image processing unit 7, the image recording unit 8, the display unit 9, and the communication control unit 10 They are connected via line 12.

  In addition, the image processing apparatus 100 may include a mobile terminal used in a mobile communication network such as a mobile phone or a smartphone having an imaging function, or a communication terminal such as a PDA (Personal Data Assistants). You may comprise from the digital camera etc. which comprise a communication function.

  The central control unit 1 controls each unit of the image processing apparatus 100. Specifically, although not shown, the central control unit 1 includes a CPU (Central Processing Unit) and the like, and performs various control operations according to various processing programs (not shown) for the image processing apparatus 100.

  The memory 2 is composed of, for example, a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores data processed by the central control unit 1 or the operation control unit 6. For example, the memory 2 temporarily stores a reference face image F (details will be described later) captured by the automatic photographing process.

  The imaging unit (imaging means) 3 captures a predetermined subject (for example, a person) and generates a frame image. Specifically, the imaging unit 3 includes a lens unit 3a, an electronic imaging unit 3b, and an imaging control unit 3c.

The lens unit 3a includes, for example, a plurality of lenses such as a zoom lens and a focus lens, and a diaphragm for adjusting the amount of light passing through the lens. Further, the lens unit 3a is arranged so as to be exposed on the same side (subject side) as the display panel 9b so that self-portrait can be performed.
The electronic imaging unit 3b is configured by an image sensor (imaging device) such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), for example. Then, the electronic imaging unit 3b converts the optical image that has passed through the various lenses of the lens unit 3a into a two-dimensional image signal.
For example, the imaging control unit 3c scans and drives the electronic imaging unit 3b with a timing generator or a driver, and converts the optical image that has passed through the lens unit 3a into a two-dimensional image signal at predetermined intervals by the electronic imaging unit 3b. Then, the frame image is read out for each screen from the imaging region of the electronic imaging unit 3b and is output to the signal processing unit 4.

  The signal processing unit 4 performs various types of image signal processing on the analog value signal of the frame image transferred from the electronic imaging unit 3b. Specifically, for example, the signal processing unit 4 adjusts the gain for each RGB color component with respect to the analog value signal of the frame image, samples and holds it by a sample hold circuit (not shown), and performs an A / D converter. (Not shown) is converted into digital data, and a color process circuit (not shown) performs color process processing including pixel interpolation processing and γ correction processing to obtain a digital luminance signal Y and color difference signals Cb, Cr (YUV data). ) Is generated. The signal processing unit 4 outputs the generated luminance signal Y and color difference signals Cb and Cr to the memory 2 used as a buffer memory.

The motion detection unit 5 detects the motion of the image processing apparatus 100.
That is, the motion detection unit 5 includes, for example, a triaxial angular velocity sensor that detects rotational angular velocities about the three axes (roll axis, pitch axis, and yaw axis) that are orthogonal to each other. Yes. Then, the motion detection unit 5 outputs the signals sequentially detected by the triaxial angular velocity sensor to the operation control unit 6 as motion information, for example, when photographing a subject.

The operation control unit 6 includes a first image acquisition unit 6a, a detection processing unit 6b, a first calculation unit 6c, a first determination unit 6d, and an imaging control unit 6e.
In addition, although each part of the operation control part 6 is comprised from the predetermined logic circuit, for example, the said structure is an example and is not restricted to this.

The first image acquisition unit 6 a acquires frame images sequentially captured by the imaging unit 3.
Specifically, the first image acquisition unit 6 a sequentially acquires, from the memory 2, image data of frame images related to live view images that are sequentially captured by the imaging unit 3 and sequentially generated by the signal processing unit 4.

The detection processing unit 6b detects a face, a constituent part of the face, and a line of sight from the frame image acquired by the first image acquisition unit 6a.
That is, the detection processing unit 6b performs face detection processing for each frame image sequentially captured by the imaging unit 3 and sequentially acquired by the first image acquisition unit 6a, and detects a face region including the face of the person who is the subject. In addition, the constituent parts of the face such as eyes and mouth are detected from the face area detected by the face detection process. Further, the detection processing unit 6b performs a gaze detection process for each frame image sequentially captured by the imaging unit 3 and sequentially acquired by the first image acquisition unit 6a to detect the gaze of the person who is the subject.
Further, the detection processing unit 6b detects the entire body of a person who is a subject for each frame image sequentially captured by the imaging unit 3 and sequentially acquired by the first image acquisition unit 6a, and specifies the shape thereof. It is possible to determine whether the person has deviated from the frame image due to a change in the shape of the identified person, or whether the person has changed the body orientation within the frame image.
The face detection process, the component part detection process, the line-of-sight detection process, and the process for specifying the shape of a person are known techniques, and thus detailed description thereof is omitted here. For example, AAM (Active Appearance Model) may be used in the component part detection process. In the line-of-sight detection process, for example, the eye position is detected, and the line of sight is detected from the area ratio of the white eyes on both sides of the black eye. You may do it.

The first calculation unit (calculation means) 6c calculates a privacy level.
Here, the privacy level indicates the degree of difficulty of identifying that the face of a person included in an image (for example, a frame image or the like) is the face of a specific person. An image that can be easily identified as having a lower privacy level has a relatively low privacy level, while an image that is difficult to identify as a face of a specific person has a relatively higher privacy level. Specifically, the privacy level is the state of a person's face included in the image and the constituent parts of the face (eg, eyes, mouth, etc.) (eg, angle, size, whether or not it is hidden, color, etc.) , And changes depending on the state of an external shield (for example, sunglasses, a mask, etc.) that hides the face of the person and the constituent parts of the face.
That is, a state in which the face of the person included in the frame image can be identified as a face of a specific person, for example, the face of the person can be detected, and the face and line of sight of the person are in the front direction (predetermined direction). Using the state as the reference state, the relative change in the entire face of the person and the constituent parts of the face from the reference state, the change in the ratio of the person's face or the constituent parts of the face being hidden, the frame image Privacy based on changes in the size of the person's face or the imaging distance to the person's face, changes in the external shield that hides the person's face or parts of the face, changes in the color of the person's face, etc. The degree changes.

Here, relative changes in the entire face of the person and the constituent parts of the face are, for example, that the person who is the subject rotates the face around a predetermined axis (for example, the yaw axis or the pitch axis; see FIG. 6). An action, an action in which a person rotates the entire body around the yaw axis to rotate the face around the yaw axis, an action to move the line of sight away from the front direction, and the lens unit 3a of the image processing apparatus 100 This may be caused by performing an operation of displacing the image processing apparatus 100 in the vertical direction or the horizontal direction so that the optical axis direction is inclined with respect to the front direction of the human face. Note that the operation of rotating the person's face around the roll axis and the operation of rotating the image processing apparatus 100 around the optical axis direction of the lens unit 3a are excluded because they have little influence on the face detection processing. However, for example, since the privacy level of an image can be set more finely, these operations may be included.
In addition, a change in the ratio of the person's face or the constituent parts of the face being hidden may be caused by, for example, the person who is the subject by his / her own hand or hair, or an external shield (for example, sunglasses or mask). This may be caused by performing an operation of hiding the entire face or a constituent part of the face (for example, eyes and mouth).
The change in the size of the face of the person included in the frame image or the imaging distance to the person's face includes, for example, an operation for adjusting the physical distance between the person who is the subject and the image processing apparatus 100, This may be caused by performing an operation for adjusting the zoom magnification (focal length) of 3 or an operation for designating a region to be trimmed from the captured image.
In addition, a change in an external shield that hides the face of a person or a constituent part of the face may be caused by, for example, the type of the external shield that hides eyes, such as glasses, sunglasses, an eye patch, or an eye patch. Moreover, even if it is a mask, it may arise by the difference in the structural part of the face to conceal, such as a mask used for the mouth and a mask used for the nose.
The change in the color of the person's face may be, for example, the face turned black due to backlight or the white due to overexposure depending on the light rays applied to the subject person and the exposure state of the image processing apparatus 100. This may occur when the face is painted or when the face is painted with a different color from the skin color.

  Specifically, the first calculation unit 6c detects a human face based on the detection result of the detection processing unit 6b on the frame images sequentially captured by the imaging unit 3 and sequentially acquired by the first image acquisition unit 6a. A state in which the person's face and line of sight are in the front direction (predetermined direction) is specified as the reference state. Then, the first calculation unit 6c hides the face of the person or the constituent part of the face relative to the frame image (reference face image F) in the reference state, relative changes in the entire face of the person and constituent parts of the face. Change of the ratio, the size of the face of the person included in the frame image or the change of the imaging distance to the face of the person, the change of the external shield that hides the face of the person or the constituent part of the face, the face of the person Detecting a change in color and calculating the privacy level.

For example, the first calculation unit 6c includes the shape of the face detection frame corresponding to the person's face area included in the reference face image F and the person's face area included in the frame image sequentially acquired by the first image acquisition unit 6a. Is compared with the shape of the face detection frame corresponding to, and a change in the rotation angle about a predetermined axis (for example, a yaw axis or a pitch axis) of the face of the person who is the subject with respect to the reference state is specified. That is, for example, when the face of a person rotates around the yaw axis, the face detection frame becomes a vertically long rectangle, and when the face of a person rotates around the pitch axis, the face detection frame becomes a horizontally long rectangle. Then, the first calculation unit 6c calculates, for example, the center axis and rotation angle of the rotation of the human face from the change in the shape of the face detection frame, and the rotation angle about the predetermined axis of the human face with respect to the reference state The amount of change or rate of change is specified. Here, even when the person rotates the entire body about 180 ° about the yaw axis, that is, when the face is turned backward, the first calculation unit 6c causes the detection processing unit 6b to The rotation of the whole body is detected, and the change in the rotation angle can be specified from the detection result. The first calculation unit 6c calculates the rotation angle of the image processing device 100 based on the motion information output from the motion detection unit 5, and the optical axis of the lens unit 3a of the image processing device 100 with respect to the reference state. The change amount or change rate of the direction of the person's face relative to the front may be specified.
In addition, for example, the first calculation unit 6c compares the person's line of sight included in the reference face image F with the person's line of sight included in the frame image sequentially acquired by the first image acquisition unit 6a, and the reference state The change in the line of sight of the person who is the subject with respect to is identified.
The first calculation unit 6c then changes or changes the rotation angle around a predetermined axis of the person's face with respect to the specified reference state, and the optical axis of the lens unit 3a of the image processing apparatus 100 with respect to the reference state. The degree of privacy is calculated based on the amount or rate of change of the orientation of the person's face relative to the front and the change in the line of sight of the person who is the subject relative to the reference state. For example, when the face of a person is rotated to the left or right around the yaw axis, or the line of sight is moved to the left or right, as shown in FIG. The privacy level is calculated with the correlation that the privacy level is the lowest when the position is moved to the left (or right), and the privacy level gradually increases as the face direction is rotated to the left (or right). Further, for example, when the face of a person is rotated up and down around the pitch axis or the line of sight is moved up and down, as shown in FIG. The degree of privacy is the lowest when only the line of sight is moved down (or up), and the degree of privacy is calculated with a correlation in which the degree of privacy gradually increases as the face direction is rotated down (or up). To do. Further, for example, when the face of a person is rotated left and right and up and down around the yaw axis and the pitch axis, and the line of sight is moved up and down and up and down, as shown in FIG. The privacy level is the lowest when the face direction is front and only the line of sight is moved to the lower left (or lower right, upper left, upper right), and the face direction is rotated to the lower left (or lower right, upper left, upper right) The privacy level is calculated with a correlation in which the privacy level gradually increases as the process is continued.

Note that the first calculation unit 6c may individually set the privacy level without having a correlation with respect to the rotation angle about the predetermined axis of the person's line of sight and the face.
In addition, the above-described method for identifying the change in the rotation angle around the predetermined axis of the person's face is an example, and is not limited to this. For example, the face detection process supports the rotation angle of the face. A plurality of classifiers may be used, and the detection result of which classifier is detected may be used. Further, the above-described method for identifying the change in the line of sight is an example and is not limited to this, and can be arbitrarily changed as appropriate.

Further, for example, the first calculation unit 6c counts the number of constituent parts of the face detected from the reference face image F and the number of constituent parts of the face detected from the frame images sequentially acquired by the first image acquisition unit 6a. To identify a change in the ratio of the constituent parts of the person's face with respect to the reference state being hidden. Specifically, the first calculation unit 6c determines the composition of the person's face with respect to the reference state based on the change in the number of constituent parts of the face such as eyes and mouth detected from the face area detected in the face detection process. Identify the change in the ratio that the part is hidden.
And the 1st calculation part 6c calculates a privacy degree based on the change of the ratio by which the structural part of the person's face with respect to the specified reference | standard state is hidden. For example, as shown in FIG. 8A, the first calculation unit 6c has the lowest privacy level in a state where any one of the constituent parts of the face (for example, mouth) is hidden, The degree of privacy is calculated with a correlation in which the degree of privacy gradually increases as the number of constituent parts hidden among the constituent parts increases.
Note that the above-described method for identifying the change in the ratio of the constituent parts of the person's face being hidden is an example, and is not limited thereto, and can be arbitrarily changed as appropriate.

In addition, for example, the first calculation unit 6c configures the number of pixels constituting the face of the person included in the reference face image F and the face of the person included in the frame image sequentially acquired by the first image acquisition unit 6a. A change in the size of the face of the person included in the frame image with respect to the reference state is identified by comparing the number of pixels. That is, for example, as the physical distance between the person and the image processing apparatus 100 increases, the number of pixels constituting the face of the person included in the frame image decreases. For example, the first calculation unit 6c identifies a change in the size of the person's face included in the frame image with respect to the reference state from a change in the number of pixels constituting the face area including the face of the person. The first calculation unit 6c converts the imaging distance (subject distance) from the focal length of the imaging unit 3 to the person's face, and the size of the person's face included in the frame image with respect to the reference state from the change in the imaging distance. You may identify the change in height.
Then, the first calculation unit 6c calculates the privacy level based on the change in the size of the person's face included in the frame image with respect to the specified reference state. For example, as illustrated in FIG. 8B, the first calculation unit 6c has a lower privacy level as the face size of the person included in the frame image is larger, and a higher privacy level as the face size is larger. The degree of privacy is calculated by correlation.
Note that the above-described method for specifying the change in the size of the person's face is merely an example, and is not limited thereto, and can be arbitrarily changed as appropriate.

Further, for example, the first calculation unit 6c determines the external shielding object detected from the reference face image F and the type of the external shielding object detected from the frame image sequentially acquired by the first image acquisition unit 6a. In comparison, the change of the external shield with respect to the reference state is specified. Specifically, the first calculation unit 6c learns a large amount of face images with sunglasses from the face area detected in the face detection process, for example, if it is sunglasses. By learning a large amount of applied face images, external shields such as sunglasses and masks are detected from within the face area, and the change in the type of detected external shields results in the detection of external shields relative to the reference state. Identify changes.
And the 1st calculation part 6c calculates a privacy degree based on the change of the external shielding object with respect to the specified reference | standard state. For example, with reference to the shielding object table ST illustrated in FIG. 9A, the first calculation unit 6c calculates the privacy level according to the type of the external shielding object. In this shielding object table ST, the types of external shielding objects and the privacy level are stored in association with each other.
Note that the above-described method for identifying the change in the external shielding object is an example and is not limited to this. For example, a known object recognition technique is used to detect an object that is different from the constituent part of the face. It may be recognized and detected as an external shield, and can be arbitrarily changed as appropriate.
Further, the type of external shielding object to be detected is an example, and is not limited to this. For example, a hat or bandana that hides hair may be used.
Further, when a plurality of external shielding objects are detected, a higher privacy level may be calculated.

Further, for example, the first calculation unit 6c calculates the color of the person's face detected from the reference face image F and the color of the person's face detected from the frame images sequentially acquired by the first image acquisition unit 6a. In comparison, a change in the face color of the person relative to the reference state is specified. Specifically, the first calculation unit 6c measures the average RGB value of the skin color area excluding the component part having a color different from the skin color such as eyes and mouth from the face area detected in the face detection process, The difference between the R value G value B value of the person's face detected from the face image F and the R value G value B value of the person's face detected from the frame image sequentially acquired by the first image acquisition unit 6a. By adding the absolute values, a change in the color of the person's face is specified.
Then, the first calculation unit 6c calculates the privacy level based on the change in the face color of the person with respect to the specified reference state. For example, as illustrated in FIG. 9B, the first calculation unit 6 c increases the privacy as the difference between the face color of the person detected from the reference face image F and the face color of the person included in the frame image decreases. The degree of privacy is calculated with a correlation in which the degree of privacy is higher as the degree is lower and the difference is larger.
Note that the above-described method for identifying the change in the face color of a person is an example, and is not limited to this. For example, the person face color included in the frame image is detected from the reference face image F. The ratio of the area of the color area that differs by a predetermined value or more from the average RGB value of the color of the person's face is measured, and the privacy level is calculated according to the size of the area ratio. It can be changed arbitrarily.

  In addition, the first calculation unit 6c sequentially calculates the privacy level for each frame image related to the live view image that is sequentially captured by the imaging unit 3 and sequentially acquired by the first image acquisition unit 6a in the automatic shooting process, The calculated privacy level is sequentially output to the first determination unit 6d.

The first calculation unit 6c is configured to change the relative ratio of the entire face of the person and the constituent parts of the face from the reference state described above, the change of the ratio of the face of the person or the constituent parts of the face being hidden, The privacy level is determined based on at least one of a change in the size of the face of the person included in the frame image or an imaging distance to the person's face, a change in an external shield, and a change in the color of the person's face. What is necessary is just to calculate. Moreover, when using these plural as a reference | standard, the 1st calculation part 6c may be evaluated for every item separately, and you may make it calculate privacy degree comprehensively from the result.
The privacy level is calculated on the condition that the person's body is in the frame image. When the person's body is out of the frame image, the privacy level is not calculated and the automatic photographing process is not performed.

The first determination unit (determination unit) 6d determines whether or not the privacy level calculated by the first calculation unit 6c is higher than a predetermined determination value.
That is, the first determination unit 6d determines whether the privacy level sequentially calculated by the first calculation unit 6c is higher than the determination value for each live view image sequentially captured by the imaging unit 3 in the automatic shooting process. (Whether or not a predetermined condition is satisfied) is determined. Specifically, the first determination unit 6d first acquires from the memory 2 a desired privacy level that is designated in advance by the user and stored in the memory 2 as a determination value. Then, the first determination unit 6d sequentially acquires the privacy level sequentially calculated by the first calculation unit 6c, and determines whether or not the acquired privacy level is higher than the determination value.
The determination value may be set in advance as a default, for example, a value empirically obtained from a plurality of privacy levels calculated in the privacy level calculation process.

The imaging control unit (control unit) 6e controls the imaging unit 3 to execute a predetermined process related to the image.
That is, when the first determination unit 6d determines that the privacy level is higher than a predetermined determination value, the shooting control unit 6e controls the imaging unit 3 to capture a recording image. Specifically, the imaging control unit 6e is in a state where the privacy level sequentially calculated by the first calculation unit 6c is equal to or less than the determination value based on the determination result by the first determination unit 6d, for example, in the automatic imaging process. In response to a change to a higher state than the determination value, the imaging unit 3 outputs a recording image capturing instruction to cause the imaging unit 3 to capture the recording image.
Note that, for example, the imaging control unit 6e causes the imaging unit 3 to perform recording when the privacy level sequentially calculated by the first calculation unit 6c changes from a state higher than the determination value to a state equal to or lower than the determination value. An image may be taken. In other words, for example, when the privacy level of the live view image becomes too high, the user performs an operation that reduces the privacy level of the image (for example, an operation that the face direction approaches the front). In the process of decreasing the privacy level, the imaging control unit 6e may cause the imaging unit 3 to capture an image for recording based on the determination result by the first determination unit 6d.

  Further, the shooting control unit 6e controls the imaging unit 3 to take a recording image in response to a shooting instruction operation by the user. Specifically, for example, the shooting control unit 6e causes the imaging unit 3 to record an image for recording in response to a shooting instruction operation performed by the user via the operation input unit 11 described later in manual shooting processing. Is output to cause the imaging unit 3 to capture a recording image.

The image processing unit 7 encodes still image data generated by the signal processing unit 4 in a predetermined compression format (for example, JPEG format) to generate image data for recording a still image.
Further, the image processing unit 7 decodes the image data of the still image related to the display target read from the memory 2 or the image recording unit 8 according to a corresponding predetermined encoding method and outputs the decoded image data to the display unit 9. At this time, the image processing unit 7 may be transformed into a predetermined size (for example, VGA or full HD size) and output to the display unit 9 based on, for example, a display resolution of the display panel 9b described later.

The image recording unit 8 includes, for example, a non-volatile memory (flash memory) or the like, and records image data for recording a still image encoded by the image processing unit 7 in a predetermined compression format. Specifically, the image recording unit (recording unit) 8 acquires a recording image photographed by the photographing control unit 6e as the privacy image P in both the automatic photographing process and the manual photographing process, and The privacy level calculated in the privacy level calculation process before photographing the privacy image P is acquired, and the privacy level is recorded in association with the image data of the privacy image P as Exif (Exchangeable Image File Format) information.
The image recording unit 8 may be configured, for example, such that a recording medium (not shown) is detachable and controls reading of data from the loaded recording medium and writing of data to the recording medium.

  The display unit 9 includes a display control unit 9a and a display panel 9b.

  The display control unit 9a performs control to display a predetermined image in the display area of the display panel 9b based on image data of a predetermined size read from the memory 2 or the image recording unit 8 and decoded by the image processing unit 7. . Specifically, the display controller 9a includes a VRAM (Video Random Access Memory), a VRAM controller, a digital video encoder, and the like. The digital video encoder reads out the luminance signal Y and the color difference signals Cb and Cr decoded by the image processing unit 7 and stored in the VRAM (not shown) from the VRAM through the VRAM controller at a predetermined reproduction frame rate. Based on these data, a video signal is generated and output to the display panel 9b.

The display panel 9b displays an image taken by the imaging unit 3 in the display area based on the video signal from the display control unit 9a. Specifically, the display panel 9b displays the live view image while sequentially updating a plurality of frame images generated by photographing the subject by the imaging unit 3 at a predetermined frame rate in the still image shooting mode.
The display panel 9b displays an image recorded in the image recording unit 8 in the image reproduction mode. At this time, the display control unit 9a displays the privacy image P recorded in the image recording unit 8 on the display panel 9b as a display control unit in a classification or order based on the associated privacy level ( (See FIG. 10). For example, as shown in FIG. 10, the display control unit 9 a displays a plurality of privacy images P recorded in the image recording unit 8 for each predetermined privacy level (for example, 10, 30, 50, 70, 100, etc.). The images are classified as image groups, and the classified image groups are displayed on the display panel 9b. At this time, any one privacy image P may be displayed as a representative image Ps as a thumbnail for each image group corresponding to the privacy level.
For example, the display control unit 9a rearranges the plurality of privacy images P recorded in the image recording unit 8 according to the associated privacy level, and displays the privacy images P in the rearranged order. You may display on the panel 9b.

  Examples of the display panel 9b include a liquid crystal display panel and an organic EL (Electro-Luminescence) display panel. However, the display panel 9b is an example and is not limited thereto.

The communication control unit 10 transmits and receives data via the communication antenna 10a and a communication network (not shown).
That is, the communication antenna 10a is a predetermined communication method (for example, W-CDMA (Wideband Code Division Multiple Access) method, CDMA2000 method, etc.) adopted by the image processing apparatus 100 for communication with a wireless base station (not shown). It is an antenna capable of transmitting and receiving data compatible with GSM (Global System for Mobile Communications; registered trademark) system and the like. The communication control unit 10 transmits / receives data to / from the radio base station via the communication antenna 10a using a communication channel set in the communication method according to a communication protocol corresponding to a predetermined communication method. Specifically, the communication control unit (transmission means) 10 transmits the privacy image P photographed by the photographing control unit 6e to an external recording server (predetermined external device) S via the communication antenna 10a.

The recording server S is, for example, a server constituting a cloud and has a function of opening a Web page (for example, an image public page) on the Internet as a Web (World Wide Web) server. Then, for example, the recording server S receives various images transmitted from a communication terminal such as the image processing apparatus 100 and publishes them as contents on a Web page.
As a result, the content published on the web page established by the recording server S becomes viewable by the user of the communication terminal that can access the web page via the communication network.
The recording server S may have any configuration as long as it is configured from a computer connectable to a communication network, and detailed description thereof is omitted.

The communication network is, for example, a communication network that connects the image processing apparatus 100 to an external device such as the recording server S via a wireless base station, a gateway server (not shown), or the like. The communication network is a communication network constructed using a dedicated line or an existing general public line, and various line forms such as a WAN (Wide Area Network) and a LAN (Local Area Network) can be applied. Is possible.
The communication network includes, for example, various communication network networks such as a telephone line network, ISDN line network, dedicated line, mobile communication network, communication satellite line, CATV line network, IP network, VoIP (Voice over Internet Protocol). Includes gateways, internet service providers, etc.

  Note that the configuration of the communication control unit 10 described above is an example and is not limited thereto, and can be arbitrarily changed as appropriate. For example, although not illustrated, a wireless LAN module is mounted and an access point (Access The communication network may be accessible via the (Point).

  The operation input unit 11 is for performing a predetermined operation of the image processing apparatus 100. Specifically, the operation input unit 11 includes a shutter button related to a subject shooting instruction, a selection determination button related to a selection instruction for a shooting mode and a function, a zoom button related to a zoom amount adjustment instruction, and the like (all not shown). And a predetermined operation signal is output to the central control unit 1 in accordance with the operation of each button of the operation unit.

<Automatic shooting process>
Next, automatic photographing processing by the image processing apparatus 100 will be described with reference to FIG.
FIG. 2 is a flowchart illustrating an example of an operation related to the automatic photographing process.

As shown in FIG. 2, when imaging of the live view image of the subject by the imaging unit 3 is started, the signal processing unit 4 outputs a signal of an analog value of the frame image related to the live view image transferred from the electronic imaging unit 3 b. Are subjected to various image signal processing to generate digital image data (step S1). The signal processing unit 4 outputs the generated image data to the memory 2, and the memory 2 temporarily stores the input image data.
The first image acquisition unit 6a of the operation control unit 6 reads out and acquires the image data of the frame image related to the live view image from the memory 2 (step S2).

Next, the detection processing unit 6b performs line-of-sight detection processing on the frame image to be processed acquired by the first image acquisition unit 6a (step S3), and whether or not the line of sight of the person who is the subject is in front. Is determined (step S4).
If it is determined that the line of sight is not the front (step S4; NO), the first image acquisition unit 6a reads out and acquires image data of a new frame image related to the live view image from the memory 2 (step S5). ), The process returns to step S33. In step S3, the detection processing unit 6b performs line-of-sight detection processing on the new frame image acquired by the first image acquisition unit 6a in substantially the same manner as described above (step S3).
On the other hand, when it is determined in step S4 that the line of sight is the front (step S4; YES), the operation control unit 6 shifts to a standby state for automatic photographing (step S6).
In other words, if the line of sight of the person who is the subject is not in front (camera line of sight), the automatic shooting standby state is not entered. For example, in a state other than the camera line of sight that is not intended for shooting. It is possible to prevent the automatic shooting from being performed.

  Thereafter, the detection processing unit 6b performs detection processing on the frame images sequentially acquired by the first image acquisition unit 6a, and the first calculation unit 6c is configured to detect the human face based on the detection result by the detection processing unit 6b. And a state where the person's line of sight is in the front direction (predetermined direction) is specified as a reference state (step S7). Further, the first calculation unit 6 c outputs the frame image in the reference state to the memory 2 and temporarily stores it as the reference face image F. The temporarily stored reference face image F is assumed to be in a state where the face is directed in the front direction (predetermined direction).

  Next, the first image acquisition unit 6a reads out and acquires image data of a new frame image related to the live view image from the memory 2 (step S8), and the operation control unit 6 determines the privacy level of the new frame image. Privacy level calculation processing (see FIGS. 3 and 4) is performed (step S9; details will be described later).

Then, the first determination unit 6d of the operation control unit 6 determines whether or not the privacy level calculated in the privacy level calculation process in step S9 is higher than the determination value set in the determination value setting process ( Step S10). Specifically, the first determination unit 6d reads out and acquires a determination value from the memory 2, and determines whether or not the privacy level calculated in the privacy level calculation process is higher than the determination value.
If it is determined that the privacy level is not higher than the determination value (step S10; NO), the operation control unit 6 returns the process to step S8, and executes the subsequent processes. That is, in step S8, the first image acquisition unit 6a acquires image data of a new frame image, and performs privacy level calculation processing in step S9.

On the other hand, if it is determined in step S10 that the privacy level is higher than the determination value (step S10; YES), the imaging control unit 6e controls the imaging unit 3 to capture an image for recording the subject ( Step S11). Specifically, for example, the shooting control unit 6e sets a timer for automatically shooting the subject after a predetermined time has elapsed, and causes the imaging unit 3 to take an image when the predetermined time set by the timer has elapsed. Then, image data is generated by the signal processing unit 4. Then, the image processing unit 7 encodes the image data generated by the signal processing unit 4 in a predetermined compression format (for example, JPEG format) and generates image data of an image for recording.
Thereafter, the image recording unit 8 acquires the recording image from the image processing unit 7 as the privacy image P, acquires the privacy level calculated in the privacy level calculation process, and adds the privacy level to the image data of the privacy image P. Is associated and recorded as Exif information (step S12).
Thereby, the automatic photographing process is finished.

  In the above-described automatic photographing process, a single privacy image P is photographed based on the privacy level. However, this is not an example, and the privacy level is, for example, a predetermined level. A plurality of privacy images P that are higher than the determination value may be captured and recorded in the image recording unit 8, and the user may select a desired privacy image P from the recorded privacy images P. .

<Privacy level calculation process>
Next, privacy level calculation processing by the image processing apparatus 100 will be described with reference to FIGS. 3 and 4.
3 and 4 are flowcharts illustrating an example of an operation related to the privacy level calculation process.

As shown in FIG. 3, first, the detection processing unit 6b performs face detection processing on a frame image to be processed (for example, the new frame image acquired in step S8 in FIG. 2) (step S3). S21), it is determined whether or not a face area including the face of the person who is the subject has been detected (step S22).
Here, if it is determined that the person who is the subject is in the frame image but the face area is not detected (step S22; NO), for example, the person rotates the entire body about 180 ° around the yaw axis. Thus, it is considered that the face is facing backward, and the first calculation unit 6c calculates the privacy level as the highest value (step S23). Thus, the privacy level calculation process is terminated.

  On the other hand, if it is determined in step S22 that a face area has been detected (step S22; YES), the first calculation unit 6c starts wearing items such as sunglasses and a mask from the face area detected from the frame image to be processed ( An external shield) is detected (step S24), and it is determined whether or not a mounted product (external shield) is detected (step S25). Here, when it is determined that the wearing product (external shielding object) is detected (step S25; YES), the wearing product (external shielding object) is referred to with reference to the shielding object table ST (FIG. 9A). The point for calculating the privacy level is evaluated and determined in accordance with the type (step S26). On the other hand, if it is determined that the mounted product (external shielding object) is not detected (step S25; NO), the process of step S26 is skipped.

  Next, the first calculation unit 6c measures the color of the skin color area of the face area detected from the frame image to be processed, and the first calculation unit 6c acquires the reference face image F from the memory 2. Also for the reference face image F, the skin color detection process is performed to measure the color of the skin color area of the face area and calculate the difference between the measured skin colors (step S27). Then, it is determined whether or not a skin color different from the normal color having a calculated skin color difference equal to or greater than a predetermined value is detected (step S28). If it is determined that a skin color different from the normal color is detected (step S28) S28; YES), the point for calculating the privacy according to the skin color (difference in the calculated skin color) is evaluated and determined (step S29). On the other hand, if it is determined that a skin color different from normal is not detected (step S28; NO), the process of step S29 is skipped.

Next, the first calculation unit 6c specifies the number of pixels constituting the face area detected from the frame image to be processed as the face size (step S30). Further, the first calculation unit 6c acquires the reference face image F from the memory 2, and also specifies the number of pixels constituting the face area of the person's face for the reference face image F.
Then, the first calculation unit 6c identifies a change in the size of the face of the person included in the frame image with respect to the reference state from the change in the number of pixels constituting the face area, and changes the size of the specified face. Accordingly, the point for calculating the privacy level is evaluated and determined (step S31).

Next, the first calculation unit 6c determines the shape of the face detection frame corresponding to the face area of the person included in the reference face image F and the shape of the face detection frame corresponding to the face area of the person included in the new frame image. And the rotation center axis and rotation angle of the human face are calculated from the change in the shape of the face detection frame (step S32).
Then, the first calculation unit 6c determines whether or not the face direction of the person detected from the processing target frame image is the front (step S33).

If it is determined in step S33 that the face orientation is not front (step S33; NO), the first calculation unit 6c changes the rotation angle change amount around a predetermined axis of the person's face relative to the reference state or The rate of change is specified, and the point for calculating the privacy level is evaluated and determined according to the change in the specified rotation angle of the face (step S34).
On the other hand, if it is determined in step S33 that the face direction is the front (step S33; YES), the first calculation unit 6c skips the process of step S34.

Moving to FIG. 4, the detection processing unit 6 b detects the constituent parts of the face such as eyes and mouth from the face area detected by the face detection process in the frame image to be processed (step S <b> 35). The detection processing unit 6b also detects constituent parts of the face such as eyes and mouth from the face area for the reference face image F.
Then, the first calculation unit 6c evaluates the points for calculating the privacy degree according to the change in the number of the constituent parts of the face, that is, the change in the ratio of the constituent parts of the person's face with respect to the reference state. (Step S36).

Next, the detection processing unit 6b determines whether eyes are detected from the frame image to be processed (step S37).
If it is determined that eyes are detected (step S37; YES), the detection processing unit 6b performs a line-of-sight detection process on the frame image to be processed (step S38), and the line of sight of the person who is the subject is detected. Is determined to be front (step S39).

If it is determined in step S39 that the line of sight is not the front (step S39; NO), the first calculation unit 6c identifies a change in the line of sight of the person who is the subject relative to the reference state, and the line of sight of the identified person is determined. The points for calculating the privacy level are evaluated and determined according to the change (step S40).
On the other hand, if it determines with a line of sight being a front in step S39 (step S39; YES), the 1st calculation part 6c will skip the process of step S40.
If it is determined in step S37 that no eyes have been detected (step S37; NO), the processes in steps S38 to S40 are skipped.

And the 1st calculation part 6c is a point evaluation according to the change of the size of the face in step S31 as a result of the point evaluation according to the color of the skin in step S29 as a result of the point evaluation according to the wearing goods in step S26. As a result of the point evaluation according to the change in the rotation angle of the face in step S34, as a result of the point evaluation according to the change in the ratio in which the constituent parts of the face are hidden in step S36, the line of sight of the person in step S40 Based on the result of the point evaluation according to the change, the privacy level is calculated using a predetermined conversion formula (step S41). That is, the first calculation unit 6c has a relatively low privacy level for a frame image with a relatively small change in the person from the reference state, and a frame image with a relatively large change in the person from the reference state. The privacy level is calculated so that the privacy level is relatively high.
Note that the conversion formula for calculating the privacy level is used to calculate the privacy level by comprehensively evaluating all the evaluation items. For example, the evaluation item to be prioritized (for example, the orientation of the face) May be specified, and when the point evaluation for the evaluation item is not performed, the privacy level may be calculated from the result of the point evaluation for the other evaluation items.
Thus, the privacy level calculation process is terminated.

<Manual shooting process>
Next, manual photographing processing by the image processing apparatus 100 will be described with reference to FIG.
FIG. 5 is a flowchart illustrating an example of an operation related to manual imaging processing.

  As shown in FIG. 5, when imaging of the live view image of the subject by the imaging unit 3 is started, the signal processing unit 4 outputs an analog value signal of the frame image related to the live view image transferred from the electronic imaging unit 3b. Are subjected to various image signal processing to generate digital image data (step S13). The signal processing unit 4 outputs the generated image data to the memory 2, and the memory 2 temporarily stores the input image data.

Next, the central control unit 1 determines whether or not an imaging instruction (imaging instruction operation) by pressing the shutter button of the operation input unit 11 has been performed (step S14).
If it is determined that the shooting instruction operation has been performed (step S14; YES), the shooting control unit 6e controls the imaging unit 3 to shoot an image for recording the subject (step S15). Specifically, for example, the imaging control unit 6e causes the imaging unit 3 to image a subject and causes the signal processing unit 4 to generate image data. Then, the image processing unit 7 encodes the image data generated by the signal processing unit 4 in a predetermined compression format (for example, JPEG format) and generates image data of an image for recording.

  Subsequently, the first image acquisition unit 6a of the operation control unit 6 reads out and acquires the image data of the frame image related to the live view image temporarily stored from the memory 2 (step S16), and the operation control unit 6 Performs the above-described privacy level calculation processing (see FIGS. 3 and 4) (step 17).

Thereafter, the image recording unit 8 acquires the recording image from the image processing unit 7 as the privacy image P, acquires the privacy level calculated in the privacy level calculation process, and adds the privacy level to the image data of the privacy image P. Is associated and recorded as Exif information (step S18).
Thus, the manual photographing process is finished.

As described above, according to the image processing apparatus 100 of the first embodiment, the face of a person included in the frame image captured by the imaging unit 3 that is one type of image is identified as the face of a specific person. As a measure for determining whether or not the privacy level indicating the degree of difficulty satisfies the predetermined condition, the privacy level is calculated, and the calculated privacy level is determined to be higher than the predetermined determination value. In this case, as one of the controls for executing the predetermined processing relating to the image, the image pickup unit 3 is controlled so that a recording image (privacy image P) is captured. It eliminates the need for image processing such as processing and mask processing, and does not deteriorate the appearance by applying a mosaic or mask locally, so that a more natural process according to the desired privacy level is achieved. It can be acquired Ibashi image P.
Then, by sending a recording image (privacy image P) photographed on the basis of the privacy level to a predetermined external device such as the recording server S, it is possible to publish the image on the Web page as content.

  Further, the privacy level is sequentially calculated for each frame image sequentially captured by the imaging unit 3, and the privacy level is determined based on the determination result of whether the sequentially calculated privacy level is higher than a predetermined determination value. Since the image for recording is taken when triggered by a change from a state equal to or lower than the determination value to a state higher than the predetermined determination value, for example, the user himself changes the privacy level while capturing a live view image. By performing such an operation and adjusting the privacy level of each frame image, the privacy level is shifted to a state higher than a predetermined determination value, whereby an image corresponding to the desired privacy level can be acquired.

In addition, a reference state in which a person's face included in the frame image can be identified as a specific person's face (for example, a person's face can be detected and the person's face and line of sight are in a predetermined direction, etc. The degree of privacy can be calculated based on the relative change of the face of the person or the part constituting the face of the person. For example, the privacy level can be calculated based on the amount or rate of change of the rotation angle around a predetermined axis of the person's face, or the amount of change or rate of change of the orientation of the image processing apparatus 100 with respect to the person's face. it can. In other words, by using the relative change of the person's face from the reference state or the part constituting the person's face, for example, the rotation angle of the face is determined in advance for each privacy degree, It is no longer necessary to rotate the face or adjust the orientation of the image processing apparatus 100, so that the usability can be improved and the expression form of the privacy image P acquired based on the privacy level can be varied. it can.
Furthermore, the privacy level is calculated based on the change in the ratio of the face of the person in the frame image or the part constituting the person's face being hidden, the size of the person's face included in the frame image, or the person's face The privacy level is calculated based on the change in the imaging distance to the face, the privacy level is calculated based on the change in the external shield that hides the face of the person or the face, or the human face Since the privacy level is calculated based on the color change, the privacy level calculation criteria may be increased, and the expression mode of the privacy image P acquired based on the privacy level may be made more diverse. it can.

  Further, by recording the privacy image P and the privacy level in association with each other, the privacy image P can be displayed on the display panel 9b in the classification or order based on the associated privacy level. Even when a plurality of privacy images P are recorded, the user-desired privacy image P can be easily selected, and usability can be improved.

Further, in the manual shooting process, a privacy level indicating a degree of difficulty of identifying that the face of the person included in the frame image is a specific person's face is calculated, and the calculated privacy level is used. As one of the controls for executing the predetermined processing relating to the image, since the control is performed to record the image in association with the image captured as the recorded image in accordance with the imaging instruction from the user, it is possible to respond when the recorded image is called up and displayed. It is possible to display the attached privacy level together and output a recorded image in which the privacy level is associated with the external device. Then, as one of determining whether or not the privacy level satisfies a predetermined condition, the user or the external device refers to the privacy level associated with the displayed or output recorded image, and the privacy level When it is determined that is higher than a predetermined determination value, it can be determined whether or not to be a disclosure target, or a disclosure process can be performed.
Further, as one of the controls for executing the predetermined processing relating to the image, by displaying the calculated privacy degree together with the calculated display during the review display for displaying the image captured according to the photographing instruction by the user for a predetermined time, It is possible to determine whether or not the user is to publish or record, and to publish or record the captured image according to a predetermined operation during review display.

In the embodiment described above, the determination value of the privacy level may be automatically set.
The determination value setting process performed by the image processing apparatus 100 will be described below with reference to FIG.
FIG. 11 is a flowchart illustrating an example of an operation related to the determination value setting process.

  The determination value setting process is a process performed before the above-described automatic photographing process, and is performed in a state where a mode for setting a determination value used in the automatic photographing process is selected.

As shown in FIG. 11, when imaging of a live view image of a subject by the imaging unit 3 is started, the signal processing unit 4 outputs a signal of an analog value of a frame image related to the live view image transferred from the electronic imaging unit 3b. Are subjected to various image signal processing to generate digital image data (step S51). The signal processing unit 4 outputs the generated image data to the memory 2, and the memory 2 temporarily stores the input image data.
The first image acquisition unit 6a of the operation control unit 6 reads out and acquires the image data of the frame image related to the live view image from the memory 2 (step S52).

Next, the detection processing unit 6b performs face detection processing on the frame image acquired by the first image acquisition unit 6a to detect a face area including the face of the person who is the subject, and also detects line-of-sight detection processing. And the line of sight of the person who is the subject is detected (step S53).
Subsequently, the first calculation unit 6c is capable of detecting a person's face based on the results of the face detection process and the line-of-sight detection process by the detection processing unit 6b, and a reference in which the person's face and line of sight are in the front direction. It is determined whether or not the state has been specified (step S54).

If it is determined in step S54 that the reference state is not specified (step S54; NO), the first image acquisition unit 6a reads out image data of a new frame image related to the live view image from the memory 2. Obtain (step S55), and the process returns to step S53. In step S53, the detection processing unit 6b performs face detection processing and line-of-sight detection processing on the new frame image acquired by the first image acquisition unit 6a in substantially the same manner as described above (step S53). .
On the other hand, when it is determined in step S54 that the reference state has been specified (step S54; YES), the first image acquisition unit 6a reads the image data of the new frame image related to the live view image from the memory 2. Obtaining (step S56), the operation control unit 6 performs a privacy level calculation process (see FIGS. 3 and 4) for calculating the privacy level of the new frame image (step S57). Note that the image data of the frame image in the reference state (reference face image F) may be temporarily stored in the memory 2.

Then, the operation control unit 6 determines whether or not the privacy level calculated in the privacy level calculation process is set as a determination value used for determining the privacy level in the automatic shooting process (step S58). Specifically, for example, the display control unit 9a causes the display panel 9b to display a confirmation screen (not shown) for the privacy level calculated in the privacy level calculation process. Then, after confirming the privacy level by the user, the operation control unit 6 is calculated based on whether or not an instruction using the calculated privacy level as a determination value is input based on a predetermined operation of the operation input unit 11. It is determined whether or not the privacy level is used as a determination value.
If it is determined that the calculated privacy level is not used as the determination value (step S58; NO), the operation control unit 6 returns the process to step S56, and executes the subsequent processes. That is, in step S56, the first image acquisition unit 6a acquires image data of a new frame image, and performs privacy level calculation processing in step S57.
On the other hand, when it is determined in step S58 that the calculated privacy level is set as the determination value (step S58; YES), the operation control unit 6 sets the calculated privacy level as the determination value (step S59). Specifically, the first calculation unit 6c of the operation control unit 6 outputs the calculated privacy level to the memory 2, and the memory 2 temporarily stores the input privacy level as a determination value.
Thereby, the determination value setting process is terminated.

[Embodiment 2]
The image processing apparatus 200 according to the second embodiment will be described below with reference to FIG.
FIG. 12 is a block diagram showing a schematic configuration of an image processing apparatus 200 according to the second embodiment to which the present invention is applied.

As shown in FIG. 12, the image processing apparatus 200 according to the present embodiment includes a central control unit 1, a memory 2, an operation control unit 206, an image processing unit 7, an image recording unit 8, a display unit 9, A communication control unit 10 and an operation input unit 11 are provided.
The central control unit 1, the memory 2, the operation control unit 206, the image processing unit 7, the image recording unit 8, the display unit 9, and the communication control unit 10 are connected via a bus line 12.

  The image processing apparatus 200 according to the second embodiment has substantially the same configuration as that of the image processing apparatus 100 according to the first embodiment, except for those described in detail below, and detailed description thereof is omitted.

  The operation control unit 206 includes a second image acquisition unit 206a, a detection processing unit 6b, a second calculation unit 206c, a second determination unit 206d, and an acquisition control unit 206f.

The second image acquisition unit 206 a acquires a recorded image from the image recording unit 8.
Specifically, the second image acquisition unit 206a acquires, for example, a recorded image recorded in the image recording unit 8 as a processing target of an image acquisition process (described later). Note that when a plurality of recorded images are recorded in the image recording unit 8, for example, all the recorded images may be processed in the image acquisition process, or based on a predetermined operation of the operation input unit 11 by the user. Only the designated recorded image may be the processing target of the image acquisition process.

The second calculation unit (calculation unit) 206c calculates the privacy level.
In other words, the second calculation unit 206c determines that the face of the person included in the recorded image is the face of a specific person based on the detection result of the detection processing unit 6b on the recorded image acquired by the second image acquisition unit 206a. An identifiable state, for example, a state in which a person's face can be detected and the person's face and line of sight are in the front direction (predetermined direction) is set as a virtual reference state. Then, the second calculation unit 206c specifies the relative change in the entire face of the person and the constituent parts of the face from the virtual reference state, and calculates the privacy level.
That is, the second calculation unit 206c sets a virtual reference state for the recorded image acquired by the second image acquisition unit 206a, assuming that the person's face is facing the front and the line of sight is the front. . Then, the second calculation unit 206c performs a predetermined axis (for example, a yaw axis or a pitch axis; for example, the entire face of the person detected from the recorded image or a constituent part of the face with respect to the set virtual reference state; FIG. The virtual change of the rotation angle centered on (see) is specified. In addition, the second calculation unit 206c specifies a virtual change in the line of sight of the person detected from the recorded image with respect to the set virtual reference state.
The second calculation unit 206c then virtually changes the rotation angle around the predetermined axis of the person's face with respect to the specified virtual reference state, and the line of sight of the person who is the subject with respect to the virtual reference state. The privacy level is calculated based on the virtual change.

Further, the second calculation unit 206c calculates the privacy level based on the ratio of the face of the person or the part constituting the person's face being hidden.
In other words, the second calculation unit 206c specifies the number of constituent parts (for example, eyes and mouths) of the person's face detected from the recorded image acquired by the second image acquisition unit 206a. Then, the second calculation unit 206c, for example, the person who is the subject uses his / her hands and hair, or an external shield (for example, sunglasses and mask), etc. The ratio of the constituent parts of the human face to the number of the constituent parts of the face that would be specified if the operation of hiding the mouth or the like is not performed (for example, “3” in the case of both eyes and mouth) Based on this, the privacy level is calculated.

Further, the second calculation unit 206c calculates the privacy level based on the size of the person's face included in the recorded image or the imaging distance to the person's face.
That is, for example, the second calculation unit 206c calculates the number of pixels constituting the face area including the face of the person detected from the recorded image acquired by the second image acquisition unit 206a of the face of the person included in the recorded image. Specify as size. Then, the second calculation unit 206c calculates the privacy level based on the size of the person's face included in the specified recorded image. Also, the second calculation unit 206c acquires, for example, the focal length of the imaging unit 3 from the Exif information associated with the image data of the recorded image, converts it to the imaging distance to the person's face, and the privacy level. Is calculated.

In addition, the second calculation unit 206c calculates the privacy level based on a change in the external shielding object with respect to the reference state.
That is, the second calculation unit 206c compares the external shielding object detected from the reference face image F with the type of the external shielding object detected from the recorded image acquired by the second image acquisition unit 206a. Identify external shield changes relative to reference conditions. And the 2nd calculation part 206c calculates a privacy degree based on the change of the external shielding object with respect to the specified reference | standard state.
Note that the specific method of calculating the privacy level based on the change of the external shield with respect to the reference state by the second calculation unit 206c is substantially the same as the method by the first calculation unit 6c of the first embodiment, and here, Detailed description is omitted.

In addition, the second calculation unit 206c calculates the privacy level based on the change in the face color of the person with respect to the reference state.
That is, the second calculation unit 206c compares the color of the person's face detected from the reference face image F with the color of the person's face detected from the recorded image acquired by the second image acquisition unit 206a. The change in the color of the person's face relative to the reference state is specified. Then, the second calculation unit 206c calculates the privacy level based on the change in the face color of the person with respect to the specified reference state.
The specific method of calculating the privacy level based on the change in the face color of the person with respect to the reference state by the second calculation unit 206c is substantially the same as the method by the first calculation unit 6c of the first embodiment. Then, detailed explanation is omitted.

The second determination unit (determination unit) 206d determines whether or not the privacy level calculated by the second calculation unit 206c is higher than a predetermined determination value.
That is, the second determination unit 206d determines whether or not the privacy level calculated by the second calculation unit 206c for the recorded image is higher than the determination value in the image acquisition process. Specifically, the second determination unit 206d acquires a desired privacy level stored in the memory 2 as a determination value from the memory 2, and the privacy level calculated by the second calculation unit 206c is higher than the determination value. It is determined whether or not.

The acquisition control unit 206f acquires an image on which a predetermined process is executed.
That is, the acquisition control unit 206f acquires the recorded image whose privacy level is determined to be higher than the determination value by the second determination unit 206d as the privacy image P on which a predetermined process (for example, transmission process) is executed. .

<Image acquisition processing>
Next, image acquisition processing by the image processing apparatus 200 will be described with reference to FIG.
FIG. 13 is a flowchart illustrating an example of an operation related to the image acquisition process.

As shown in FIG. 13, the second image acquisition unit 206a reads the image data of any one of the recorded images from the image recording unit 8, and acquires it as a processing target of the image acquisition process (step S61).
Then, the operation control unit 206 performs a privacy level calculation process (see FIGS. 14 and 15) for calculating the privacy level of the acquired recorded image (step S62; details will be described later).

Then, the second determination unit 206d of the operation control unit 206 determines whether or not the privacy level calculated in the privacy level calculation process in step S62 is higher than the determination value (step S63). Specifically, the second determination unit 206d reads out and acquires the determination value from the memory 2, and determines whether or not the privacy level calculated in the privacy level calculation process is higher than the determination value.
Here, if it is determined that the privacy level is higher than the determination value (step S63; YES), the acquisition control unit 206f acquires the recording image to be processed as the privacy image P transmitted to the recording server S (step S63). S64).

Thereafter, the operation control unit 206 determines whether or not all the recorded images recorded in the image recording unit 8 have been processed as processing targets of the image acquisition process (step S65).
If it is determined in step S63 that the privacy level is not higher than the determination value (step S63; NO), the operation control unit 206 skips the process in step S64 and similarly records all the recorded images. Is processed as an image acquisition process target.

  If it is determined in step S65 that all the recorded images have not been processed as image acquisition processing targets (step S65; NO), the second image acquisition unit 206a receives a new recorded image from the image recording unit 8. Is read out and acquired as a processing target of the image acquisition process (step S66), and then the process returns to step S62. In step S62, the operation control unit 206 performs a privacy level calculation process for calculating the privacy level of the acquired new recorded image in substantially the same manner as described above (step S62).

  On the other hand, if it is determined in step S65 that all the recorded images have been processed as image acquisition processing targets (step S65; YES), the image acquisition processing is terminated.

<Privacy level calculation process>
Next, privacy level calculation processing by the image processing apparatus 200 will be described with reference to FIGS. 14 and 15.
14 and 15 are flowcharts illustrating an example of an operation related to the privacy level calculation process.

As shown in FIG. 14, first, the detection processing unit 6b performs face detection processing on a recording image to be processed (for example, the recording image acquired in step S51 of FIG. 13) (step S71). Then, it is determined whether or not a face area including a human face has been detected (step S72).
Here, if it is determined that a face area has been detected (step S72; YES), the second calculation unit 206c starts wearing items such as sunglasses and a mask (external shielding object) from the face area detected from the recorded image to be processed. ) Is detected (step S73), and it is determined whether or not a mounted product (external shield) is detected (step S74). Here, if it is determined that an attached product (external shield) has been detected (step S74; YES), the attached product (external shield) is referred to with reference to the shield table ST (FIG. 9A). The point for calculating the privacy level is evaluated and determined in accordance with the type of (step S75). On the other hand, when it is determined that the mounted product (external shielding object) is not detected (step S74; NO), the process of step S75 is skipped.

  Next, the second calculation unit 206c measures the color of the skin color area of the face area detected from the processing target recording image, and the second calculation unit 206c acquires the reference face image F from the memory 2. As for the reference face image F, the skin color detection process is performed to measure the color of the skin color area of the face area and calculate the difference between the measured skin colors (step S76). Then, it is determined whether or not a skin color different from the normal color having a calculated skin color difference equal to or greater than a predetermined value is detected (step S77). If it is determined that a skin color different from the normal color is detected (step S77) S77; YES), the point for calculating the privacy level according to the skin color (calculated skin color difference) is evaluated and determined (step S78). On the other hand, if it is determined that a skin color different from normal is not detected (step S77; NO), the process of step S78 is skipped.

Next, the second calculation unit 206c specifies the number of pixels constituting the face area detected from the processing target recording image as the face size (step S79). Then, the second calculation unit 206c evaluates and determines a point for calculating the privacy level according to the specified face size (step S80).
On the other hand, if it is determined that no face area has been detected (step S72; NO), the privacy level calculation process is terminated.

Next, the second calculation unit 206c can detect that the face of the person included in the recorded image is the face of a specific person, for example, the face of the person, and the face and line of sight of the person. Is set as a virtual reference state (step S81). Thereafter, the second calculation unit 206c calculates a rotation center axis and a rotation angle of the detected human face with respect to the set virtual reference state (step S82).
Then, the second calculation unit 206c determines whether or not the face direction of the person detected from the processing target recording image is the front (step S83).

If it is determined in step S83 that the face direction is not front (step S83; NO), the second calculation unit 206c has a rotation angle about a predetermined axis of the person's face with respect to the virtual reference state. A change amount or a change rate is specified, and a point for calculating a privacy level is evaluated and determined according to a virtual change in the specified rotation angle of the face (step S84).
On the other hand, when it is determined in step S83 that the face direction is the front (step S83; YES), the second calculation unit 206c skips the process of step S84.

  Moving to FIG. 15, the detection processing unit 6 b detects the constituent parts of the face such as eyes and mouth from the face area detected in the face detection process in the recorded image to be processed (step S <b> 85). Then, the second calculation unit 206c is specified according to the number of detected face constituent parts, that is, if the person does not perform an operation of hiding the entire face or the face constituent parts (for example, eyes and mouth). The number of points for calculating the degree of privacy is evaluated according to the ratio of the number of components of the face of the person to the number of components (eg, “3” for both eyes and mouth). (Step S86).

Next, the detection processing unit 6b determines whether eyes are detected from the recorded image to be processed (step S87).
If it is determined that eyes are detected (step S87; YES), the detection processing unit 6b performs a line-of-sight detection process on the recording image to be processed (step S88), and the line of sight of the person who is the subject is detected. Whether or not is the front is determined (step S89).

If it is determined in step S89 that the line of sight is not the front (step S89; NO), the second calculation unit 206c identifies a change in the line of sight of the person with respect to the virtual reference state, and the line of sight of the identified person. The point for calculating the privacy level is evaluated and determined in accordance with the virtual change (step S90).
On the other hand, when it is determined in step S89 that the line of sight is the front (step S89; YES), the second calculation unit 206c skips the process of step S90.
If it is determined in step S87 that no eyes are detected (step S87; NO), the processes in steps S88 to S90 are skipped.

Then, the second calculation unit 206c, as a result of the point evaluation according to the wearing product at step S75, as a result of the point evaluation according to the skin color at step S78, and as a result of the point evaluation according to the face size at step S80. As a result of the point evaluation in accordance with the change in the rotation angle of the face with respect to the virtual reference state in step S84, the result of the point evaluation in accordance with the ratio in which the constituent parts of the face are hidden in step S86, the virtual in step S90 The privacy level is calculated based on the result of the point evaluation corresponding to the change in the line of sight of the person with respect to the correct reference state (step S91).
Thus, the privacy level calculation process is terminated.

  As described above, according to the image processing apparatus 200 of the second embodiment, it is possible to determine the degree of difficulty in identifying a person's face included in a recorded image that is one type of image as a specific person's face. The privacy level is calculated as one of determining whether or not the privacy level shown satisfies the predetermined condition, and when it is determined that the calculated privacy level is higher than the predetermined determination value, the predetermined level related to the image As one example of control for executing the process, the image is acquired as an image to be transmitted to a predetermined external device. Therefore, as in the first embodiment, for example, mosaic processing or masking is performed on an image to be disclosed or recorded. No more image processing such as processing, and more natural privacy images according to the desired privacy level without deteriorating the appearance by applying mosaic or mask locally It is possible to get.

In addition, a virtual reference state in which a person's face included in the recorded image can be identified as a specific person's face (for example, the person's face can be detected and the person's face and line of sight are in a predetermined direction. The degree of privacy can be calculated based on the relative change in the face of the person or the part constituting the face of the person from a certain state or the like. Accordingly, in substantially the same manner as in the first embodiment, it is possible to improve usability by using the relative change of the face of the person from the virtual reference state or the part constituting the face of the person. The expression mode of the privacy image P acquired on the basis of the privacy level can be varied.
Furthermore, the privacy level is calculated based on the ratio of the person's face in the recorded image or the part constituting the person's face being hidden, and the size of the person's face included in the recorded image or the person's face is also calculated. The privacy level is calculated based on the imaging distance, the privacy level is calculated based on a change in an external shield that hides the face of the person or the constituent part of the face, and the color of the person's face changes. Since the privacy level is calculated based on this, the privacy level calculation criteria can be increased, and the expression mode of the privacy image P acquired based on the privacy level can be made more diverse.

  Further, by recording the privacy image P and the privacy level in association with each other, the privacy image P can be displayed on the display panel 9b in the classification or order based on the associated privacy level. Even when a plurality of privacy images P are recorded, the user-desired privacy image P can be easily selected, and usability can be improved.

The present invention is not limited to the first and second embodiments, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, in the first and second embodiments, the privacy image P is transmitted to the external recording server S and disclosed, but this is an example and the present invention is not limited to this. For example, an external server that publishes the transmitted privacy image P may display only the privacy image P without recording the privacy image P, and may sequentially delete the displayed privacy image P. The image processing apparatus 100 of 1 or the image processing apparatus 200 of the second embodiment may be provided with a server function so that the privacy image P can be viewed by accessing the image processing apparatuses 100 and 200 from an external terminal. In this case, for example, for each privacy image P, whether to make it public may be automatically set according to the associated privacy level.

  In the first and second embodiments, the size of the face of the person included in the privacy image P is recorded in association with the image data of the privacy image P as Exif information, and the display control unit 9a The privacy image P recorded in the image recording unit 8 may be displayed on the display panel 9b in a classification or order based on the size of the face of the person associated with the privacy image P.

  Further, the configurations of the image processing apparatuses 100 and 200 are merely examples as illustrated in the first and second embodiments, and are not limited thereto. For example, the image processing apparatus 100 is mounted with the imaging unit 3, but is not limited thereto, and is connected to an external imaging unit so as to be able to perform information communication and shooting control without mounting the imaging unit. Also good.

In addition, in the first embodiment, the functions as the determination unit and the control unit are realized by driving the first determination unit 6d and the imaging control unit 6e under the control of the central control unit 1. However, the present invention is not limited to this, and may be realized by executing a predetermined program or the like by the central control unit 1.
That is, a program including a determination processing routine and a control processing routine is recorded in a program memory (not shown). Whether or not the degree of privacy indicating the degree of difficulty of identifying the CPU of the central control unit 1 as a specific person's face by the determination processing routine satisfies a predetermined condition is determined. You may make it function as a means to determine. Further, the CPU of the central control unit 1 is caused to function as a means for controlling execution of the predetermined process relating to the image (privacy image P) when the privacy degree is determined to satisfy the predetermined condition by the control processing routine. Anyway.

Similarly, in the second embodiment, the functions as the calculation unit and the control unit are realized by driving the second calculation unit 206c and the acquisition control unit 206f under the control of the central control unit 1. However, the present invention is not limited to this, and may be realized by executing a predetermined program or the like by the central control unit 1.
That is, a program including a calculation processing routine and a control processing routine is recorded in a program memory (not shown). Then, the CPU of the central control unit 1 is caused to function as a means for calculating the degree of privacy indicating the degree of difficulty in identifying the face of the person included in the image as the face of a specific person by the calculation processing routine. Anyway. Further, the CPU of the central control unit 1 may be caused to function as a means for controlling execution of predetermined processing relating to an image (privacy image P) using the calculated privacy level by a control processing routine.

  Furthermore, as a computer-readable medium storing a program for executing each of the above processes, a non-volatile memory such as a flash memory or a portable recording medium such as a CD-ROM is applied in addition to a ROM or a hard disk. Is also possible. A carrier wave is also used as a medium for providing program data via a predetermined communication line.

Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
Determining means for determining whether or not a privacy level indicating a degree of difficulty in identifying a person's face included in the image as a specific person's face satisfies a predetermined condition;
Control means for controlling execution of predetermined processing relating to an image when the determination means determines that the privacy level satisfies a predetermined condition;
An image processing apparatus comprising:
<Claim 2>
A calculation means for calculating a privacy level indicating a degree of difficulty in identifying the face of the person included in the image as a face of a specific person;
The image processing apparatus according to claim 1, wherein the determination unit determines whether the privacy level calculated by the calculation unit is higher than a predetermined determination value.
<Claim 3>
The calculating means is configured to calculate a relative position of the person's face or a part constituting the person's face from a predetermined reference state that can be identified as the face of the person included in the image. The image processing apparatus according to claim 2, wherein the privacy level is calculated based on a change.
<Claim 4>
The image processing apparatus according to claim 3, wherein the calculation unit calculates the privacy degree based on a change amount or a change rate of a rotation angle centered on a predetermined axis of the person's face.
<Claim 5>
The image processing apparatus according to claim 3, wherein the predetermined reference state includes a state in which the face of the person can be detected and the face and line of sight of the person are in a predetermined direction.
<Claim 6>
The said calculation means calculates the said privacy degree based on the variation | change_quantity or rate of change of the direction with respect to the said person's face of the said image processing apparatus, The Claim 3 characterized by the above-mentioned. Image processing device.
<Claim 7>
The said calculation means calculates the said privacy degree based on the change of the ratio in which the part which comprises the said person's face or the said person's face is concealed. The image processing apparatus according to item.
<Claim 8>
The said calculation means calculates the said privacy degree based on the change of the magnitude | size of the person's face contained in the said image, or the imaging distance to the said person's face, The any one of Claims 3-7 characterized by the above-mentioned. The image processing apparatus according to one item.
<Claim 9>
The said calculating means calculates the said privacy degree based on the change of the external shielding object which hides the structural part of the person's face or the person's face contained in the said image. The image processing apparatus according to any one of the above.
<Claim 10>
The image processing apparatus according to claim 3, wherein the calculation unit calculates the privacy level based on a change in a face color of a person included in the image.
<Claim 11>
The image is an image captured by an imaging unit,
The calculating means calculates the privacy level of the image captured by the imaging means;
The control unit performs control to cause the imaging unit to capture an image for recording when the determination unit determines that the privacy level is higher than a predetermined determination value. The image processing apparatus according to any one of the above.
<Claim 12>
The calculating means sequentially calculates the privacy level for each image sequentially captured by the imaging means,
The determination means sequentially determines whether or not the privacy level sequentially calculated by the calculation means is higher than the predetermined determination value,
The control means, based on the determination result by the determination means, triggered by the fact that the privacy level has changed from a state below the predetermined determination value to a state higher than the predetermined determination value. The image processing apparatus according to claim 11, wherein an image is taken.
<Claim 13>
A recording unit that records an image for recording taken by the control unit in association with the privacy level calculated by the calculation unit;
13. The display control unit according to claim 11, further comprising: a display control unit configured to display the image recorded in the recording unit on the display unit in a classification or order based on the privacy level associated with the image. The image processing apparatus described.
<Claim 14>
The image is an image recorded in a recording means,
The calculating means calculates the privacy level of the image recorded in the recording means;
The control unit performs control to acquire an image determined by the determination unit as having a degree of privacy higher than a predetermined determination value as an image to be transmitted to a predetermined external device. The image processing apparatus according to any one of the above.
<Claim 15>
Calculating means for calculating a privacy level indicating a degree of difficulty of identifying that a person's face included in the image is a face of a specific person;
Control means for controlling execution of predetermined processing relating to an image using the privacy level calculated by the calculating means;
An image processing apparatus comprising:
<Claim 16>
The image is an image captured by an imaging unit,
16. The control unit according to claim 15, wherein the control unit performs control to record the privacy level calculated by the calculation unit in the recording unit in association with an image photographed as a recording image by the imaging unit. The image processing apparatus described.
<Claim 17>
An image processing method using an image processing apparatus,
A process of determining whether or not a privacy level indicating a degree of difficulty in identifying a person's face included in the image as a specific person's face satisfies a predetermined condition;
A process for controlling execution of a predetermined process relating to an image when it is determined that the privacy level satisfies a predetermined condition;
An image processing method comprising:
<Claim 18>
An image processing method using an image processing apparatus,
Processing for calculating a privacy level indicating a degree of difficulty in identifying the face of the person included in the image as a face of a specific person;
A process for controlling execution of a predetermined process relating to an image using the calculated privacy level;
An image processing method comprising:
<Claim 19>
In the computer of the image processing device,
A function of determining whether or not a privacy level indicating a degree of difficulty in identifying a person's face included in the image as a specific person's face satisfies a predetermined condition;
A function of controlling execution of a predetermined process relating to an image when it is determined that the privacy level satisfies a predetermined condition;
A program characterized by realizing.
<Claim 20>
In the computer of the image processing device,
A function for calculating a privacy level indicating a degree of difficulty in identifying a person's face included in the image as a face of a specific person;
A function for controlling execution of a predetermined process related to an image using the calculated privacy level;
A program characterized by realizing.

100, 200 Image processing device 1 Central control unit 2 Memory 3 Imaging unit 4 Signal processing unit 6, 206 Operation control unit 6a First image acquisition unit 6b Detection processing unit 6c First calculation unit 6d First determination unit 6e Imaging control unit 206a Second image acquisition unit 206c Second calculation unit 206d Second determination unit 206f Acquisition control unit 8 Image recording unit 9 Display unit 9a Display control unit 9b Display panel 10 Communication control unit F Reference face image P Privacy image S Recording server ST Shield table

Claims (17)

Calculating means for calculating a privacy level indicating a degree of difficulty of identifying that a person's face included in the image is a face of a specific person;
Determination means for determining whether or not the degree of privacy calculated by the calculation means is higher than a predetermined determination value ;
Control means for controlling execution of predetermined processing relating to an image when the determination means determines that the degree of privacy is higher than a predetermined determination value ;
An image processing apparatus comprising: The calculating means is configured to calculate a relative position of the person's face or a part constituting the person's face from a predetermined reference state that can be identified as the face of the person included in the image. based on the change, the image processing apparatus according to claim 1, characterized in that to calculate the privacy of. The image processing apparatus according to claim 2 , wherein the calculation unit calculates the privacy level based on a change amount or a change rate of a rotation angle about a predetermined axis of the person's face. The predetermined reference state, capable of detecting the face of the person, and image processing apparatus according to claim 2 or 3 face and sight line of the person is characterized in that it comprises a state which is a predetermined direction. It said calculation means on the basis of the direction of variation or the rate of change with respect to the face of the person in the image processing apparatus, according to any one of claims 2 to 4, characterized in that to calculate the privacy of Image processing device. Said calculation means, based on the change in the ratio of parts constituting a face or a face of the person of the person is hidden, any one of claims 2 to 5, characterized in that to calculate the privacy of The image processing apparatus according to item. Said calculation means, based on a change of the imaging distance to size or face of the person in the face of a person included in the image, any one of claims 2-6, characterized in that to calculate the privacy of The image processing apparatus according to one item. Said calculation means, based on a change of the face or outside the shield to hide the construction site of a face of the person of the person included in the image, of claims 2 to 7, characterized in that to calculate the privacy of The image processing apparatus according to any one of the above. The image processing apparatus according to any one of claims 2 to 8 , wherein the calculation unit calculates the privacy level based on a change in a face color of a person included in the image. The image is an image captured by an imaging unit,
The calculating means calculates the privacy level of the image captured by the imaging means;
Wherein, when said the privacy of the determining means is determined to be higher than a predetermined judgment value, claims 1 to 9, characterized in that performs control to capture an image for recording on said image pickup means The image processing apparatus according to any one of the above. The calculating means sequentially calculates the privacy level for each image sequentially captured by the imaging means,
The determination means sequentially determines whether or not the privacy level sequentially calculated by the calculation means is higher than the predetermined determination value,
The control means, based on the determination result by the determination means, triggered by the fact that the privacy level has changed from a state below the predetermined determination value to a state higher than the predetermined determination value. The image processing apparatus according to claim 10 , wherein an image is taken. A recording unit that records an image for recording taken by the control unit in association with the privacy level calculated by the calculation unit;
The display control means for displaying the image recorded in the recording means on the display means in a classification or order based on the privacy level associated with the image. The image processing apparatus according to 10 or 11 . The image is an image recorded in a recording means,
The calculating means calculates the privacy level of the image recorded in the recording means;
The control means according to claim 1 to 9, characterized in that the control of the privacy of by the determining means an image that is determined to be higher than a predetermined judgment value is acquired as an image to be transmitted to a predetermined external device The image processing apparatus according to any one of the above. Calculating means for calculating a privacy level indicating a degree of difficulty of identifying that a person's face included in the image is a face of a specific person;
A recording unit that records the image and the privacy level calculated by the calculation unit in association with each other;
Display control means for classifying the images recorded in the recording means on the basis of the privacy level associated with the images, and displaying any one image on the display means for each classification;
An image processing apparatus comprising: Calculating means for calculating a privacy level indicating a degree of difficulty of identifying that a person's face included in the image is a face of a specific person;
A recording unit that records the image and the privacy level calculated by the calculation unit in association with each other;
Display control means for rearranging the images recorded in the recording means on the basis of the privacy level associated with the image, and displaying the images on the display means in the rearranged order;
An image processing apparatus comprising: An image processing method using an image processing apparatus,
Processing for calculating a privacy level indicating a degree of difficulty in identifying the face of the person included in the image as a face of a specific person;
A process of determining whether or not the privacy level calculated by the calculating means is higher than a predetermined determination value ;
A process for controlling execution of a predetermined process relating to an image when the determination unit determines that the privacy level is higher than a predetermined determination value ;
An image processing method comprising: In the computer of the image processing device,
A function for calculating a privacy level indicating a degree of difficulty of identifying that a person's face included in the image is a face of a specific person;
A function for determining whether or not the privacy level calculated by the calculating means is higher than a predetermined determination value ;
A function of controlling execution of a predetermined process relating to an image when the degree of privacy is determined to be higher than a predetermined determination value by the determination unit;
A program characterized by realizing.