JP2017097653A - Image processing device, image editing device, and information sharing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a specified person to visually recognize a subject, while preventing the subject included in LF image data from being visually recognized by many and unspecified number of persons.SOLUTION: An image processing device 200 includes an image acquisition part 252 which acquires a light-field image capable of changing a focus position which is a position in a depth direction brought into focus, after photographing, an information acquisition part 253 which acquires the first identification information of a first user corresponding to at least a partial area included in the light-field image and acquires second identification information corresponding to a second user browsing the light-field image, and a display control part 255 which outputs re-focus image data in which the focus position is set in the position different from the position in the depth direction of the subject included in a partial area on the basis of the relationship between the first identification information and the second identification information.SELECTED DRAWING: Figure 10

Description

  The present invention relates to an image processing apparatus, an image editing apparatus, and an information sharing system.

  In recent years, an imaging device called a light field (hereinafter abbreviated as “LF”) camera has been put into practical use. In this imaging device, a microlens array is arranged on the image sensor, and light incident on the imaging device is divided by the microlens, thereby to provide light information indicating the intensity of light coming from a plurality of directions and a direction indicating the incident direction. LF image data including information is generated. In Patent Document 1, restriction information, which is information for making it difficult to visually recognize a subject selected in LF image data, is recorded in association with LF image data, and is selected when displaying an image based on LF image data. A method for displaying a blurred subject is disclosed.

Japanese Patent Laying-Open No. 2015-84517

  In recent years, it has become possible for an unspecified number of people to browse photos using services such as SNS (Social Network Service). When the user who uploaded the LF image data to the SNS server publishes the LF image data to which the restriction information is added, if the conventional method is used, the selected subject is always displayed in a blurred state. Therefore, even when a specific person (for example, a close friend) who wants to view the subject by the user who uploaded the LF image data browses an image based on the LF image data, the subject is displayed in a blurred state. There was a problem that.

  Therefore, the present invention has been made in view of the above points, and allows a specific person to visually recognize a subject while preventing the subject included in the LF image data from being viewed by an unspecified number of people. For the purpose.

  In order to solve the above-described problem, the image processing apparatus according to the first aspect of the present invention includes an image acquisition unit that acquires a light field image that can be changed after photographing a focus position that is a position in the depth direction in focus. Information for acquiring first identification information of a first user corresponding to at least a part of a region included in the light field image and acquiring second identification information corresponding to a second user browsing the light field image Refocus in which the focus position is set at a position different from the position in the depth direction of the subject included in the partial area based on the relationship between the acquisition means and the first identification information and the second identification information Output means for outputting image data.

  The image editing apparatus of the present invention acquires the light field image in response to receiving an instruction to acquire a light field image that can be changed after shooting the focus position, which is a position in the depth direction in focus. Image acquisition means, information acquisition means for acquiring user identification information corresponding to the user, user identification information corresponding to the user in association with at least a part of the area included in the light field image, Generating means for generating restriction information for restricting setting of the focus position to the subject included in the area of the part, and output for outputting the light field image to which the restriction information generated by the generating means is added And means.

  An information sharing system according to the present invention is an information sharing system including a first user terminal used by a first user, a second user terminal used by a second user, and an image processing device. An image acquisition means for acquiring a light field image that can be changed after photographing the focus position, which is a position in the depth direction in focus, in accordance with an instruction from the first user terminal; and at least one included in the light field image Information acquisition means for acquiring first identification information corresponding to a region of a section and acquiring second identification information corresponding to the second user; and a relationship between the first identification information and the second identification information. Based on this, the refocus image data in which the focus position is set at a position different from the position in the depth direction of the subject area included in the part is the second unit. And an output means for outputting to The terminal, the second user terminal is characterized by comprising a display control means for displaying the refocused image data acquired from the image processing apparatus on the display unit.

  According to the present invention, there is an effect that a specific person can visually recognize a subject while preventing the subject included in the LF image data from being visually recognized by an unspecified number of people.

It is a schematic diagram of a structure of the imaging device 10 as LF camera. 3 is a schematic diagram showing a positional relationship between a microlens array 12 and each pixel of an image sensor 13. FIG. FIG. 6 is a schematic diagram for explaining the relationship between the traveling direction of incident light rays to the microlenses of the microlens array and the recording area of the image sensor 13; It is a schematic diagram for demonstrating the information of the light ray which injects into the image sensor. It is a schematic diagram for demonstrating a refocus calculation process. 6 is a schematic diagram for explaining a relationship between a difference in incident angle to a microlens and a recording area of the image sensor 13; It is a schematic diagram for demonstrating the adjustment process of a depth of field. It is a figure showing composition of information sharing system 1 concerning a 1st embodiment. 1 is a diagram illustrating a configuration of an image editing apparatus 100 according to a first embodiment. It is a figure which shows the structure of the image processing apparatus 200 which concerns on 1st Embodiment. 3 is a flowchart showing an operation procedure of the image editing apparatus 100 according to the first embodiment. 4 is a flowchart illustrating an operation procedure of the image processing apparatus 200 according to the first embodiment. 3 is a diagram illustrating an example of an image reproduced in the image processing apparatus 200. FIG. It is a figure which shows the structure of the information sharing system 1 which concerns on 2nd Embodiment. It is a figure which shows the structure of the server 400 which concerns on 2nd Embodiment. It is a flowchart which shows the operation | movement procedure at the time of the upload of LF image data of the server 400 and user terminal 500 which concern on 2nd Embodiment. It is a flowchart which shows the operation | movement procedure at the time of the display of the LF image of the server 400 and user terminal 500 which concern on 2nd Embodiment. It is a flowchart which shows the operation | movement procedure which concerns on the batch provision of the focus restriction information to LF image data of the server 400 and user terminal 500 which concern on 3rd Embodiment.

[Outline of LF camera]
Prior to the description of the image processing apparatus according to the present embodiment, an outline of the LF camera will be described.
FIG. 1 is a schematic diagram of a configuration of an imaging device 10 as an LF camera. The imaging device 10 includes an imaging lens 11, a microlens array 12, and an image sensor 13. The light from the subject enters the image sensor 13 after passing through the imaging lens 11 and the microlens array 12 constituting the imaging optical system. The image sensor 13 converts the incident light into an electric signal and outputs the converted electric signal. The imaging device 10 generates LF image data that can change the focus position, which is the position in the depth direction in focus, based on the electrical signal output from the image sensor 13.

  The imaging lens 11 projects light from the subject onto the microlens array 12. The imaging lens 11 is a replaceable lens that is used by being attached to the main body of the imaging device 10. The user of the imaging device 10 can change the imaging magnification by a zoom operation of the imaging lens 11.

  The microlens array 12 has a plurality of minute lenses (microlenses) arranged in a lattice pattern, and is located between the imaging lens 11 and the image sensor 13. Each microlens included in the microlens array 12 divides the incident light from the imaging lens 11 and makes the plurality of divided light incident on the plurality of pixels of the image sensor 13.

  The image sensor 13 is an image sensor having a plurality of pixels, and detects the intensity of light at each pixel. Light in different directions divided by the microlens corresponding to each pixel is incident on each pixel of the image sensor 13 that receives light from the subject.

  FIG. 2 is a schematic diagram showing the positional relationship between the microlens array 12 and each pixel of the image sensor 13. Each microlens of the microlens array 12 is arranged so as to correspond to a plurality of pixels in the image sensor 13. Light divided by the microlens corresponding to each pixel is incident on each pixel of the image sensor 13, and the image sensor 13 detects the intensity of light from a different direction for each pixel. The imaging device 10 generates light ray information indicating the intensity of light detected at each pixel of the image sensor 13.

  The incident direction of the light beam incident on each pixel of the image sensor 13 through the microlens is determined by the arrangement of a plurality of pixels corresponding to each microlens. Therefore, the incident direction is specified according to the positional relationship between each microlens and each pixel of the image sensor 13, and the imaging apparatus 10 uses, for example, coordinates for specifying the pixel as direction information indicating the specified incident direction. Generate information.

  Light from the subject enters the microlens array 12 in a direction determined according to the distance between the subject and the imaging lens 11. The light from the subject passes through a plurality of microlenses determined according to the distance between the subject and the imaging lens 11 and enters the image sensor 13. As a result, the image focused on the surfaces having different distances from the lens apex surface of the microlens array 12 is each of the plurality of pixels of the image sensor 13 at the position corresponding to the amount of eccentricity from the optical axis of each microlens. It is obtained by combining the output. Therefore, the imaging apparatus 10 can generate image data corresponding to an arbitrary focus position by performing light beam rearrangement and calculation processing (hereinafter referred to as reconstruction) based on the light beam information and the direction information. Here, the focus position is an in-focus position, and the user can obtain refocus image data corresponding to a desired focus position based on the LF image data after shooting.

FIG. 3 is a schematic diagram for explaining the relationship between the traveling direction of incident light to the microlenses of the microlens array 12 and the recording area of the image sensor 13.
An image of the subject by the imaging lens 11 is formed on the microlens array 12, and incident light on the microlens array 12 is received by the image sensor 13 via the microlens array 12. At this time, as shown in FIG. 3, the incident light beam to the microlens array 12 is received at different positions on the image sensor 13 according to the traveling direction determined for each focus position, and the image of the subject is formed for each microlens. Image.

FIG. 4 is a schematic diagram for explaining information on light rays incident on the image sensor 13.
The light rays received by the image sensor 13 will be described with reference to FIG. Here, an orthogonal coordinate system on the lens surface of the imaging lens 11 is (u, v), and an orthogonal coordinate system (x, y) on the imaging surface of the image sensor 13 is used. Further, let F be the distance between the lens surface of the imaging lens 11 and the imaging surface of the image sensor 13. In this case, the intensity of the light beam passing through the imaging lens 11 and the image sensor 13 can be expressed by a four-dimensional function L (u, v, x, y) shown in the drawing. Since the light beam incident on each microlens is incident on a different pixel according to the traveling direction, the image sensor 13 uses the above four-dimensional function L (which holds the traveling direction of the light beam in addition to the positional information of the light beam). Information corresponding to the light intensity represented by u, v, x, y) is recorded.

  Next, refocus calculation processing after imaging will be described. The refocus calculation process is a process for generating refocus image data by reconstructing an image based on the intensity of light received by pixels corresponding to different focus positions. In this specification, a surface at a position in focus in a refocus image is referred to as a refocus surface.

FIG. 5 is a schematic diagram for explaining the refocus calculation process.
As shown in FIG. 5, when the positional relationship between the imaging lens surface, the imaging surface, and the refocus plane is set, the light intensity L ′ (u, v, s) in the orthogonal coordinate system (s, t) on the refocus plane. , T) is expressed as the following equation (1).
Further, the image E ′ (s, t) obtained on the refocus plane is obtained by integrating the intensity L ′ (u, v, s, t) with respect to the lens diameter, and is expressed by the following equation (2). It is expressed in
Therefore, image data corresponding to the refocus plane at an arbitrary focus position can be reconstructed by performing the refocus calculation process based on the equation (2).

  Next, the adjustment process of the depth of field after imaging, that is, the adjustment process of the focus position in the depth direction will be described. When adjusting the depth of field, weighting is performed by multiplying each image data forming the image area assigned to each microlens by a weighting factor before the above refocusing calculation processing. For example, when generating an image having a deep depth of field, the integration process is performed using only information on light rays incident at a relatively small angle with respect to the light receiving surface of the image sensor 13. In other words, light rays having a relatively large incident angle to the image sensor 13 are not included in the integration process by multiplying by a weighting factor of 0 (zero).

FIG. 6 is a schematic diagram for explaining the relationship between the difference in the incident angle to the microlens and the recording area of the image sensor 13, and FIG. 7 is a schematic diagram for explaining the depth-of-field adjustment process. is there.
As shown in FIG. 6, the light beam having a relatively small incident angle to the image sensor 13 is located in a more central region. Therefore, when generating an image with a deep depth of field, as shown in FIG. 7, integration processing is performed using only pixel data acquired at the center of the region (shaded portion in the figure).

By performing such processing, the imaging apparatus 10 can express an image with a deep depth of field as if the aperture stop provided in a general imaging apparatus or the like is limited. The imaging apparatus 10 can also generate pan-focus image data with a deeper depth of field by further reducing the pixel data in the central portion to be used. The imaging device 10 can also adjust the depth of field to be different for each region by changing the position of the pixel data to be used for each region of the LF image data. As described above, the imaging device 10 adjusts the depth of field of the image after shooting based on the focus information including the weighting coefficient of each pixel included in the LF image data, the light ray information, and the direction information, and performs a desired operation. The focus position can be set.
Hereinafter, embodiments of an image processing apparatus and an image processing method for processing LF image data will be described.

<First Embodiment>
[Configuration of Information Sharing System 1]
FIG. 8 is a diagram illustrating a configuration of the information sharing system 1 according to the first embodiment. The information sharing system 1 is, for example, an SNS, and includes an image editing device 100, an image processing device 200, and a database 300.

  The image editing apparatus 100 is, for example, a user terminal such as a smartphone used by the first user, and is stored in the database 300 on the cloud connected via a communication line such as the Internet and a mobile phone network. Upload LF image data. The LF image data includes pixel value of each pixel and default focus information corresponding to the default focus position. The default focus information is, for example, a weighting factor for each pixel constituting the LF image used when refocusing image data set at the default focus position is reconstructed. The image editing apparatus 100 assigns to the database 300 focus restriction information that restricts setting of the focus position to the position in the depth direction of the subject included in at least a part of the region included in the LF image. Can be stored.

  The image processing apparatus 200 is a user terminal such as a smartphone used by the second user, for example, acquires LF image data stored in the database 300 on the cloud, and displays an LF image. The acquired LF image data includes a pixel value of each pixel, default focus information, and focus restriction information. The image processing apparatus 200 can display a refocus image generated by reconstructing LF image data selected in response to a request from the second user. The refocus image data corresponding to the refocus image includes focus information indicating the weight coefficient of each pixel after the focus position is changed from the default focus position indicated by the default focus information to another position.

Based on the focus restriction information included in the LF image data, the image processing apparatus 200 determines that the focus position of at least a part of the region included in the LF image is the position in the depth direction of the subject included in the part of the region. To be set to By doing in this way, the image processing apparatus 200, for example, the LF image including the face image of the first user who uploaded the LF image using the image editing apparatus 100, the second user who is not related to the first user. When the user browses, it is possible to restrict the focus on the face image of the first user and to ensure the privacy of the first user.
Details of the configurations of the image editing apparatus 100 and the image processing apparatus 200 will be described below.

[Configuration of Image Editing Device 100]
FIG. 9 is a diagram illustrating a configuration of the image editing apparatus 100 according to the first embodiment. The image editing apparatus 100 stores, for example, LF image data captured by the first user. The image editing apparatus 100 reads LF image data in response to a request from the first user, adds focus restriction information to the read LF image data, and stores it in the database 300.

The image editing apparatus 100 includes a communication unit 110, an operation unit 120, a display unit 130, a storage unit 140, and a control unit 150.
The communication unit 110 is a communication interface for transmitting and receiving LF image data and various information to and from the database 300.

  In the operation unit 120, the first user inputs user identification information (hereinafter referred to as first identification information) of the first user, or inputs information for designating LF image data to be transmitted to the database 300. For example, a touch panel provided on the display unit 130 in an overlapping manner. The first identification information is information for identifying the first user who is operating the image editing apparatus 100 as a user terminal, such as a user ID of SNS. The first user can upload the LF image data to the database 300 by inputting the first identification information when logging in to the SNS.

The display unit 130 is a display for displaying LF image data uploaded to the database 300 and an operation screen for uploading LF image data.
The storage unit 140 includes a non-volatile memory (eg, ROM), and stores an image editing program executed by the control unit 150. The storage unit 140 includes a volatile memory (for example, a RAM), and temporarily stores various data generated when the control unit 150 executes the image editing program. The storage unit 140 stores a plurality of LF image data.

  The control unit 150 is, for example, a CPU, and controls each unit of the image editing apparatus 100 by executing an image editing program stored in the storage unit 140. The control unit 150 functions as an image acquisition unit 151, an information acquisition unit 152, a focus restriction information generation unit 153, and an image output unit 154. Hereinafter, the configuration of each functional unit provided by the control unit 150 will be described.

  In response to receiving an upload instruction from the first user as an instruction to acquire LF image data to be uploaded to the database 300 via the operation unit 120, the image acquisition unit 151 stores the LF image data in the storage unit 140. Get from. The image acquisition unit 151 acquires LF image data selected by the first user from the plurality of LF image data stored in the storage unit 140. The image acquisition unit 151 outputs the acquired LF image data to the focus restriction information generation unit 153.

  The information acquisition unit 152 acquires first identification information stored in the storage unit 140. The information acquisition unit 152 outputs the acquired first identification information to the focus restriction information generation unit 153. Note that the information acquisition unit 152 may acquire the first identification information input from the operation unit 120 when the first identification information is not stored in the storage unit 140.

  The information acquisition unit 152 acquires user profile information of the first user corresponding to the first identification information from the database 300. The user profile information includes face image information of the first user as area specifying information used for specifying some areas included in the LF image. The information acquisition unit 152 acquires the first user's face image information from the user profile information. The information acquisition unit 152 outputs the extracted face image information to the focus restriction information generation unit 153. When user profile information is stored in the storage unit 140, the information acquisition unit 152 may acquire user profile information from the storage unit 140.

  When the information acquisition unit 152 acquires the face image information, the focus restriction information generation unit 153 restricts the focus position from being set in the partial area in association with at least the partial area included in the LF image. Generate focus restriction information. When the LF image includes a subject corresponding to the acquired face image information, the focus restriction information generation unit 153 generates focus restriction information corresponding to a region including the subject.

  For example, in the LF image data, the focus restriction information generation unit 153 specifies a rectangular area that includes the face image of the first user corresponding to the face image information as an area that restricts the setting of the focus position. Specify the coordinates of each vertex. Then, the focus restriction information generation unit 153 generates focus restriction information including first identification information for identifying the first user and coordinates of each vertex constituting the rectangular area. By doing in this way, the image editing apparatus 100 can apply the focus restriction to the area corresponding to the face image when the face image of the first user is included in the LF image.

  Note that the focus restriction information generation unit 153 may specify a plurality of rectangular areas and may specify the coordinates of each vertex constituting each of these rectangular areas. Then, the focus restriction information generation unit 153 may generate focus restriction information including the coordinates of each vertex constituting a plurality of rectangular areas. In addition, the focus restriction information generation unit 153 may receive a selection of a region for restricting focus in the LF image from the first user and generate focus restriction information in association with the selected region. In this way, the image editing apparatus 100 captures subjects related to the privateness of the first user, such as the first user's home and family, and subjects that the first user does not want to be seen by an unspecified number of people. Focus restriction can be applied to the display area.

  The image output unit 154 outputs the LF image data to which the focus restriction information generated by the focus restriction information generation unit 153 is added. For example, the image output unit 154 transmits the LF image data with the focus restriction information generated by the focus restriction information generation unit 153 to the database 300. Thereby, the LF image data to which the focus restriction information is added is uploaded to the database 300. Note that the image output unit 154, when the default focus information of the LF image data includes a weight coefficient corresponding to the state where the focus position is set in the area where the focus is restricted, the weight coefficient in the default focus information. LF image data that has been changed to a weighting factor corresponding to a state where the focus position is not set in the area where the focus is restricted may be output. In this way, it is possible to prevent an unspecified number of people who have acquired LF image data from browsing an image that is in focus in a region where focus is restricted.

[Configuration of Image Processing Device 200]
FIG. 10 is a diagram illustrating a configuration of the image processing apparatus 200 according to the first embodiment. The image processing apparatus 200 reads LF image data stored in the database 300 in response to a request from the second user, and displays an LF image. Further, the image processing apparatus 200 changes the focus position of the displayed LF image in response to a request from the second user.

The image processing apparatus 200 includes a communication unit 210, an operation unit 220, a display unit 230, a storage unit 240, and a control unit 250.
The communication unit 210 is a communication interface for transmitting / receiving LF image data and various types of information to / from the database 300.
In the operation unit 220, the second user inputs user identification information (hereinafter referred to as second identification information) of the second user, or inputs information for designating LF image data acquired from the database 300. For example, a touch panel provided on the display unit 230 in an overlapping manner. The second user can access the LF image data stored in the database 300 by inputting the second identification information when logging in to the SNS.

The display unit 230 is a display for displaying an image based on LF image data or refocus image data acquired from the database 300.
The storage unit 240 includes a non-volatile memory, and stores an image processing program executed by the control unit 250. The storage unit 240 includes a volatile memory, and temporarily stores various data generated by the control unit 250 executing the image processing program. Further, the storage unit 240 stores second identification information input from the second user via the operation unit 220.

  The control unit 250 is, for example, a CPU, and controls each unit of the image processing apparatus 200 by executing an image processing program stored in the storage unit 240. The control unit 250 functions as a reproduction control unit 251, an image acquisition unit 252, an information acquisition unit 253, a refocus image generation unit 254, and a display control unit 255. Hereinafter, the configuration of each functional unit provided by the control unit 250 will be described.

  For example, the reproduction control unit 251 displays an operation screen including a reproduction image selection button and a reproduction start button on the display unit 230 via the display control unit 255 described later. The playback control unit 251 includes image specifying information (for example, an image file number) for specifying LF image data input by the second user pressing the playback image selection button, and the second user giving a playback start instruction. The reproduction start instruction information indicating that the button has been pressed is acquired. The reproduction control unit 251 transmits the image specifying information to the database 300 via the communication unit 210 in response to obtaining the reproduction start instruction information.

  The image acquisition unit 252 acquires, from the database 300, LF image data whose focus position can be changed after shooting through the communication unit 210. The image acquisition unit 252 acquires LF image data corresponding to the image specifying information from the database 300 in response to the reproduction control unit 251 transmitting the image specifying information to the database 300. The image acquisition unit 252 stores the acquired LF image data in the storage unit 240. The image acquisition unit 252 outputs the acquired LF image data to the refocus image generation unit 254. In addition, the image acquisition unit 252 outputs default focus information included in the LF image data to the refocus image generation unit 254.

The information acquisition unit 253 acquires the second identification information of the second user who browses the LF image. The information acquisition unit 253 outputs the acquired second identification information to the refocus image generation unit 254.
Further, the information acquisition unit 253 acquires user profile information corresponding to the second identification information from the database 300. The user profile information includes one or more user identification information associated with the second identification information. The user identification information associated with the second identification information is, for example, user identification information of a user who has a friendship with the second user who browses the LF image in the SNS. The user profile information is used as, for example, relationship information indicating the relationship between the first user and the second user, that is, the relationship between the first identification information and the second identification information. The information acquisition unit 253 outputs one or more pieces of user identification information associated with the second identification information to the refocus image generation unit 254.
When user profile information is stored in the storage unit 240, the information acquisition unit 253 may acquire user profile information from the storage unit 240.

  Further, the information acquisition unit 253 acquires focus restriction information including first identification information corresponding to at least a part of the region included in the LF image data. For example, the information acquisition unit 253 receives the image acquisition unit in response to a partial position or region included in the LF image displayed on the display unit 230 being selected by the second user via the operation unit 220. Focus restriction information included in the acquired LF image data 252 is acquired.

  The refocus image generation unit 254 generates refocus image data in which the focus position is set based on the focus restriction information. The refocus image generation unit 254 selects the position or region selected based on the relationship between the first user and the second user when the focus restriction information is associated with the position or region selected by the second user. Refocus image data is generated by determining whether or not to focus. For example, when the focus restriction is applied to the position or area selected by the second user, the refocus image generation unit 254 determines the relationship between the second identification information and the first identification information included in the focus restriction information. Based on this, refocus image data in which the focus position is set at a position different from the position in the depth direction of the subject included in the partial area is generated.

  Specifically, the refocus image generation unit 254 acquires coordinate information indicating a rectangular area included in the focus restriction information associated with the LF image data, and the coordinate information and the position selected by the second user Alternatively, based on the coordinates indicating the area, it is determined whether the selected position or area is included in the rectangular area. When the refocus image generation unit 254 determines that the selected position or region is included in the rectangular region, the refocus image generation unit 254 determines the relationship between the second user and the first user, for example, an SNS. Friend registration information indicating whether or not a friend relationship exists is acquired from the database 300.

  The refocus image generation unit 254 selects the position or region selected by the second user when the second identification information of the second user is not in a relationship (for example, a friend relationship) previously associated with the first identification information of the first user. Refocus image data that is out of focus is generated. When the second user's second identification information is associated with the first user's first identification information in advance, the refocus image generation unit 254 focuses on the position or region selected by the second user. Generate focus image data. For example, the refocus image generation unit 254 determines the focus position so that the light intensity at the pixel corresponding to the position or region selected by the second user is the strongest based on the light ray information included in the LF image data. As a result, refocus image data is generated.

  In addition, the refocus image generation unit 254, when the first user and the second user do not satisfy a predetermined relationship, the face image area included in the rectangular area corresponding to the first identification information indicated by the focus restriction information May be detected, and a refocused image in which the detected face image region is not in focus may be generated. In this case, the refocus image generation unit 254 sets the focus position at a position different from the position in the depth direction of the face in the face image area, and in the area other than the face in the rectangular area indicated by the focus restriction information. Then, refocus image data in which a focus position is set at a position in the depth direction corresponding to a subject in an area other than the face is generated. In this way, when the position or area selected by the second user is included in the area corresponding to the first identification information, only the face of the subject included in the area can be blurred. Therefore, it is possible to obtain an image closer to a natural state than in the case where the entire rectangular area is blurred.

  The display control unit 255 functions as an output unit, and outputs the refocus image data generated by the refocus image generation unit 254 to the display unit 230. The display control unit 255 sets the focus position to a position based on the default focus information before outputting to the display unit 230 an image in which the focus position is set at a position different from the position in the depth direction of the subject based on the focus restriction information. The set refocus image data is output. Then, when the position or area specified by the second user in the image based on the refocus image data is included in a partial area indicated by the focus restriction information, the depth direction of the subject included in the partial area The refocus image data in which the focus position is set at a position different from the position is output. Accordingly, when the subject specified by the second user is a subject restricted by the first user of the image editing device 100, the image processing device 200 displays a refocused image obtained by blurring the subject. Can be displayed.

  The display control unit 255 also displays the refocus image data in which the focus position in a part of the area corresponding to the focus restriction information is set to a position different from the position of the subject included in the part of the depth direction. When outputting, information indicating that the setting of the focus position is restricted is output to the display unit 230.

[Flowchart showing Operation Procedure in Information Sharing System 1]
FIG. 11 is a flowchart showing an operation procedure of the image editing apparatus 100 according to the first embodiment.
First, when receiving an instruction to upload LF image data from the first user (S101), the image acquisition unit 151 acquires LF image data to be uploaded from the storage unit 140 (S102).

Subsequently, the information acquisition unit 152 acquires the first identification information of the first user stored in the storage unit 140 (S103).
Subsequently, the information acquisition unit 152 acquires user profile information corresponding to the first identification information acquired in S103 from the database 300 (S104).
Subsequently, the information acquisition unit 152 acquires the face image information of the first user included in the acquired user profile information (S105).

  Subsequently, the focus restriction information generation unit 153 determines whether or not the LF image indicated by the LF image data acquired in S102 includes an image corresponding to the face image indicated by the face image information acquired in S105. Thus, it is determined whether or not the first user exists in the LF image (S106). If the focus restriction information generation unit 153 determines that the first user is present in the LF image, the focus restriction information generation unit 153 proceeds to S107. If the focus restriction information generation unit 153 determines that the first user is not present in the LF image, the focus restriction information generation unit 153 performs processing in S109. Move.

Subsequently, the focus restriction information generation unit 153 generates focus restriction information corresponding to the display area including the face image of the first user (S107).
Subsequently, the image output unit 154 generates, for example, LF image data in which the focus restriction information is associated with the display area including the face image of the first user (S108).

  Subsequently, the image output unit 154 stores the LF image data in the database 300 by transmitting the LF image data to the database 300 (S109). Here, when the face image of the first user is present in the LF image, the LF image data to which the focus restriction information is added is stored in the database 300, and the face image of the first user is included in the LF image. If it does not exist, LF image data to which no focus restriction information is assigned is stored in the database 300.

FIG. 12 is a flowchart illustrating an operation procedure of the image processing apparatus 200 according to the first embodiment.
First, the reproduction control unit 251 displays an operation screen for selecting LF image data on the display unit 230, and accepts an instruction to start reproduction of LF image data (S111). The playback control unit 251 transmits image specifying information for specifying the LF image data to the database 300 in response to selection of the LF image data to be displayed on the operation screen and reception of a playback start instruction.

Subsequently, the image acquisition unit 252 acquires LF image data corresponding to the image specifying information transmitted by the reproduction control unit 251 from the database 300 (S112).
Subsequently, the refocus image generation unit 254 generates a refocus image by reconstructing the LF image data based on the default focus information included in the LF image data acquired in S112. The generated refocus image is an image set at the default focus position. The display control unit 255 displays the generated refocus image on the display unit 230 (S113).

  FIG. 13 is a diagram illustrating an example of an image reproduced in the image processing apparatus 200. FIG. 13A shows an image when the LF image data stored in the database 300 is reproduced based on the default focus information. In the image shown in FIG. 13A, “Mr. B” is not in focus, and “Mr. A” appearing next is in focus. “Mr. B” in the LF image data shown in FIG. 13A is a first user who uses the image editing apparatus 100, and focus restriction information is associated with “Mr. B”.

  In S <b> 114 of FIG. 12, the playback control unit 251 receives an instruction to select a subject to be newly focused on from the second user using the image processing device 200 via the operation unit 220. The instruction for selecting the subject received from the second user includes coordinate information indicating the position selected by the second user in the LF image. In the present embodiment, it is assumed that “Mr. B” has been selected as the subject to be newly focused.

  The information acquisition unit 253 determines whether or not focus restriction information is given to the display area corresponding to the selected subject in the LF image data (S115). The information acquisition unit 253 gives focus restriction information when the coordinates indicated by the coordinate information included in the instruction to select the subject are included in the rectangular area indicated by the coordinate information included in the focus restriction information. It is determined that If the information acquisition unit 253 determines that the focus restriction information is assigned, the process proceeds to S116. If the information acquisition unit 253 determines that the focus restriction information is not provided, the information acquisition unit 253 proceeds to S118. In the present embodiment, since the focus restriction information is associated with “Mr. B”, the information acquisition unit 253 moves the process to S116.

  In S116, the information acquisition unit 253 acquires friend registration information in the SNS of the second user of the image processing device 200 from the database 300 (S116). Subsequently, the refocus image generation unit 254 determines whether or not the first identification information and the second identification information included in the focus restriction information are related based on the friend registration information acquired by the information acquisition unit 253. It is determined whether or not the first user is registered as a friend of the second user (S117). If the refocus image generation unit 254 determines that the second user is registered as a friend of the first user, the refocus image generation unit 254 moves to S118 and generates focus information in which the focus position is set to the selected subject. If the refocus image generation unit 254 determines that the second user is not registered as a friend of the first user, the refocus image generation unit 254 maintains the focus information as default focus information.

In S119, the refocus image generation unit 254 generates refocus image data by reconstructing the LF image based on the focus information.
Subsequently, the display control unit 255 causes the display unit 230 to display an image based on the refocus image data generated in S119 (S120).

  For example, if the display control unit 255 determines in S117 that the second user is registered as a friend of the first user based on the relationship between the first identification information and the second identification information, As illustrated in FIG. 13B, an image in which the focus position is changed to a position where “Mr. B” is focused is displayed on the display unit 230. If the display control unit 255 determines in S117 that the second user is not registered as a friend of the first user based on the relationship between the first identification information and the second identification information, As illustrated in FIG. 13C, an image in which the focus position is maintained at a position where “Mr. A” is focused is displayed on the display unit 230.

  At this time, as shown in FIG. 13C, the display control unit 255 displays a message “focus is restricted” as information indicating that the setting of the focus position is restricted in the display area for the subject. Is displayed on the display unit 230. The display control unit 255 displays the message at a position related to “Mr. B” that is the subject selected by the second user. In this way, the image processing apparatus 200 can cause the second user who browses the LF image to recognize that the focus is limited, so that the second user can cause the image processing apparatus 200 to malfunction. It can prevent misunderstanding that it is.

  By the above procedure, the image processing apparatus 200 allows the LF image data when the second user who is not registered as a friend of the first user in the LF image data browses an image based on the LF image data. It is possible to restrict the focus on the first user “Mr. B” shown in FIG. 1 and to allow the specific person to visually recognize the user while ensuring the privacy of the first user.

(Modification 1)
In the above description, the display control unit 255 of the image processing device 200 includes the first identification information included in the focus restriction information and the second identification information that identifies the second user who uses the image processing device 200. Although an example of outputting refocus image data in which the focus position is set at a position different from the position in the depth direction of some areas included in the LF image data based on the relationship has been described, In addition, coordinates for specifying a region to be focused may be included, and the first identification information may not be included.

  In this case, the database 300 stores the LF image data in association with the first identification information corresponding to the first user who transmitted the LF image data to the database 300. Then, the information acquisition unit 253 acquires the first identification information associated with the LF image data to be reproduced from the database 300, and the refocus image generation unit 254 includes the first identification information, the second identification information, and the like. Based on the relationship, refocus image data in which the focus position is set at a position different from the position in the depth direction of the area corresponding to the coordinates included in the focus restriction information is generated. Then, the display control unit 255 outputs the refocus image data. Thus, when the first user who uploaded the LF image data to the database 300 and the second user who browses the image corresponding to the LF image data are not registered as friends, the second user can browse In the image to be displayed, the first user can blur and display an area that the unspecified number of people do not want to be seen, so that the specific user can visually recognize the first user while ensuring the privacy of the first user. Is possible.

(Modification 2)
In the above description, an example has been described in which the focus position is not set in the area corresponding to the coordinates included in the focus restriction information in the default focus information. However, there may be a case where focus restriction is applied to an area where the focus position is set in the default focus information.

  Therefore, when the default focus image is displayed, the information acquisition unit 253 determines whether or not the focus position is set in the area where the focus is restricted, and determines that the focus position is set in the area. In this case, the second identification information may be acquired. Based on the relationship between the first identification information included in the focus restriction information and the acquired second identification information, the refocus image generation unit 254 determines the depth direction of the subject included in the focus restricted area. Refocus image data in which the focus position is set at a position different from the position is generated. Then, the display control unit 255 causes the display unit 230 to display an image based on the refocus image data. By doing so, the second user who views the image is registered as a friend of the first user included in the area where the focus is restricted at the time of display based on the default focus information in S113 of FIG. If not, the focus position is not set for the first user, and a specific person having a predetermined relationship with the first user can view the image of the first user while ensuring the privacy of the first user. It becomes possible to do so.

(Modification 3)
In the above description, when the second user is registered as a friend of the first user, the image processing apparatus 200 focuses on “Mr. B” who is the first user as shown in FIG. An example of displaying an image in which “Mr. A” is not in focus has been described. In contrast, when the second user is registered as a friend of the first user, the image processing apparatus 200 outputs an image in which both “Mr. A” and “Mr. B” are in focus. Good. In this way, when “Mr. A” is the second user, the second user can view an image in which both the first user who is in a friend relationship and himself / herself are in focus. Become.

(Modification 4)
In the above description, the example in which the focus restriction information includes the coordinates of the area to which the focus restriction is applied has been described. However, instead of the coordinates of the area to which the focus restriction is applied, or together with the coordinates of the area to which the focus restriction is applied, Depth information corresponding to the area to be applied may be included in the focus restriction information. The depth information corresponding to the area where the focus is restricted is information indicating the range of the focus position in the depth direction where the subject included in the area where the focus is restricted is focused. Based on the relationship between the first identification information and the second identification information, the image processing apparatus 200 controls whether or not the focus is adjusted to the focus position range indicated by the depth information included in the focus restriction information. A specific person having a predetermined relationship with the first user can view the image of the first user while ensuring the privacy of the first user.

[Effect in the first embodiment]
As described above, in the information sharing system 1 according to the first embodiment, the image editing device 100 associates with the first user in association with at least a partial area included in the LF image acquired from the first user. Focus restriction information that includes corresponding first identification information and restricts setting of the focus position is generated, and LF image data to which the focus restriction information is added is output. In the information sharing system 1 according to the first embodiment, the image processing apparatus 200 acquires the first identification information corresponding to at least a part of the region included in the LF image and browses the LF image. The second identification information corresponding to the two users is acquired, and the focus position is set at a position different from the position in the depth direction of the partial area based on the relationship between the first identification information and the second identification information. Output the refocused image data.

  By doing in this way, when the second user browses the LF image, the second user is predetermined with the first user corresponding to the first identification information included in the partial area focus restriction information of the LF image. When there is no relationship, it is impossible to view an image in which the partial area is in focus. Therefore, it is possible to allow a specific person to visually recognize the subject while preventing the subject included in the LF image data from being visually recognized by an unspecified number of people. As a result, for example, a specific person having a predetermined relationship with the first user can view the image of the first user while ensuring the privacy of the first user included in a partial area of the LF image data. be able to.

<Second Embodiment>
Next, the second embodiment will be described. In the first embodiment, an example in which the image editing apparatus 100 is a terminal used by the first user and the image processing apparatus 200 is a terminal operated by the second user has been described, but the second embodiment is described. Is different from the first embodiment in that the server 400 on the cloud functions as an image editing device and an image processing device. The second embodiment will be described below. In the following description, the description of the same configuration as in the first embodiment will be omitted as appropriate.

[Configuration of Information Sharing System 1]
FIG. 14 is a diagram illustrating a configuration of the information sharing system 1 according to the second embodiment. The information sharing system 1 includes a database 300, a server 400, and a plurality of user terminals 500 (500a, 500b, 500c). The server 400 has the same functions as those of the image editing apparatus 100 and the image processing apparatus 200 according to the first embodiment. The user terminal 500 is a communication terminal such as a smartphone, for example, and uploads LF image data to the database 300 via the server 400 or displays an image based on the LF image data provided via the server 400. be able to. In the present embodiment, the user terminal 500a is a first user terminal that attaches focus restriction information to LF image data, and the user terminal 500b is a second user terminal that views a refocus image based on the LF image data. Shall.

  FIG. 15 is a diagram illustrating a configuration of the server 400 according to the second embodiment. In FIG. 15, a configuration unit having the same name as that of the image processing apparatus 200 illustrated in FIG. 10 has a function equivalent to that of the configuration unit included in the image processing apparatus 200. The server 400 stores the LF image data uploaded from the user terminal 500a connected via the communication line in the database 300. The server 400 also provides the LF image data stored in the database 300 to the user terminal 500b connected via the communication line.

The server 400 includes a communication unit 410, a storage unit 440, and a control unit 450.
The communication unit 410 is a communication interface for transmitting and receiving LF image data and various types of information between the database 300 and the user terminal 500.
The storage unit 440 includes a nonvolatile memory, and stores an image processing program executed by the control unit 450. The storage unit 440 includes a volatile memory, and temporarily stores various data generated by the control unit 450 executing the image processing program.

  The control unit 450 is, for example, a CPU, and controls each unit of the server 400 by executing an image processing program stored in the storage unit 440. The control unit 450 functions as an image acquisition unit 452, an information acquisition unit 453, a refocus image generation unit 454, a focus restriction information generation unit 456, an image output unit 457, and an output unit 458. Hereinafter, the configuration of each functional unit provided by the control unit 450 will be described.

The image acquisition unit 452 receives LF image data to be uploaded to the database 300 from the user terminal 500 via the communication unit 410.
The information acquisition unit 453 receives the upload request transmitted from the user terminal 500a together with the LF image data to be uploaded. The upload request includes first identification information that is user identification information for identifying the first user of the user terminal 500a that performed the upload. The information acquisition unit 453 acquires the uploaded user profile information of the first user from the database 300 based on the first identification information. The information acquisition unit 453 acquires the face image information of the first user from the user profile information. The information acquisition unit 453 outputs the first identification information and the extracted face image information to the focus restriction information generation unit 456.

  When the information acquisition unit 453 acquires the face image information, the focus restriction information generation unit 456 generates focus restriction information that restricts setting of the focus position in at least one region included in the LF image data. The function of the focus restriction information generation unit 456 is the same as the function of the focus restriction information generation unit 153 according to the first embodiment, and thus the description thereof is omitted.

  The image output unit 457 outputs the LF image data with the focus restriction information generated by the focus restriction information generation unit 456 to the database 300. The functions of the image output unit 457 are the same as the functions of the image output unit 154 according to the first embodiment, and thus description thereof is omitted.

  The information acquisition unit 453 receives LF image reproduction start instruction information from the user terminal 500b via the communication unit 410. The reproduction start instruction information includes second identification information that is user identification information for identifying the second user of the user terminal 500b that has issued the reproduction start instruction. Based on the second identification information, the information acquisition unit 453 receives from the database 300 user profile information of the second user of the user terminal 500b that has given the playback start instruction. The user profile information of the second user acquired by the information acquisition unit 453 includes friend registration information of the second user.

  The refocus image generation unit 454 refocus image data in which the focus position is set at a position different from the position in the depth direction of the subject included in at least a part of the area included in the LF image data based on the focus restriction information. Is generated. The functions of the refocus image generation unit 454 are the same as the functions of the refocus image generation unit 254 according to the first embodiment, and thus description thereof is omitted.

  The output unit 458 outputs the refocus image data generated by the refocus image generation unit 454 to the user terminal 500b that has transmitted the reproduction start instruction information. The output unit 458 transmits the refocus image data in which the focus position is set at a position different from the position in the depth direction of the partial area of the LF image based on the focus restriction information, and transmits the reproduction start instruction information. Output to 500b. The user terminal 500b has a display control unit that displays the refocus image data acquired from the server 400 on its own display unit, and is included in the LF image in response to the acquisition of the refocus image data. It is possible to display an image in which the area is blurred on its own display unit.

[Flowchart showing Operation Procedure in Information Sharing System 1]
FIG. 16 is a flowchart illustrating an operation procedure when uploading LF image data of the server 400 and the user terminal 500a according to the second embodiment.
First, when receiving an instruction to upload LF image data from the first user, the user terminal 500a acquires LF image data to be uploaded from its own storage unit and transmits it to the server 400 (S201).
The image acquisition unit 452 acquires LF image data transmitted from the user terminal 500a (S202).

Subsequently, the user terminal 500a acquires the first identification information in response to receiving the LF image data upload instruction from the first user, and transmits the first identification information to the server 400 (S203).
The information acquisition unit 453 receives the first identification information transmitted from the user terminal 500a (S204).
The processing from S205 to S210 is the same as the processing from S104 to S109 in the flowchart showing the operation procedure of the image editing apparatus 100 shown in FIG.

FIG. 17 is a flowchart illustrating an operation procedure when the server 400 and the user terminal 500b according to the second embodiment display an LF image.
First, the user terminal 500b displays an operation screen for selecting LF image data on its display unit, and accepts an instruction to start playback of the LF image (S211). The user terminal 500b transmits image specifying information for specifying the LF image data to the server 400 in response to selection of LF image data to be displayed on the operation screen and reception of a reproduction start instruction.

Subsequently, the image acquisition unit 452 acquires LF image data from the database 300 based on the image specifying information received from the user terminal 500b (S212).
Subsequently, the refocus image generation unit 454 reconstructs the LF image data based on the default focus information included in the LF image data acquired in S212. The output unit 458 transmits the refocused image data that has been reconstructed to the user terminal 500b (S214).
When the user terminal 500b receives image data (S214), the display control unit of the user terminal 500b displays an image on its own display unit based on the image data (S215).

  Subsequently, the user terminal 500b receives an instruction from the user to select a subject to be newly focused (S216). The user terminal 500b transmits subject selection information indicating that a subject has been selected to the server 400. The subject selection information includes image specifying information and position information indicating the position selected in the LF image.

When the information acquisition unit 453 of the server 400 receives the subject selection information, the image acquisition unit 452 acquires LF image data from the database 300 based on the image specifying information included in the subject selection information (S217).
Based on the position information included in the subject selection information, the refocus image generation unit 454 determines whether or not focus restriction information is given to the display area corresponding to the selected subject in the LF image data (S218). ). If the refocus image generation unit 454 determines that the focus restriction information is provided, the process proceeds to S219. If the refocus image generation unit 454 determines that the focus restriction information is not provided, the process proceeds to S221.
The processing from S219 to S222 is the same as the processing from S116 to S119 of the flowchart showing the operation procedure of the image processing apparatus 200 shown in FIG.

In S223, the output unit 458 transmits the refocus image data generated in S222 to the user terminal 500b.
When the user terminal 500b receives the refocus image data (S224), the display control unit of the user terminal 500b displays an image based on the refocus image data on its display unit (S225).

[Effects of Second Embodiment]
As described above, in the information sharing system 1 according to the second embodiment, the server 400 has functions equivalent to those of the image editing apparatus 100 and the image processing apparatus 200 according to the first embodiment. In this way, even if the user terminal 500 is a general-purpose terminal, the user can select an image in which a subject included in a partial area of the LF image is focused by an unspecified number of people. It is possible to prevent browsing.

<Third Embodiment>
Subsequently, a third embodiment will be described. The third embodiment is different from the second embodiment in that focus restriction information is collectively added to a plurality of LF image data uploaded to the database 300 on the cloud. Hereinafter, the third embodiment will be described with reference to a flowchart showing operation procedures of the server 400 and the user terminal 500a when the focus restriction information is given to the LF image data.

FIG. 18 is a flowchart illustrating an operation procedure related to batch provision of focus restriction information to LF image data of the server 400 and the user terminal 500a according to the third embodiment.
First, the user terminal 500a displays a selection screen for selecting an LF image to which focus restriction information is added from a plurality of LF image data uploaded by the first user on its display unit. The user terminal 500a accepts an instruction to collectively apply focus restriction information to a plurality of selected LF image data. The user terminal 500a transmits instruction information for instructing batch provision to the server 400 (S301). The instruction information includes image specifying information for each of the plurality of LF image data.

  When the information acquisition unit 453 receives the instruction information for instructing batch provision of focus restriction information transmitted from the user terminal 500a, the image acquisition unit 452 reads from the database 300 based on the image specifying information included in the instruction information. LF image data is acquired (S302).

Subsequently, the user terminal 500a acquires the first identification information for identifying the first user from its own storage unit in response to receiving the collective provision instruction of the focus restriction information from the first user. 400 is transmitted (S303).
The information acquisition unit 453 receives the first identification information transmitted from the user terminal 500a (S304).
Since the processing from S305 to S309 is the same as the processing from S205 to S209 of the flowchart showing the operation procedure of the server 400 shown in FIG. 16 in the second embodiment, the description thereof is omitted.

In step S310, the focus restriction information generation unit 456 determines whether or not the processing shown in steps S307 to S309 has been performed on all the LF image data to which focus restriction information is collectively applied. If the focus restriction information generation unit 456 determines that processing has been performed for all LF image data, the focus restriction information generation unit 456 proceeds to S <b> 311, and determines that processing has not been performed for all LF image data, shifts the processing to S <b> 307.
In step S <b> 311, the image output unit 457 transmits all processed LF image data to the database 300 and updates the LF image data in the database 300.

[Effect in the third embodiment]
As described above, since the server 400 according to the third embodiment can collectively apply focus restriction information to a plurality of LF image data, the user wants to upload a large number of LF image data. In addition, user convenience can be improved.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment, A various deformation | transformation and change are possible within the range of the summary. is there.

DESCRIPTION OF SYMBOLS 100 ... Image editing apparatus 151 ... Image acquisition part 152 ... Information acquisition part 154 ... Image output part 200 ... Image processing apparatus 252 ... Image acquisition part 253 ... Information acquisition part 255 ... Display control unit

Claims (14)

Image acquisition means for acquiring a light field image that can be changed after shooting a focus position that is a position in the depth direction in focus;
Information acquisition for acquiring first identification information of a first user corresponding to at least a part of a region included in the light field image and acquiring second identification information corresponding to a second user browsing the light field image Means,
Based on the relationship between the first identification information and the second identification information, refocus image data in which the focus position is set at a position different from the position in the depth direction of the subject included in the partial area is output. Output means for
An image processing apparatus comprising: The information acquisition means acquires the first identification information included in restriction information indicating that the setting of the focus position is restricted to a position in a depth direction of a subject included in the partial area;
The output means sets the focus position at a position different from the position in the depth direction of the subject included in the partial area when the first identification information and the second identification information are not associated with each other. Outputting the refocus image data.
The image processing apparatus according to claim 1. The output means includes an image output before outputting the refocus image data and a position or region designated by the second user included in the partial region in an image in which a focus position is set to a default position. The refocus image data in which the focus position is set at a position different from the position in the depth direction of the subject included in the partial area is output.
The image processing apparatus according to claim 1. The information acquisition means acquires the relationship information from a server storing relationship information indicating a relationship between the first identification information and the second identification information;
The output means determines whether or not the first identification information and the second identification information have a predetermined relationship based on the relationship information.
The image processing apparatus according to claim 1. The output means sets the focus position when outputting the refocus image data in which the focus position in the partial area is set to a position different from the position in the depth direction of the partial area. It is characterized by outputting information indicating that there are restrictions.
The image processing apparatus according to claim 1. The information acquisition means acquires coordinate information indicating coordinates of a rectangular area including the partial area,
The output means outputs refocus image data in which the focus position is set at a position different from the position in the depth direction of the subject corresponding to the face image included in the rectangular area.
The image processing apparatus according to claim 1. In response to receiving an instruction to acquire a light field image that can be changed after shooting a focus position that is a position in the depth direction in focus, an image acquisition unit that acquires the light field image;
Information acquisition means for acquiring user identification information corresponding to the user;
Restriction information that includes user identification information corresponding to the user in association with at least a part of the area included in the light field image and restricts setting of the focus position on a subject included in the part of the area. Generating means for generating
Output means for outputting the light field image to which the restriction information generated by the generation means is attached;
An image editing apparatus comprising: The information acquisition means acquires area specifying information used for specifying the partial area,
The generation unit generates the restriction information corresponding to the partial region when the information acquisition unit acquires the region specifying information.
The image editing apparatus according to claim 7. The information acquisition means acquires the user's face image information as the region specifying information,
The generation unit generates the restriction information corresponding to the face image area when the light field image includes a face image area corresponding to the face image information.
The image editing apparatus according to claim 8. An image processing method executed by an image processing apparatus,
An image acquisition step for acquiring a light field image that can be changed after shooting a focus position that is a position in the depth direction in focus;
A first identification information acquisition step of acquiring first identification information of a first user corresponding to at least a part of the area included in the light field image;
A second identification information acquisition step of acquiring second identification information corresponding to a second user viewing the light field image;
A refocus image in which the focus position is set at a position different from the position in the depth direction of the subject included in the partial area based on the relationship between the acquired first identification information and the second identification information. An output step for outputting data;
An image processing method comprising: In the first identification information acquisition step, the first identification information included in the focus restriction information indicating that the setting of the focus position is restricted to the position in the depth direction of the subject included in the partial area Acquired,
In the output step, when the first identification information and the second identification information are not associated with each other, the focus position is set at a position different from the position in the depth direction of the subject included in the partial area. Output refocused image data.
The image processing method according to claim 10. An image editing method executed by an image editing apparatus,
An image acquisition step of acquiring the light field image in response to receiving an instruction to acquire a light field image that can be changed after shooting the focus position that is a position in the depth direction in focus;
An information acquisition step of acquiring user identification information corresponding to the user;
A restriction that includes first identification information corresponding to the user in association with at least a part of the area included in the light field image, and restricts setting of the focus position on a subject included in the part of the area. A generation step for generating information;
An output step of outputting the light field image to which the restriction information generated in the generation step is attached;
An image editing method characterized by comprising: An information sharing system comprising a first user terminal used by a first user, a second user terminal used by a second user, and an image processing device,
The image processing apparatus includes:
In response to an instruction from the first user terminal, an image acquisition unit that acquires a light field image that can be changed after shooting a focus position that is a position in the depth direction in focus;
Information acquisition means for acquiring first identification information corresponding to at least a part of the region included in the light field image and acquiring second identification information corresponding to the second user;
Based on the relationship between the first identification information and the second identification information, the refocus image data in which the focus position is set at a position different from the position in the depth direction of the subject area included in the part is Output means for outputting to the second user terminal,
The second user terminal is
It has a display control means for displaying the refocus image data acquired from the image processing device on a display unit,
Information sharing system.   The program for functioning a computer as each means of the image processing apparatus of any one of Claim 1 to 6.