JP6632384B2 - Image reproducing apparatus and image reproducing method - Google Patents

Description

  The present invention relates to an image reproducing apparatus and an image reproducing method for reproducing an image.

  2. Description of the Related Art In recent years, an imaging device called a light field (hereinafter abbreviated as “LF”) camera has been put to practical use. In this imaging device, a microlens array is arranged on an image sensor, and light incident on the imaging device is divided by a microlens, so that light beam information indicating the intensity of light beams arriving from a plurality of directions and a direction indicating an incident direction. LF image data including information is generated. Patent Literature 1 discloses a method of reconstructing an image set to an arbitrary focus by performing a predetermined refocus calculation process on LF image data.

JP 2009-159357 A

  When continuously reproducing a large number of image data, the position in the depth direction where the focus is focused may be gradually changed. In such a case, there is a problem that it is complicated to set a focus position for each of a large number of image data.

  Therefore, the present invention has been made in view of the above points, and has as its object to facilitate setting of a focus position when reproducing a large number of image data.

The image reproducing apparatus according to the present invention includes a storage unit configured to store a plurality of light field image data in which a focus position, which is a focused depth position, can be changed after shooting, and the plurality of light field image data is a plurality of images. Receiving means for receiving a first focus position set in the first image data and a second focus position set in the second image data when reproduced as moving image data including data; setting means for the third image the 3 focus position used when reproducing the data to be played between the second image data is set based on the first focus position and the second focus position, the first image data, have a, a reproducing means for reproducing the moving image data including the second image data and third image data, the receiving The means further receives an instruction to select the third image data from the plurality of image data, and the reproducing means corresponds to the instruction received by the receiving means, the focus position being set to the third focus position. characterized Rukoto is displayed on the display unit of the third image data.

In the image reproducing method according to the present invention, a plurality of light field image data that can be changed after photographing a focus position, which is a focused depth position, which is executed by a computer, is reproduced as moving image data including a plurality of image data. Receiving a first focus position set in the first image data and a second focus position set in the second image data when reproducing, and reproducing between the first image data and the second image data. Setting a third focus position used when reproducing the third image data to be performed based on the first focus position and the second focus position; and setting the first image data, the second image data, and the And a reproducing step of reproducing the moving image data including three image data. Further accepts an instruction for selecting the third image data from the image data, in the regeneration step, the focus position is set to the third focusing position, the third image data corresponding to the instruction accepted in the accepting step It is characterized by being displayed on a display unit.

  According to the present invention, it is possible to easily set a focus position when a large number of image data are reproduced.

FIG. 2 is a schematic diagram of a configuration of an imaging device as an LF camera. FIG. 2 is a schematic diagram illustrating a positional relationship between a microlens array 12 and each pixel of an image sensor 13. FIG. 2 is a schematic diagram for explaining a relationship between a traveling direction of a light beam incident on a microlens of a microlens array and a recording area of an image sensor. FIG. 3 is a schematic diagram for describing information on light rays incident on an image sensor. FIG. 9 is a schematic diagram for explaining refocus calculation processing. FIG. 3 is a schematic diagram illustrating a relationship between a difference in incident angle to a microlens and a recording area of an image sensor. It is a schematic diagram for explaining the adjustment processing of the depth of field. FIG. 9 is a diagram illustrating three image data reproduced based on three LF image data included in the LF moving image data. FIG. 2 is a diagram illustrating a configuration of the image reproducing device 1. FIG. 4 is a diagram illustrating an example of a relationship between a focus position and a reproduction time. FIG. 5 is a diagram showing a focus position setting screen 30 for a user to set a focus position. FIG. 5 is a diagram for describing a method in which a setting unit sets a focus position. 5 is a flowchart of an operation in which the image reproducing device 1 determines a focus position. FIG. 14 is a diagram for describing an operation of the image reproducing device 1 according to a third modification. FIG. 15 is a diagram illustrating a relationship between a reproduction time and a focus position used in Modification Example 3.

[Overview of LF camera]
Prior to the description of the image reproducing apparatus according to the present embodiment, an outline of the LF camera will be described.
FIG. 1 is a schematic diagram of a configuration of an imaging device 10 as an LF camera. The imaging device 10 includes an imaging lens 11, a micro lens array 12, and an image sensor 13. Light from a subject passes through an imaging lens 11 and a microlens array 12 that constitute an imaging optical system, and then enters an image sensor 13. The image sensor 13 converts the incident light into an electric signal and outputs the converted electric signal. The imaging device 10 generates LF image data capable of changing a focus position, which is a focused depth position, based on an electric signal output from the image sensor 13.

  The imaging lens 11 projects light from a subject onto the microlens array 12. The imaging lens 11 is an interchangeable lens that is used by being attached to the main body of the imaging device 10. The user of the imaging device 10 can change the imaging magnification by performing a zoom operation on the imaging lens 11.

  The microlens array 12 has a plurality of microlenses (microlenses) arranged in a lattice and is located between the imaging lens 11 and the image sensor 13. Each micro lens included in the micro lens array 12 splits the incident light from the imaging lens 11 and makes the plurality of split lights incident on a plurality of pixels of the image sensor 13.

  The image sensor 13 is an image sensor having a plurality of pixels, and detects light intensity at each pixel. Each pixel of the image sensor 13 that receives light from the subject receives light in a different direction, which is divided by a microlens corresponding to each pixel.

  FIG. 2 is a schematic diagram showing a positional relationship between the microlens array 12 and each pixel of the image sensor 13. Each micro lens of the micro lens array 12 is arranged so as to correspond to a plurality of pixels in the image sensor 13. Light divided by a microlens corresponding to each pixel is incident on each pixel of the image sensor 13, and the image sensor 13 detects the intensity of light from a different direction for each pixel. The imaging device 10 generates light beam information indicating the intensity of light detected at each pixel of the image sensor 13.

  The incident direction of the light beam incident on each pixel of the image sensor 13 via the micro lens is determined by the arrangement of a plurality of pixels corresponding to each micro lens. Therefore, the incident direction is specified according to the positional relationship between each microlens and each pixel of the image sensor 13, and the imaging device 10 may use, for example, coordinates for specifying the pixel as direction information indicating the specified incident direction. Generate information.

  Light from the subject enters the microlens array 12 in a direction determined according to the distance between the subject and the imaging lens 11. Then, light from the subject passes through a plurality of microlenses determined according to the distance between the subject and the imaging lens 11 and enters the image sensor 13. As a result, an image focused on a surface of the microlens array 12 at a different distance from the lens apex surface has a plurality of pixels of the image sensor 13 at a position corresponding to the amount of eccentricity of each microlens from the optical axis. Obtained by combining the outputs. Therefore, the imaging apparatus 10 can generate image data corresponding to an arbitrary focus position by performing light ray rearrangement and calculation processing (hereinafter, referred to as reconstruction) based on light ray information and direction information. Here, the focus position is a focused position, and the user can obtain refocus image data corresponding to a desired focus position based on the LF image data after shooting.

FIG. 3 is a schematic diagram for explaining a relationship between a traveling direction of a light beam incident on a microlens of the microlens array 12 and a recording area of the image sensor 13.
The image of the subject by the imaging lens 11 is formed on the microlens array 12, and the light incident on the microlens array 12 is received by the image sensor 13 via the microlens array 12. At this time, as shown in FIG. 3, light rays incident on the microlens array 12 are received at different positions on the image sensor 13 according to the traveling direction determined for each focus position, and an image of the subject is formed for each microlens. Image.

FIG. 4 is a schematic diagram for describing information on light rays incident on the image sensor 13.
The light beam received by the image sensor 13 will be described with reference to FIG. Here, the orthogonal coordinate system on the lens surface of the imaging lens 11 is (u, v), and the orthogonal coordinate system (x, y) on the imaging surface of the image sensor 13 is. Further, the distance between the lens surface of the imaging lens 11 and the imaging surface of the image sensor 13 is F. In this case, the intensity of the light beam passing through the imaging lens 11 and the image sensor 13 can be represented by a four-dimensional function L (u, v, x, y) shown in the figure. Since the light beam incident on each microlens is incident on a different pixel depending on the traveling direction, the image sensor 13 holds the four-dimensional function L ( Information corresponding to the intensity of the light beam represented by u, v, x, y) is recorded.

  Next, a description will be given of the refocus calculation processing after imaging. The refocus calculation process is a process of generating refocus image data by reconstructing an image based on the intensity of light received by pixels corresponding to different focus positions. In this specification, the surface at the position where the focus is in the refocus image is referred to as a refocus surface.

FIG. 5 is a schematic diagram for explaining the refocus calculation processing.
As shown in FIG. 5, when the positional relationship between the imaging lens surface, the imaging surface, and the refocus surface is set, the light intensity L ′ (u, v, s) in the rectangular coordinate system (s, t) on the refocus surface. , T) is represented by the following equation (1).
The image E '(s, t) obtained on the refocusing surface is obtained by integrating the intensity L' (u, v, s, t) with respect to the lens aperture. Is represented by
Therefore, by performing the refocus calculation processing based on the equation (2), it is possible to reconstruct the image data corresponding to the refocus plane at an arbitrary focus position.

  Next, the process of adjusting the depth of field after imaging, that is, the process of adjusting the focus position in the depth direction will be described. When adjusting the depth of field, prior to the above-described refocus calculation processing, weighting is performed by multiplying each image data forming an image area assigned to each microlens by a weighting coefficient. For example, when generating an image with a large depth of field, the integration process is performed using only information on light rays that enter the light receiving surface of the image sensor 13 at a relatively small angle. In other words, a light beam having a relatively large angle of incidence on the image sensor 13 is not included in the integration process by multiplying by a weighting factor of 0 (zero).

FIG. 6 is a schematic diagram for explaining the relationship between the difference in the incident angle to the microlens and the recording area of the image sensor 13, and FIG. 7 is a schematic diagram for explaining the depth of field adjustment process. is there.
As shown in FIG. 6, a light beam having a relatively small angle of incidence on the image sensor 13 is located in a more central region. Therefore, when generating an image with a large depth of field, as shown in FIG. 7, the integration process is performed using only the pixel data acquired at the center of the region (the hatched portion in the drawing).

By performing such processing, the imaging device 10 can express an image with a large depth of field, as if the aperture stop provided in a general imaging device or the like was narrowed. The imaging device 10 can also generate pan-focus image data with a deeper depth of field by further reducing the pixel data of the central part to be used. Further, the imaging device 10 can also adjust the depth of field to be different for each area by changing the position of the pixel data to be used for each area of the LF image data. As described above, the imaging device 10 adjusts the depth of field of the image after shooting based on the focus information including the weight coefficient of each pixel included in the LF image data, the light beam information, and the direction information, and It can be set to the focus position.
Hereinafter, embodiments of an image reproducing apparatus and an image reproducing method for reproducing LF image data will be described.

[Overview of Image Reproduction Apparatus 1]
The image reproducing device 1 according to the present embodiment can reproduce moving image data based on LF moving image data including a plurality of LF image data. The LF moving image data has a plurality of image frames (hereinafter, referred to as “frames”) each including the LF image data, and the image reproducing device 1 reconstructs the LF image data of each frame to obtain the image data. Moving image data can be reproduced by sequentially reproducing the focus images. When the user specifies the focus position of the LF image data of the two frames (the first frame and the second frame), the image playback device 1 automatically sets the focus position of the LF image data of the frame between the two frames. Set.

FIG. 8 is a diagram showing image data corresponding to three frames reproduced based on three LF image data included in the LF moving image data.
FIG. 8A shows the first image data D1 corresponding to the first frame to be reproduced first among the two frames specified by the user. FIG. 8A2 is a depth map showing a focus position in the depth direction of the first image data D1. Reference symbol R1 in FIG. 8A2 indicates a focus range where focus is achieved, and corresponds to the depth of field of the first image data D1. As shown in FIG. 8 (a2), in FIG. 8 (a1), the focus position is set at a position where the left object X is focused (hereinafter, referred to as a first focus position), and the right object Y is set at the right position. Is out of focus.

  FIG. 8C1 shows the second image data D2 corresponding to the second frame to be reproduced later among the two frames specified by the user. FIG. 8C2 is a depth map of the second image data D2. Reference sign R2 in FIG. 8 (c2) indicates a focus range in focus, and corresponds to the depth of field of the second image data D2. In FIG. 8 (c1), the focus position is set at a position where the right object Y is focused (hereinafter, referred to as “second focus position”), and the left object X is not focused.

  FIG. 8B1 shows the third image data D3 corresponding to a frame reproduced between the first frame and the second frame. FIG. 8B2 is a depth map of the third image data D3. When the user sets the first focus position and the second focus position, the image reproducing device 1 sets the focus position of the third image data D3 (hereinafter, the third focus position) based on the first focus position and the second focus position. ) Is set. As shown in FIG. 8 (b2), the third focus position is set in a range R3 between the first focus position and the second focus position. And Y are out of focus.

  The image reproducing apparatus 1 reproduces the first image data D1, the third image data D3, and the second image data D2 in this order, so that the state in which the object X is in focus is changed to the state in which the object Y is in focus. The focus position can be changed. When there is a large number of image data between the first image data D1 and the second image data D2, the image reproducing device 1 smoothly shifts from a state where the object X is focused to a state where the object Y is focused. The focus position can be changed.

[Configuration of Image Reproduction Apparatus 1]
Hereinafter, the configuration of the image reproducing device 1 will be described in detail.
FIG. 9 is a diagram illustrating a configuration of the image reproducing device 1 according to the present embodiment. The image playback device 1 includes a storage unit 20, a reception unit 21, a setting unit 22, a display unit 23, and a playback unit 24.

  The storage unit 20 is, for example, a storage medium such as a hard disk or a memory, and stores a plurality of LF image data in which a focus position, which is a focused depth position, can be changed after shooting. The storage unit 20 stores, for example, LF moving image data including a plurality of LF image data.

  The receiving unit 21 is configured to set a first focus position set to the first image data and a second focus set to the second image data when the plurality of LF image data is reproduced as moving image data including the plurality of image data. Accept the position. The receiving unit 21 receives, for example, a first focus position and a second focus position input by a user via a user interface such as a mouse and a touch panel.

  The accepting unit 21 accepts, as the first focus position, for example, instruction information for designating a focus range when the first image data is reproduced. Specifically, the accepting unit 21 accepts instruction information for designating the foremost position and the deepest position where the focus is achieved when the first image data is reproduced. The receiving unit 21 may receive instruction information indicating a first focus position that is the center position of the focus range and instruction information indicating the size of the focus range. The receiving unit 21 notifies the setting unit 22 of these pieces of instruction information.

  The setting unit 22 sets a focus position for each LF image data. When the setting unit 22 receives, from the reception unit 21, instruction information indicating the foremost position in focus and the foremost position in the first image data, the setting unit 22 sets the center position of the focus range defined by the position indicated by the instruction information. Set the first focus position. Similarly, when receiving the instruction information indicating the foremost position in focus and the foremost position in the second image data from the receiving unit 21, the second position is set at the center position of the focus range defined by the position indicated by the instruction information. Set the focus position.

  Then, the setting unit 22 determines a third focus position used at the time of reproducing the third image data reproduced between the first image data and the second image data based on the first focus position and the second focus position. Set. The setting unit 22 sets, for example, a third focus position between the first focus position and the second focus position. The setting unit 22 sets a plurality of third focus positions corresponding to a plurality of image data between the first image data and the second image data to a second focus position as the image data is reproduced closer to the second image data. Set to a position close to.

  FIG. 10 is a diagram illustrating an example of the relationship between the focus position and the reproduction time used by the setting unit 22 to set the third focus position. In FIG. 10A, the third focus position changes from the same front side to the rear side by a certain amount while the unit time elapses. In this case, the setting unit 22 calculates a third focus position between the two focus positions based on a linear function of the reproduction time. That is, the setting unit 22 calculates the third focus position such that the reproduction time and the third focus position have a linear relationship.

In FIG. 10B, the amount by which the third focus position changes decreases with time. In this case, the setting unit 22 calculates the third focus position between the two focus positions based on the logarithm of the reproduction time.
In FIG. 10C, the amount by which the third focus position changes increases with time. In this case, the setting unit 22 calculates a third focus position between the two focus positions based on a quadratic function of the reproduction time. As described above, the setting unit 22 can change the third focus position with the change of the reproduction time in different modes. The setting unit 22 may calculate the third focus position by using a polynomial such as spline curve interpolation other than the relationship shown in FIG.

  The storage unit 20 stores the reproduction time of the third image data between the first image data and the second image data and the third focus between the first focus position and the second focus position as shown in FIG. The relationship with the position is stored in advance. When the storage unit 20 stores a plurality of relationships between the third focus position and the reproduction time, the reception unit 21 displays an operation screen for the user to select the above relationship used for setting the third focus position on the display unit. 23, and receives an instruction to specify the relationship used for setting the third focus position. Then, the setting unit 22 sets the third focus position based on the relationship indicated by the instruction received by the receiving unit 21. In this way, the user can easily specify a focus position suitable for a scene or a display effect.

Returning to FIG. 9, the description of the configuration of the image reproducing device 1 will be continued.
The display unit 23 is, for example, a liquid crystal display. The display unit 23 displays a screen including the refocus image data generated by the playback unit 24 and a user interface image (hereinafter, referred to as a UI image).

  The reproduction unit 24 generates refocus image data by reconstructing the LF image data stored in the storage unit 20, and reproduces the generated refocus image data by outputting the generated refocus image data to the display unit 23. The reproduction unit 24 reproduces, for example, moving image data including the first image data D1, the second image data D2, and the third image data D3 illustrated in FIG. At this time, the reproducing unit 24 reproduces image data having a depth of field corresponding to the focus range received by the receiving unit 21.

The reproduction unit 24 includes a reconstructing unit 241, a screen generating unit 242, a depth map generating unit 243, a UI image generating unit 244, and a combining unit 245.
The reconstructing unit 241 reconstructs the LF image data acquired from the storage unit 20 based on the focus position set by the setting unit 22, and generates a refocused image.

The screen generation unit 242 generates screen data for displaying the refocused image generated by the reconstructing unit 241 on the display unit 23.
The depth map generation unit 243 generates the depth map shown in FIG. The UI image generation unit 244 generates UI image data for the user to set a focus position. For example, the UI image generation unit 244 generates UI image data by generating image data for a user to set a focus range and superimposing the image data on a depth map. The combining unit 245 combines the screen data generated by the screen generating unit 242 and the UI image data generated by the UI image generating unit 244 to generate display image data. The combining unit 245 outputs the generated display image data to the display unit 23.

[Focus position setting screen]
FIG. 11 is a diagram showing a focus position setting screen 30 for the user to set a focus position. The focus position setting screen 30 includes a first focus position setting screen 31, a second focus position setting screen 32, a preview screen 33, and a frame designation screen.

  The first focus position setting screen 31 is a screen for setting the first focus position, and corresponds to the first image data D1 corresponding to the first frame specified on the frame specifying screen 34 and the first image data D1. And a depth map. Cursors C1 and C2 for designating a focus range corresponding to the first focus position are displayed on the depth map.

  As the user moves the cursor C1 upward, the focus position becomes deeper. Further, the more the user moves the cursor C2 downward, the closer the focus position is. The area between the cursor C1 and the cursor C2 is the focus range. The depth of field increases as the distance between the cursor C1 and the cursor C2 increases, and the depth of field decreases as the distance between the cursor C1 and the cursor C2 decreases.

  The second focus position setting screen 32 is a screen for setting the second focus position, and corresponds to the second image data D2 corresponding to the second frame specified on the frame specifying screen 34 and the second image data D2. And a depth map. Cursors C1 and C2 for designating a focus range corresponding to the second focus position are displayed on the depth map.

  In the preview screen 33, the setting unit 22 is configured to set the first focus position and the second focus position based on the third focus position of the image data between the first image data and the second image data. A preview screen of the refocus image generated by the restructuring unit 241 is displayed. By pressing the preview playback button 331, the user can display any image data between the first frame and the second frame designated on the frame designation screen 34 on the preview screen.

  The frame designation screen 34 is a screen for the user to designate a frame for setting a focus position. On the frame designation screen 34, a cursor 341 for designating the first frame, a cursor 342 for designating the second frame, and a cursor 343 indicating the position of the frame displayed on the preview screen are displayed. The receiving unit 21 receives, via the cursor 343, an instruction to select one image data from a plurality of image data. In this case, the reproduction unit 24 causes the display unit 23 to display the third image data corresponding to the selection instruction received by the reception unit 21 in which the focus position is set to the third focus position.

  In a region 344, a refocus image based on the first image data and the second image data is displayed. In the area 345, a band image indicating the focus position is displayed. As described above, the reproducing unit 24 causes the display unit 23 to display the band image indicating the focus position set corresponding to each image data in association with the plurality of image data included in the moving image data. The focus position of the band image on the frame designation screen 34 is set closer to the upper side of the area 345, and the focus position is set closer to the lower side of the area 345, and the cursor 341 is moved to the near side. This indicates that the focus position changes from the near side to the far side from the first frame shown to the second frame shown by the cursor 342.

  In FIG. 11, discontinuous points occur in the band image before and after the cursor 341 and the cursor 342. This indicates that the scene changes before and after the cursor 341 and the cursor 342. When the scene does not change before and after the cursor 341 and the cursor 342, it is desirable that the band image continuously changes.

[How to set focus position]
FIG. 12 is a diagram for explaining a method in which the setting unit 22 sets a focus position. FIG. 12A shows refocus image data corresponding to four frames. In FIG. 12A, the first image data D1, the third image data D3, the fourth image data D4, the fourth image data D4, which are refocus images corresponding to four frames having frame numbers of 2000, 2060, 2120, and 2180, respectively. Two image data D2 is shown. The third image data D3 and the fourth image data D4 are two image data among a plurality of image data between the first image data D1 and the second image data D2. The focus positions of the third image data D3 and the fourth image data D4 are based on the first focus position of the first frame of frame number 2000 and the second focus position of the second frame of frame number 2180 specified by the user. Set to position.

  FIG. 12B is a depth map corresponding to each of the refocused image data shown in FIG. This depth map is a user interface for the user to specify the first focus position of the first frame of frame number 2000 and the second focus position of the second frame of frame number 2180. The depth map also functions as a user interface that displays a focus position of image data corresponding to a frame between a plurality of frames for which the user has designated a focus position.

  The depth map shows a numerical value 0 indicating the foreground focus position and a numerical value 100 indicating the deepest focus position. The user can use the cursor displayed on the depth maps of the first frame and the second frame. By moving C1 and C2, a focus range R1 in the first frame and a focus range R2 in the second frame are designated. The setting unit 22 sets the first focus position at the center position of the focus range R1 specified by the user, and sets the second focus position at the center position of the focus range R2.

  The setting unit 22 sets a third focus position, which is a focus position of the third frame, and a fourth focus position, which is a focus position of the fourth frame, between the first focus position and the second focus position. According to FIG. 12B, it can be seen that the focus position gradually changes from the near side (0) to the far side (100) from the first frame of the frame number 2000 to the second frame of the frame number 2180.

  The setting unit 22 sets a focus range R3 in the third frame and a focus range R4 in the fourth frame based on the focus range R1 and the focus range R2. For example, when the focus range R1 and the focus range R2 are equal, the setting unit 22 sets the focus range R3 and the focus range R4 to the same size as the focus range R1 and the focus range R2. When the focus range R2 is larger than the focus range R1, the setting unit 22 sets a plurality of frames between the first frame and the second frame such that the focus range gradually increases from the focus range R1 to the focus range R2. Set the focus range. The setting unit 22 sets the focus range R3 to the same value as the focus range R1 corresponding to the first frame in which the third frame is a close frame among the first frame and the second frame, and sets the focus range R3 in the frame in which the fourth frame is close. The focus range R4 may be set to the same value as the focus range R2 corresponding to a certain second frame.

  FIG. 12C is a table showing the relationship between the frame number and the focus position. As shown in FIG. 12C, it can be seen that the focus position changes from the near side to the far side as the frame number increases. FIG. 12C shows an example in which the focus range of each frame has the same value. However, the present invention is not limited to this, and the focus range of each frame may change.

Subsequently, a procedure in which the setting unit 22 determines the focus positions of a plurality of frames between the first frame and the second frame based on the first focus position of the first frame and the second focus position of the second frame explain.
As shown in FIG. 12C, the range of the first focus position of the first frame is set to 10 to 20. The range of the second focus position of the second frame is set to 80 to 90. When the user specifies the focus position range of the two frames, the setting unit 22 changes the frame numbers from the frame number 2001 to the frame number 2179, which are the frames between the first frame of the frame number 2000 and the second frame of the frame number 2180. Calculate the focus position.

  The setting unit 22 calculates, based on the focus positions of the two frames, the focus positions of a plurality of frames between the two frames so as to linearly change with a change in the frame number. As a result, the focus position of each frame from the frame number 2001 to the frame number 2179 calculated by the setting unit 22 is as shown in FIG.

Here, the front side of the focus range of the n-th frame is FFn, and the back side is FEn. The frame number of the first frame for which the user has designated the focus position is s, and the frame number of the second frame is e. Also, the front of the focus range of the s-th frame is FFs, the back side is FEs, the front of the focus range of the e-th frame is FFe, and the back side is FEe. In this case, the setting unit 22 can calculate the focus ranges FFn to FEn of each frame based on the following equations (3) and (4).
FFn = FFs + (FFe-FFs) / (es) × n (3)
FEn = FEs + (FEe-FEs) / (es) × n (4)

[Flowchart of operation for determining focus position]
FIG. 13 is a flowchart of an operation in which the image reproducing device 1 determines a focus position. First, the receiving unit 21 receives a first focus position of the first frame (S11). The receiving unit 21 receives a second focus position of the second frame (S12).

  Subsequently, the setting unit 22 sets a focus position of another frame between the first frame and the second frame based on the first focus position and the second focus position received by the receiving unit 21 (S13). . The setting unit 22 assigns the focus position set in S13 to the LF image data corresponding to each frame (S14). Specifically, the setting unit 22 updates the information indicating the focus position of the LF image data stored in the storage unit 20 to the information indicating the set focus position. According to the above procedure, the focus positions of the plurality of LF image data stored in the storage unit 20 can be sequentially changed in a range from the first focus position to the second focus position.

<Modification 1>
In the above description, the storage unit 20 stores the LF moving image data composed of a plurality of LF image data, and the reproducing unit 24 reproduces the moving image data whose focus position sequentially changes to the position set by the setting unit 22. Although the example which performs is explained, it is not limited to this. The storage unit 20 may store the LF still image data as LF image data, and the playback unit 24 may sequentially play back a plurality of refocus image data in which the focus position is set at the position set by the setting unit 22. By doing so, even when one piece of LF still image data is reproduced as a slide show, the user merely sets at least two focus positions and sequentially changes the focus positions to transfer a plurality of refocus image data. It becomes possible to reproduce.

<Modification 2>
In the above description, the user specifies the in-focus range in the first image data corresponding to the first frame and the in-focus range in the second image data corresponding to the second frame, so that the receiving unit 21 Has described the case of receiving the first focus position and the second focus position, but the method of receiving the first focus position and the second focus position is not limited to this. The receiving unit 21 may receive designation of an object to be focused in the first image data and the second image data.

  In this case, the receiving unit 21 sets a position where the object specified in the first image data is in focus as the first focus position. Further, the receiving unit 21 sets a position where the object specified in the second image data is in focus as a second focus position. Then, when the receiving unit 21 receives an instruction to specify an object, the setting unit 22 determines a third focus position between the first focus position and the second focus position at a position where the object is in focus. That is, the setting unit 22 sets the focus position of the LF image data of the frame between the first frame and the second frame to a position where the designated object is focused. The setting unit 22 determines the focus position of the specified object by referring to, for example, information included in the LF image data and indicating the position where each pixel is focused.

  The setting unit 22 may determine the third focus position using the relationship between the reproduction time and the focus position illustrated in FIG. 10 and the position of the object in the depth direction. For example, when the position in the depth direction of the object that the user wants to focus on is not included in the focus range determined based on the relationship between the reproduction time and the focus position shown in FIG. At the same time, the depth of field is increased so that the object is in focus. In this case, the receiving unit 21 may receive designation of an object to be focused only on the first image data. By doing so, even if the position in the depth direction of the object that the user wants to focus on is not included in the focus range determined based on the relationship between the reproduction time and the focus position shown in FIG. Thus, it is possible to maintain a state in which an object designated by the user is in focus. Note that, instead of accepting the second focus position, the accepting unit 21 may accept designation of an object to be focused in the second frame, and may set a position at which the accepted object is in focus as the second focus position.

<Modification 3>
In the above description, the case where the setting unit 22 sets the focus position of the image data between the first image data and the second image data has been described. However, the setting unit 22 sets the focus position after the second image data. The focus positions of the plurality of image data may be set based on the first focus position and the second focus position. The setting unit 22 sets, for example, a focus position of one or more pieces of image data before a scene change occurs among a plurality of pieces of image data after the second image data based on the first focus position and the second focus position. decide. When the setting unit 22 determines the focus positions of a plurality of image data after the second image data, the relationship shown in FIG. 10 can be used.

  FIG. 14 is a diagram for describing an operation of the image reproducing device 1 according to the third modification. FIG. 15 is a diagram illustrating the relationship between the reproduction time and the focus position used in the third modification. When the user specifies the first focus position of the first image data D1 corresponding to the frame number 1000 and the second focus position of the second image data D2 corresponding to the frame number 1360, the setting unit 22 sets the reproduction time t3 A focus position is set from the third image data D3 reproduced at the time to the fourth image data D4 reproduced at the reproduction time t4. The third image data D3 is image data of the frame number 1361 reproduced next to the second image data D2. The fourth image data D4 is the image data of the frame number 1365 immediately before the scene change occurs.

  In addition, when the focus position of the image data after the second image data D2 reaches the innermost position or the foremost position, the setting unit 22 sets the focus position of the subsequent image data to the innermost position or It may be fixed at the foremost position.

  The receiving unit 21 receives an instruction to specify a range of image data for which a focus position is determined based on the first focus position of the first image data D1 and the second focus position of the second image data D2. You may. For example, the reception unit 21 displays an operation screen for selecting whether to automatically set the focus position of the image data after the second image data on the display unit 23, thereby receiving the user's instruction. be able to. In addition, the receiving unit 21 may receive an instruction for specifying up to which image data of the image data after the second image data the focus position is to be set.

[Effects of Image Reproducing Apparatus 1 of Present Embodiment]
As described above, in the image reproducing apparatus 1 according to the first embodiment, the receiving unit 21 sets the first focus position set to the first image data and the second focus position set to the second image data. The setting unit 22 receives the position, and sets the third focus used when reproducing the third image data reproduced between the first image data and the second image data based on the first focus position and the second focus position. Set the position. By doing so, the user can reproduce moving image data composed of a plurality of refocused image data while sequentially changing the focus position only by setting the focus position of some LF still image data. Will be possible.

As described above, the present invention has been described using the embodiment, but the technical scope of the present invention is not limited to the scope described in the above embodiment, and various modifications and changes are possible within the scope of the gist. is there.
For example, in the image reproducing device 1, the above operation can be realized by the CPU reading and executing a storage medium storing a program for executing the image reproducing process. In this case, for example, the CPU functions as the receiving unit 21, the setting unit 22, and the reproducing unit 24.

DESCRIPTION OF SYMBOLS 1 ... Image reproduction apparatus 20 ... Storage part 21 ... Reception part 22 ... Setting part 24 ... Reproduction part

Claims (10)

Storage means for storing a plurality of light field image data that can be changed after shooting a focus position that is a focus position in the depth direction,
When the plurality of light field image data is reproduced as moving image data including a plurality of image data, a first focus position set in the first image data and a second focus position set in the second image data are received. Reception means,
A setting for setting a third focus position used when reproducing the third image data reproduced between the first image data and the second image data, based on the first focus position and the second focus position. Means,
Reproducing means for reproducing the moving image data including the first image data, the second image data, and the third image data,
The receiving unit further receives an instruction to select the third image data from the plurality of image data,
The image reproducing apparatus according to claim 1, wherein said reproducing means causes a display unit to display said third image data corresponding to an instruction received by said receiving means, wherein a focus position is set to said third focus position. Storage means for storing a plurality of light field image data that can be changed after shooting a focus position that is a focus position in the depth direction,
When the plurality of light field image data is reproduced as moving image data including a plurality of image data, a first focus position set in the first image data and a second focus position set in the second image data are received. Reception means,
A setting for setting a third focus position used when reproducing the third image data reproduced between the first image data and the second image data, based on the first focus position and the second focus position. Means,
The receiving unit may include a position of the third image data between the first image data and the second image data, and a third focus position between the first focus position and the second focus position. Receiving an instruction to specify the relationship of
The image reproducing apparatus according to claim 1, wherein the setting unit determines the third focus position based on the relationship indicated by the instruction received by the receiving unit. Storage means for storing a plurality of light field image data that can be changed after shooting a focus position that is a focus position in the depth direction,
When the plurality of light field image data is reproduced as moving image data including a plurality of image data, a first focus position set in the first image data and a second focus position set in the second image data are received. Reception means,
A setting for setting a third focus position used when reproducing the third image data reproduced between the first image data and the second image data, based on the first focus position and the second focus position. Means,
The receiving unit receives an instruction to specify an object included in the first image data,
The image reproducing apparatus according to claim 1, wherein, when the receiving unit receives an instruction to specify the object, the setting unit determines the third focus position at a position where the object is in focus. Storage means for storing a plurality of light field image data that can be changed after shooting a focus position that is a focus position in the depth direction,
When the plurality of light field image data is reproduced as moving image data including a plurality of image data, a first focus position set in the first image data and a second focus position set in the second image data are received. Reception means,
A setting for setting a third focus position used when reproducing the third image data reproduced between the first image data and the second image data, based on the first focus position and the second focus position. Means,
The image reproducing device according to claim 1, wherein the setting unit determines a focus position of the plurality of image data after the second image data based on the first focus position and the second focus position. The setting means sets a focus position of one or more of the plurality of pieces of image data before the scene change occurs among the plurality of pieces of image data after the second image data, by the first focus position and the second focus position. Characterized in that it is determined based on
The image reproducing device according to claim 4. Executed by computer,
A first focus set in the first image data when a plurality of light field image data in which a focus position, which is an in-focus position in the depth direction, can be changed after shooting is reproduced as moving image data including a plurality of image data. A receiving step of receiving a position and a second focus position set in the second image data;
A setting for setting a third focus position used when reproducing the third image data reproduced between the first image data and the second image data, based on the first focus position and the second focus position. Steps and
A reproducing step of reproducing the moving image data including the first image data, the second image data, and the third image data,
In the receiving step, an instruction to select the third image data from the plurality of image data is further received,
In the reproducing step, the third image data corresponding to the instruction received in the receiving step , wherein the focus position is set to the third focus position, is displayed on a display unit. Executed by computer,
A first focus set in the first image data when a plurality of light field image data in which a focus position, which is an in-focus position in the depth direction, can be changed after shooting is reproduced as moving image data including a plurality of image data. A receiving step of receiving a position and a second focus position set in the second image data;
A setting for setting a third focus position used when reproducing the third image data reproduced between the first image data and the second image data, based on the first focus position and the second focus position. And a step,
In the receiving step, the position of the third image data between the first image data and the second image data, the third focus position between the first focus position and the second focus position, Receiving an instruction to specify the relationship of
In the setting step, the third focus position is determined based on the relationship indicated by the instruction received in the receiving step. Executed by computer,
A first focus set in the first image data when a plurality of light field image data in which a focus position, which is an in-focus position in the depth direction, can be changed after shooting is reproduced as moving image data including a plurality of image data. A receiving step of receiving a position and a second focus position set in the second image data;
A setting for setting a third focus position used when reproducing the third image data reproduced between the first image data and the second image data, based on the first focus position and the second focus position. And a step,
In the receiving step, an instruction for specifying an object included in the first image data is received,
In the setting step, when an instruction to specify the object is received in the receiving step, the third focus position is determined at a position where the object is in focus. Executed by computer,
A first focus set in the first image data when a plurality of light field image data in which a focus position, which is an in-focus position in the depth direction, can be changed after shooting is reproduced as moving image data including a plurality of image data. A receiving step of receiving a position and a second focus position set in the second image data;
A setting for setting a third focus position used when reproducing the third image data reproduced between the first image data and the second image data, based on the first focus position and the second focus position. And a step,
In the setting step, a focus position of the plurality of image data after the second image data is determined based on the first focus position and the second focus position.   A program for causing a computer to execute each step of the image reproducing method according to claim 6.