JP2018029340A - Image processing apparatus and method for controlling the same, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video processing apparatus that can play back a dynamic image while changing a focus state thereof, and to control the focus state according to the playback speed to enable a user to easily recognize a subject image.SOLUTION: A video playback device 100 reconstructs to play back image data in a designated focus state. An image creation part 104 creates a first image with a shallow depth of field in the designated focus state, and a second image with a deep depth of field related to a pan-focus video focused on a plurality of subjects. When the playback speed is equal to or faster than a threshold, control means performs control to create and play back the second image or the second image and first image. When the playback speed is less than the threshold, the playback control part 103 and a display focus information determination part 108 perform control to create and play back the first image.SELECTED DRAWING: Figure 8

Description

  The present invention relates to a video processing technique capable of changing and reproducing a moving image focus state.

  In recent years, an imaging device called a light field camera has been put into practical use. In this imaging apparatus, incident light is divided by a microlens array disposed on an image sensor, whereby light information in a plurality of directions can be captured to obtain light ray information. An image having an arbitrary focal length and depth of field (hereinafter referred to as focus information) can be constructed by executing predetermined calculation processing based on the light intensity and the light incident direction after shooting. An advantage of a light field camera (hereinafter referred to as LF camera) is that an arbitrary image can be constructed by calculation processing after photographing using light ray information. The same advantage can be obtained when moving images are captured by the LF camera, and image data can be reconstructed during reproduction after moving image shooting, and moving image reproduction can be performed while reflecting arbitrary focus information.

  Conventionally, a technique for displaying another image during fast-forward playback of a moving image is known. In the technique described in Patent Document 1, a 2D (two-dimensional) image is displayed to improve visibility when fast-forwarding a 3D (three-dimensional) moving image. Patent Document 2 discloses that the depth of field of an image can be adjusted or the degree of blur can be adjusted after shooting.

JP 2011-97451 A JP 2009-159357 A

  However, when a user searches for a desired scene by performing fast-forward playback or the like on a moving image shot with an LF camera, the scene may be missed depending on the focus information setting. This is because it is difficult to recognize a subject other than the in-focus area when the user views a moving image while focusing on a specific depth in the video. Therefore, the user needs to pay close attention not to miss a desired scene even in an out-of-focus area.

  In Patent Document 1, a 2D image is displayed in order to improve visibility during fast-forward playback. However, consideration is not given to recognition of a subject other than the in-focus area in a video in which focus information can be arbitrarily designated. In the method of switching from 3D display to 2D display, the focus information itself does not change, so the problem of recognizing a subject other than the in-focus area cannot be solved.

  An object of the present invention is to make it easier for a user to recognize a subject image by controlling the focus state in accordance with the playback speed in a video processing apparatus capable of changing and playing back a moving image. To do.

  In order to solve the above-described problems, an apparatus according to the present invention is a video processing apparatus that reproduces moving image data, and includes a control unit that performs reproduction of the moving image data, and a reproduction speed of the moving image data. And generating means for generating images having different depths of field from the moving image data.

  According to the present invention, the subject image can be easily recognized by controlling the focus state according to the reproduction speed.

It is a schematic diagram which shows the structural example (A) and (B) inside a light field camera. 3 is a schematic diagram showing a positional relationship between a microlens array 12 and each pixel of an image sensor 13. FIG. FIG. 5 is a schematic diagram for explaining a relationship between a traveling direction of incident light to a microlens and a recording area of the image sensor 13. It is a schematic diagram for demonstrating the information of the light ray which injects into the image sensor. It is a schematic diagram for demonstrating a refocus calculation process. FIG. 6 is a schematic diagram illustrating a relationship between a difference in incident angle to a microlens and a recording area of the image sensor 13. It is a schematic diagram for demonstrating the adjustment process of a depth of field. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram illustrating a configuration example of a video playback device in order to describe a first embodiment of the present invention. It is a flowchart explaining the operation | movement of the display focus information determination part of FIG. It is a figure which shows the example of a screen at the time of switching (A) at the time of normal reproduction | regeneration, the example of a screen at the time of switching from normal reproduction to 2 times speed reproduction (B), and the example of a screen when switching from 2 times speed reproduction to normal reproduction. It is a block diagram which shows the structural example of the apparatus which concerns on 2nd Embodiment of this invention. It is a figure which shows the example of a screen at the time of switching reproduction speed from normal reproduction to double speed reproduction in 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the apparatus which concerns on 3rd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Before describing a video processing apparatus according to an embodiment of the present invention, an imaging apparatus used for capturing a moving image that can specify a focus state will be described. 1A and 1B illustrate a schematic configuration of an LF camera. Light from the subject that has passed through the imaging lens 11 and entered the microlens array 12 is photoelectrically converted by the image sensor 13 to obtain an electrical signal. The imaging data obtained here is called light field data (hereinafter referred to as LF data).

  The imaging lens 11 projects light from the subject onto the microlens array 12. The imaging lens 11 is replaceable and is used by being attached to the main body of the imaging device 10. The user can change the imaging magnification by a zoom operation of the imaging lens 11. The microlens array 12 is configured by arranging microlenses (microlenses) in a lattice pattern, and is positioned between the imaging lens 11 and the image sensor 13. Each microlens constituting the microlens array 12 divides the incident light from the imaging lens 11 and outputs the divided light to the image sensor 13. The image sensor 13 constituting the imaging unit is an imaging element having a plurality of pixels, and detects the intensity of light at each pixel. Light divided by each microlens is incident on each pixel of the image sensor 13 that receives light from the subject.

FIG. 2 is a schematic diagram showing the positional relationship between the microlens array 12 and each pixel of the image sensor 13.
Each microlens of the microlens array 12 is arranged so that a plurality of pixels in the image sensor 13 correspond to each other. Light divided by each microlens enters each pixel of the image sensor 13, and the intensity (light ray information) of light from different directions can be detected by each pixel. Further, the incident direction (direction information) of the light beam incident on each pixel of the image sensor 13 through the microlens is known according to the positional relationship between each microlens and each pixel of the image sensor 13. That is, information on the light traveling direction is detected together with the light intensity distribution. Images on focal planes having different distances from the lens apex surface of the microlens array 12 are obtained by combining the outputs of the pixels of the image sensor 13 at positions corresponding to the amount of eccentricity from the optical axis of each microlens. It is done. The light beam is expressed by a function parameterized by two parallel planes using parameters such as position, azimuth, and wavelength. That is, the incident direction of light to each pixel is determined by the arrangement of a plurality of pixels corresponding to each microlens.
As described above, the imaging apparatus 10 acquires light ray information and direction information, and generates image data specifying an arbitrary focus position and viewpoint by performing light ray rearrangement and calculation processing (hereinafter referred to as reconstruction). it can. This ray information and direction information are included in the LF data.

FIG. 3 is a schematic diagram for explaining the relationship between the traveling direction of incident light to the microlenses of the microlens array 12 and the recording area of the image sensor 13.
An image of the subject by the imaging lens 11 is formed on the microlens array 12, and incident light on the microlens array 12 is received by the image sensor 13 via the microlens array 12. At this time, as shown in FIG. 3, the light rays incident on the microlens array 12 are received at different positions on the image sensor 13 according to the traveling direction of the subject, and are similar to the shape of the imaging lens 11. An image is formed for each microlens.

FIG. 4 is a schematic diagram for explaining information of light rays incident on the image sensor 13.
The light rays received by the image sensor 13 will be described with reference to FIG. An orthogonal coordinate system (u, v) on the lens surface of the imaging lens 11 is used, an orthogonal coordinate system (x, y) on the imaging surface of the image sensor 13 is used, and the imaging of the lens surface of the imaging lens 11 and the image sensor 13 is performed. Let F be the distance to the surface. Then, the intensity of the light beam passing through the imaging lens 11 and the image sensor 13 can be expressed by a four-dimensional function L (u, v, x, y).
Since the light beam incident on each microlens is incident on a different pixel according to the traveling direction, the image sensor 13 stores the above-described four-dimensional function L (u , V, x, y) are recorded.

また、リフォーカス面で得られるイメージＥ’（ｓ，ｔ）は、上記強度Ｌ’をレンズ口径に関して積分したものとなるので、以下の（２）式のように表される。

従って、この（２）式からリフォーカス演算処理を行うことで、任意の焦点（リフォーカス面）に設定した画像を再構築することができる。 Next, refocus calculation processing after imaging will be described. FIG. 5 is a schematic diagram for explaining the refocus calculation process. As shown in FIG. 5, when the positional relationship between the imaging lens surface, the imaging surface, and the refocus plane is set, the light intensity L ′ (u, v, s) in the orthogonal coordinate system (s, t) on the refocus plane. , T) is expressed as the following equation (1).

Further, the image E ′ (s, t) obtained on the refocus plane is obtained by integrating the intensity L ′ with respect to the lens aperture, and is represented by the following equation (2).

Therefore, an image set at an arbitrary focal point (refocus plane) can be reconstructed by performing the refocus calculation process from the equation (2).

  Next, a process for adjusting the depth of field after imaging will be described. Prior to the refocusing calculation, weighting is performed by multiplying the image data forming the image area assigned to each microlens by a weighting factor. For example, when it is desired to generate an image having a deep depth of field, integration processing is performed using only information on light rays incident at a relatively small angle with respect to the light receiving surface of the image sensor 13. In other words, light rays having a relatively large incident angle to the image sensor 13 are not included in the integration process by multiplying by a weighting factor of 0 (zero).

  FIG. 6 is a schematic diagram for explaining the relationship between the difference in the incident angle to the microlens and the recording area of the image sensor 13, and FIG. 7 is a schematic diagram for explaining the depth-of-field adjustment process. is there. As shown in FIG. 6, the light beam having a relatively small incident angle to the image sensor 13 is located in a more central region. Therefore, as shown in FIG. 7, the integration process is performed using only the pixel data acquired at the center of the region (the hatched portion in the figure). By performing such processing, an image with a deep depth of field can be expressed as if an aperture stop provided in a general imaging device or the like is stopped. By further reducing the pixel data in the central portion to be used, it is possible to generate a pan-focus image having a deeper depth of field. As described above, the depth of field of the image can be adjusted after shooting based on the actually acquired LF data (light ray information).

[First Embodiment]
FIG. 8 is a block diagram illustrating a configuration example of the video reproduction device 100 according to the first embodiment of the present invention. The operation input unit 101 receives an input signal by a user operation. The operation input unit 101 outputs playback control information for specifying or changing the playback speed of the moving image to the playback control unit 103, and outputs specified focus information for specifying the focus position to the display focus information determination unit. The designated focus information includes information such as the focal length and the depth of field according to the designated focus state. For example, it may be information indicating a focus state in which focus control is automatically performed during autofocus shooting, or information indicating a focus state manually designated by the user during manual focus shooting. The storage unit 102 stores and holds the moving image LF data.

  The reproduction control unit 103 reads out the moving image LF data from the storage unit 102 and outputs it to the image generation unit 104. The reproduction control information from the operation input unit 101 includes reproduction speed information, and the reproduction control unit 103 controls the frame to be read and the reading speed according to the reproduction speed information. When the playback speed is designated or changed, the playback control unit 103 outputs playback state transition information to the composite screen generation unit 107 in order to display playback speed information on the screen. The playback control unit 103 also outputs playback speed information to the display focus information determination unit 108 in order to determine the focus information of the video to be displayed.

  The image generation unit 104 performs the refocus calculation process and the depth of field adjustment process on the moving image LF data input from the reproduction control unit 103 to reconstruct an image. At that time, the image is reconstructed according to the display focus information input from the display focus information determination unit 108, and the generated image is output to the image composition unit 105 as image (video) data in a desired focus state. .

The image synthesis unit 105 generates video data obtained by combining the image (video) input from the image generation unit 104 and the composite screen input from the synthesis screen generation unit 107, and outputs the video data to the video display unit 106. The video display unit 106 is, for example, a liquid crystal panel or a plasma display, and displays a video according to video data output from the image synthesis unit 105.
The composite screen generation unit 107 generates data of a playback speed display screen for displaying the playback speed information according to the playback state transition information from the playback control unit 103, and outputs the data to the image composition unit 105. Further, when an operation guide display instruction is input from the display focus information determination unit 108, the composite screen generation unit 107 generates data of an operation guide display screen for changing the focus information and outputs it to the image composition unit 105. Note that the composite screen generated by the composite screen generation unit 107 may be displayed on the video display unit 106 and controlled so as to be deleted after a predetermined time.

  The display focus information determination unit 108 determines the focus information (display focus information) of the video to be displayed based on the designated focus information from the operation input unit 101 and the playback speed information from the playback control unit 103, and displays the determination result as an image. The data is output to the generation unit 104. Further, the display focus information determination unit 108 determines whether or not there is an operation guide display instruction according to the change of the display focus information, and outputs the determination result to the composite screen generation unit 107. Here, the specification of the focus information by the user is performed when an instruction signal is input by the button operation in the vertical direction on the operation input unit 101 when the operation guide display screen 24 shown in FIG. 10C is displayed. Shall. At that time, the focus information is changed. The operation guide display screen 24 can be displayed at an arbitrary timing when the operation guide display screen is displayed on the operation input unit 101 even during playback at a normal playback speed. The detailed operation of the display focus information determination unit 108 will be described later.

  FIG. 9 is a flowchart illustrating an example of determination processing performed by the display focus information determination unit 108. The start condition of the display focus information determination process is that the designated focus information is input from the operation input unit 101 or the reproduction speed information is input from the reproduction control unit 103. The display focus information determination unit 108 starts processing when the above-described start condition is satisfied, compares the absolute value of the playback speed with a threshold value based on the playback speed information from the playback control unit 103, and determines the absolute value of the playback speed. It is determined whether or not the value is greater than or equal to a threshold value (S101). The threshold value indicates the minimum value of the reproduction speed at which the user feels that it is difficult to recognize the video when the user specifies focus information and performs a change operation. The threshold value is not a fixed value and may be arbitrarily set by the user.

  The display focus information determination unit 108 sets the focus information (designated focus information) designated by the user from the operation input unit 101 as the focus information (display focus information) of the video to be displayed when the absolute value of the playback speed is less than the threshold value. (S102). On the other hand, if the playback speed is equal to or higher than the threshold, the display focus information determination unit 108 sets pan focus information as display focus information (S103), and advances the process to S106. The pan focus focus information is focus information related to a pan focus video in which a plurality of subjects are focused. The pan-focus video is a video with a deep depth of field that is focused on a wide depth range including a subject in a predetermined focus area as well as a subject outside the area. By displaying the pan-focus video, the user can easily recognize the subject over a wide range, and thus it is possible to prevent a desired scene from being overlooked during the search.

After S102, the process proceeds to S104, and the display focus information determination unit 108 determines whether or not the current focus information is pan focus focus information. If the current focus information is pan focus focus information, the process proceeds to S105. If the current focus information is not pan focus focus information, the process proceeds to S106.
When the current focus information is pan focus focus information, a situation where the user changes from a playback speed that is equal to or higher than a threshold to a playback speed that is lower than the threshold is, for example, a case where a search for a desired scene is completed. In this case, when the focus information is switched from the pan focus information to the designated focus information, the display focus information determination unit 108 displays the operation information display screen for changing the focus information to the composite screen generation unit 107 so that the user can easily change the focus information. The display instruction of 24 is issued (S105). Next, the display focus information determination unit 108 outputs the display focus information to the image generation unit 104 (S106).

FIGS. 10A to 10C are diagrams illustrating screen examples when the playback speed is switched. An example is shown in which the threshold value shown in S101 of FIG. 9 is set to a value corresponding to 1.5 × speed playback, changed from normal playback (1 × speed playback) to 2 × speed, and then switched to 1 × speed. The designated focus information is information for focusing on the first subject 21 and is not focused on the second subject 22. The video reconstructed using the designated focus information is a video with a shallow depth of field in which a narrow depth range of the subject designated by the user is focused.
FIG. 10A shows an example of a screen during normal playback. In this case, since the playback speed is less than the threshold value (1.5 times speed), the display focus information determination unit 108 determines that the display focus information is designated focus information. Therefore, the focus information designated by the user operation is reflected on the display video.

  FIG. 10B shows a screen example when the normal playback is changed to the double speed playback. In this case, since the playback speed is equal to or higher than the threshold (1.5 times speed), the display focus information determination unit 108 determines the display focus information as the pan focus focus information. Therefore, pan focus information that enables focusing on a plurality of subjects is reflected in the display image. Since the pan-focus video is reproduced at double speed, the user can recognize the subject 22 in a state where the subject 22 that is not focused during normal reproduction is also in focus. Also, based on the playback state transition information from the playback control unit 103, the composite screen generation unit 107 generates data of the playback speed display screen 23, and this screen is combined with the display video.

  FIG. 10C shows an example of a screen when switching from double speed playback to normal playback. In this case, since the playback speed is less than the threshold value (1.5 times speed), the display focus information determination unit 108 determines the display focus information as designated focus information. In addition, since the current focus information is pan focus focus information, the composite screen generation unit 107 generates data of the operation guide display screen 24 according to an operation guide display instruction from the display focus information determination unit 108. Also, based on the playback state transition information from the playback control unit 103, the composite screen generation unit 107 generates data of the playback speed display screen 23, and this screen (normal playback speed display screen) is combined with the display video. The

  According to the first embodiment, when moving image data that can be arbitrarily designated as a focus state is reproduced, if the reproduction speed is less than a threshold value, the first image with a shallow depth of field that is generated according to the designated focus state. Is controlled to reproduce. When the playback speed is equal to or higher than the threshold value, control is performed to generate and play back a second image having a deep depth of field related to a pan-focus video in which a plurality of subjects are in focus. As a result, the user can easily recognize each subject and prevent a desired scene from being overlooked. For example, fast-forward playback is used for a user to search for a desired scene from a large number of scenes, and is different from playback for viewing images. Therefore, even if pan-focus video is displayed, the purpose of scene search is Is achieved. On the other hand, at the time of reproduction at normal speed, when the reproduction speed is close to the normal speed, or when the number of frames to be skipped is small, the user can check an image generated according to previously specified focus information.

  In the present embodiment, as an apparatus for acquiring moving image LF data, an LF camera that detects light in different directions by detecting light divided by the microlens array 12 with the image sensor 13 is exemplified. For example, an embodiment in which light having different directions is detected by a multi-view camera may be used. As for the playback speed, the fast-forward playback has been mainly described. With respect to rewind playback, display focus information may be determined and changed according to the speed. In addition, in the case of playback in which a scene or chapter is jumped, the focus information is determined and changed by determining the number of frames or the time interval. In this case, when the number of frames of the video signal to be skipped at the time of scene search is less than the threshold, control is performed to generate and reproduce an image according to the designated focus information. When the number of frames of the video signal to be skipped is equal to or greater than the threshold value, control is performed to generate and reproduce an image according to the pan focus focus information. These matters are the same in the embodiments described later.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the second embodiment, during fast-forward playback of a moving image for which focus information can be specified, the first image having a shallow depth of field based on the specified focus information, and a subject related to a pan-focus video in which a plurality of subjects are in focus Processing for simultaneously displaying the second image having a deep depth of field will be described. In addition, about the component similar to the case of 1st Embodiment, by using the code | symbol already used, those detailed description is abbreviate | omitted and it demonstrates centering on difference. The method of omitting such description is the same in the embodiments described later.

FIG. 11 is a block diagram illustrating a configuration example of the video reproduction device 100.
The image generation unit 104 reconstructs an image from the moving image LF data input from the reproduction control unit 103. At that time, an image (video) is generated by reconstruction according to the display focus information input from the display focus information determination unit 108. When the display focus information is pan focus focus information, data of both the designated focus video and the pan focus video currently being generated are generated and output to the image composition unit 105. If the display focus information is other than the pan focus information, only an image (video) according to the display focus information is generated and output to the image composition unit 105.

  The image composition unit 105 generates video data obtained by combining the display focus video and pan focus video input from the image generation unit 104 and the composite screen input from the composite screen generation unit 107, and outputs the video data to the video display unit 106. To do. When two image data are input from the image generation unit 104, the image composition unit 105 performs screen control so that two images are displayed on two screens (divided display, PinP display, etc.).

  FIG. 12 shows an example of a screen when the playback speed is changed. A case where the threshold is set to a value corresponding to 1.5 × speed reproduction and the normal reproduction state is switched to 2 × speed will be exemplified. The designated focus information is information for focusing on the first subject 21 and is not focused on the second subject 22. In addition, the screen during normal playback and the screen example when switching from double speed playback to normal playback are the same as those in FIGS. 10A and 10C, and thus the description thereof is omitted.

FIG. 12 shows a screen example when switching from normal playback to double speed playback. In this case, since the playback speed is equal to or higher than the threshold (1.5 times speed), the display focus information determination unit 108 determines the display focus information as the pan focus focus information. Since the display focus information is the pan focus focus information, the image generation unit 104 simultaneously generates each data of the video 20A and the pan focus video 20B based on the designated focus information currently being displayed and outputs the data to the image composition unit 105. To do. The image synthesizing unit 105 synthesizes the video in order to display the input video 20A and 20B data on two screens. At this time, for the subject 22 that is not in focus during normal playback, the user can view the pan-focus video 20B at the same time, so that the subject 22 can be recognized. Further, based on the playback state transition information from the playback control unit 103, the composite screen generation unit 107 generates data of the playback speed display screen 23, and this screen is combined with the display video.
According to the second embodiment, a user can easily recognize a subject by simultaneously viewing a pan-focus video with a deep depth of field and a video with a shallow depth of field generated using specified focus information. Therefore, it is possible to prevent a desired scene from being overlooked.

[Third Embodiment]
Next, a third embodiment of the present invention will be described. In the third embodiment, a process for discriminating playback speed information and display focus information from video information will be described.
FIG. 13 is a block diagram illustrating a configuration example of the video display device 200. The video signal analysis unit 201 analyzes the specified focus information from the additional information of the input LF video signal, and analyzes the playback speed information from the frame information of the LF video signal. The moving image LF data related to the LF video signal is output to the image generation unit 104, and the designated focus information and the reproduction speed information are output to the display focus information determination unit 108. When the reproduction speed is changed, reproduction state transition information is output to the composite screen generation unit 107. Here, the LF video signal indicates a video signal to which various types of information such as designated focus information and frame information are added together with the moving image LF data. The format of the LF video signal is not limited to the above, and various types of information may be added by other means. The video signal analysis unit 201 performs analysis according to the format of the LF video signal, and acquires designated focus information and playback speed information. As a result, even when focus information and playback speed are specified in addition to user operations, the focus information is determined and changed in the same manner as in the above-described embodiment, so that the user can easily recognize the subject and the desired scene. Can be overlooked.

  According to the third embodiment, even when playback speed information or focus information is designated by an external device, the designated focus information and playback speed information are acquired by analyzing information related to the LF video signal. be able to.

[Other Embodiments]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

DESCRIPTION OF SYMBOLS 100 Video reproduction apparatus 103 Reproduction | regeneration control part 104 Image generation part 105 Image composition part 108 Display focus information determination part 201 Video signal analysis part

The present invention, in the images processing apparatus capable of playing by changing the focusing state of the moving image, by controlling the focusing state according to the playback speed, object the user to easily recognize the object image And

In order to solve the above problems, the device according to the present invention, when performing a control means for controlling to reproduce the moving image data, the control of said control means to reproduce the first reproduction speed corresponding to the moving image data Output the first depth of field image to the display unit, and when the control means performs control to reproduce at a second reproduction speed higher than the first reproduction speed, the first corresponding to the moving image data Output means for outputting an image having a second depth of field different from the depth of field to the display unit .

Claims (23)

A video processing device for reproducing moving image data,
Control means for performing control to reproduce the moving image data;
Generating means for generating images with different depths of field from the moving image data according to the reproduction speed of the moving image data;
A video processing apparatus comprising:   The generating means generates a first image having a shallow depth of field from the moving image data when the reproduction speed is less than a threshold, and from the moving image data when the reproduction speed is equal to or higher than the threshold. The video processing apparatus according to claim 1, wherein the second image having a depth of field deeper than the first image is generated.   When the reproduction speed is equal to or higher than the threshold value, the control means generates the first image and the second image by the generation means, and reproduces the generated first image and the second image. The video processing apparatus according to claim 2, wherein:   The generation unit acquires light ray information indicating light intensity and light beam direction information related to the captured moving image data, and uses the acquired light ray information and light beam direction information to specify a specified focus state. The video processing apparatus according to claim 2, wherein the first image is generated. Operating means for designating a focus state of the moving image data;
The generating unit acquires information on a focal length and a depth of field as focus information regarding a focus state designated by the operation unit, and uses the acquired information on the focal length and the depth of field to obtain the first information. The video processing apparatus according to claim 2, wherein an image is generated.   5. The video processing according to claim 1, wherein the control unit analyzes additional information of the moving image data and outputs focus information obtained by the analysis to the generation unit. 6. apparatus. Operating means for designating the playback speed;
The video processing apparatus according to claim 2, wherein the control unit compares the playback speed designated by the operation unit with the threshold value.   The video processing apparatus according to claim 2, wherein the control unit analyzes a playback speed from frame information of the moving image data, and compares the playback speed obtained by the analysis with the threshold value.   The control unit performs control to generate the second image by the generation unit and reproduce the generated second image in fast forward playback, rewind playback, or scene search operation. The video processing apparatus according to claim 2 or 3.   The control means performs control to generate the first image by the generation means and reproduce the generated first image when the number of frames of the moving image data to be skipped at the time of scene search is less than a predetermined number. 4. The control according to claim 2, wherein when the number of frames is equal to or greater than the predetermined number, the second image is generated by the generation unit, and the generated second image is reproduced. The video processing apparatus described.   The video processing apparatus according to claim 1, wherein the moving image data is light field data. A control method executed by a video processing device for reproducing moving image data,
A control step for performing control to reproduce the moving image data;
A step of generating an image having a different depth of field from the moving image data according to a reproduction speed of the moving image data;
A control method for a video processing apparatus, comprising:   When the reproduction speed is less than a threshold value, the generation unit generates a first image with a shallow depth of field from the moving image data. When the reproduction speed is equal to or higher than the threshold value, the generation unit generates the moving image. The method of controlling a video processing apparatus according to claim 12, further comprising: generating a second image having a depth of field deeper than the first image from data.   In the control step, when the reproduction speed is equal to or higher than the threshold value, the generation unit generates the first image and the second image, and reproduces the generated first image and the second image. The method according to claim 13, wherein control is performed.   Light information indicating light intensity and light beam direction information related to the captured moving image data is acquired, and the first image in a designated focus state is obtained using the acquired light beam information and light beam direction information. The video processing apparatus control method according to claim 13, further comprising a step of generating by the generating means.   Information on focal length and depth of field is acquired as focus information on the focus state specified by the operating means, and the first image is generated by the generating means using the acquired information on focal length and depth of field. The method of controlling a video processing apparatus according to claim 13, further comprising a generating step. Analyzing the additional information of the moving image data by a control means;
Generating an image by the generating means according to the focus information obtained by the analysis;
16. The video processing apparatus control method according to claim 12, further comprising:   15. The video processing apparatus control method according to claim 13, further comprising a step of comparing a reproduction speed designated by an operation unit with the threshold value. Analyzing the playback speed from the frame information of the moving image data by a control means;
Comparing the playback speed obtained by the analysis with the threshold;
15. The method of controlling a video processing apparatus according to claim 13 or 14, characterized by comprising:   15. The step of generating the second image by the generating means and reproducing the generated second image in the fast forward playback, rewind playback, or scene search operation. A method for controlling the video processing apparatus according to claim 1.   When the number of frames of the moving image data to be skipped at the time of scene search is less than a predetermined number, a step of generating the first image by the generation means and controlling the reproduction of the generated first image; 14. When the number is equal to or greater than the predetermined number, the step of generating the second image by the generation unit and performing control to reproduce the generated second image is included. 14. A method for controlling the video processing apparatus according to 14.   The video processing apparatus control method according to any one of claims 12 to 21, wherein the moving image data is light field data. The program for making a computer perform each step of the control method of a video processing apparatus of any one of Claim 12 to 22.

Images