JP2017063276A - Video display device, video display method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reproduce a video, recorded in a wide angle, in a manner to be easily viewed by an observer.SOLUTION: A moving image file storage unit 33 stores a video in a wider range than in the display range of a display unit 35. Based on a predetermined reproduction video condition, a reproduction processing unit 34 sets a position of the wide range video stored in the moving image file storage unit 33 of the video to be displayed in the display range of the display unit 35. At the start of reproduction, the reproduction processing unit 34 controls to trim the video in the set position from the wide range video to display on the display unit 35. The control unit 37 detects an operation from the outside during the display. When an operation from the outside is detected, the reproduction processing unit 34 controls to display the video of which display is controlled, on the basis of the operation from the outside.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a video display device, a video display method, and a program.

  2. Description of the Related Art In recent years, photographing apparatuses that record video with a super-wide angle or more half-sky or full-field angle have appeared (for example, see Patent Document 1).

  Although such a photographic device can capture the entire shootable range at the time of recording, the video playback device can display images recorded in a super-wide angle or more half-round or full-sky. In consideration of the fact that it is not possible to cope with this, some areas are trimmed and recorded.

  Also in the above-described Patent Document 1, a technique for trimming a range including a determination object and a region ahead of the user's line-of-sight direction is proposed.

Japanese Patent Laying-Open No. 2015-126451

  However, the above-mentioned conventional technique has a problem that there is a video that is not recorded because it is trimmed during recording, even though it is captured at a super wide angle or a half-sky or full-sky circumference. there were.

  Accordingly, an object of the present invention is to reproduce a video recorded at a wide angle in a manner that is easier for a user to view.

  The video display device according to the present invention stores display means, storage means for storing video in a predetermined range, and video to be displayed in the display range of the display means based on predetermined playback video conditions in the storage means. A setting means for setting a position in the recorded video, and a control for trimming the video at the position set by the setting means from the video stored in the storage means and displaying the video on the display means at the start of reproduction. 1 display control means, detection means for detecting an external operation during display by the first display control means, and when the external operation is detected by the detection means, the first display control means And a second display control unit configured to control the display unit to display the video whose display is controlled by the unit based on an operation from the outside.

  A video display method according to the present invention is a video display method for trimming a video for display in a display range of a display means from a predetermined video and displaying the trimmed video on the display means, based on a predetermined playback video condition, A step of setting a position of the video to be displayed in the display range of the display means in the predetermined video; and at the start of reproduction, the video at the set position is trimmed from the predetermined video and displayed on the display A step of controlling the display unit, a step of detecting an external operation during video display on the display means, and an operation from the outside when the external operation is detected, the display-controlled video is transmitted from the external unit. And controlling to display on the display means based on an operation.

  According to the present invention, there is provided a program for trimming a video for display in a display range of a display unit from a predetermined video and displaying the computer on the display unit based on a predetermined reproduction video condition. Setting means for setting the position of the video to be displayed in the display range of the unit in the predetermined video; at the start of playback, the video at the position set by the setting unit is trimmed from the predetermined video and is displayed on the display unit First display control means for controlling to display, detection means for detecting an external operation during display by the first display control means, and when the external operation is detected by the detection means, the first display control means A second display control means for controlling the display unit to display the video controlled by the first display control means on the basis of the external operation; It characterized thereby to function.

  According to the present invention, a video recorded at a wide angle can be reproduced more easily for the user.

It is a block diagram which shows the structure of the video recording / reproducing system 1 by embodiment of this invention. It is a block diagram which shows the structure of the head mounting apparatus 10 by this 1st Embodiment. It is a block diagram which shows the structure of camera apparatus 20a, 20b by this 1st Embodiment. 2 is a block diagram showing a configuration of a playback device 30 according to the first embodiment. FIG. It is a conceptual diagram for demonstrating the adjustment process of the head mounting apparatus 10 and camera apparatus 20a, 20b in the video recording / reproducing system 1 by this 1st Embodiment. 4 is a flowchart for explaining the operation of the playback apparatus 30 according to the first embodiment. 6 is a flowchart for explaining an example of a frame analysis process for determining a video area (trimming position) to be displayed at the time of reproduction for each predetermined reproduction video condition according to the first embodiment. It is a block diagram which shows the structure of the reproducing | regenerating apparatus 30 by this 2nd Embodiment. 12 is a flowchart for explaining a partial operation of the playback apparatus 30 according to the second embodiment. In this 2nd Embodiment, it is a conceptual diagram for demonstrating the relationship between the whole sky image and trimming position at the time of determining the video area (trimming position) which should be displayed at the time of reproduction | regeneration based on the position of a to-be-photographed object's face. . 12 is a flowchart for explaining a partial operation of the playback apparatus 30 according to the third embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

A. First Embodiment FIG. 1 is a block diagram showing a configuration of a video recording / playback system 1 according to a first embodiment of the present invention. In the figure, the video recording / playback system 1 includes a head-mounted device 10, camera devices 20 a and 20 b, and a playback device 30. The head-mounted device 10 is mounted directly on the head or on the head via a headband, a hat, a glasses frame, etc., and the movement of the photographer during the shooting of all-sky images (including wide-angle images) (Moving direction), movement of the photographer's head (face direction = line-of-sight direction), sound source direction of the environmental sound, and the like are detected and transmitted to the playback device 30 as additional information.

  Each of the camera devices 20a and 20b can capture an image with a wide angle of view (for example, 180 ° or more). The camera devices 20a and 20b are each mounted on the photographer's body, clothes, back, and the like, and the camera device 20a is mounted on the photographer's front face so as to capture a front half sky video of the photographer. Is mounted on the back of the photographer so as to capture the back half-round video of the photographer. The camera devices 20 a and 20 b each transfer the front half-sky video and the back half-sky video to the playback device 30 after imaging.

  The playback device 30 is composed of, for example, a smartphone or a dedicated terminal. The playback device 30 receives the front half-sky video from the camera device 20a, receives the back half-sky video from the camera device 20b, and generates one synchronized full-sky video based on each timeline. Further, when generating the all-sky video, the playback device 30 records the additional information received from the head-mounted device 10 as metadata according to the time code of the frame of the all-sky video being captured. , Filed into a moving image file of a predetermined format. Further, the playback device 30 performs normal image processing (playback device) by performing image processing (trimming processing, distortion correction processing, etc.) on the generated all-round video (moving image file) based on predetermined playback video conditions described later. The video can be reproduced and displayed as 30).

  FIG. 2 is a block diagram showing the configuration of the head-mounted device 10 according to the first embodiment. The head-mounted device 10 includes sensors 11, an omnidirectional microphone & directional microphone 12, an audio signal processing unit 13, a signal processing unit 14, a wireless communication unit 15, and a control unit 16.

  The sensors 11 include a GPS (Global Positioning System) unit 11-1, an azimuth / atmospheric pressure sensor 11-2, and a motion sensor 11-3. The GPS unit 11-1 calculates current position information (latitude, longitude, altitude, etc.) using an artificial satellite. The azimuth / atmospheric pressure sensor 11-2 detects which direction (direction) of the east, west, north, and south and the atmospheric pressure. The motion sensor 11-3 detects the direction (tilt) of the photographer's (head) with respect to gravity.

  The omnidirectional microphone & directional microphone 12 collects ambient environmental sounds from the start of recording. The audio signal processing unit 13 performs processing (level adjustment, noise removal, etc.) on the audio signal collected by the omnidirectional microphone & directional microphone 12. The signal processing unit 14 performs signal processing on position information detected by the sensors 11, azimuth / atmospheric pressure information, motion information, and an audio signal collected by the omnidirectional microphone & directional microphone 12 to perform predetermined data Convert to format additional information. The additional information is accompanied by a time code from the start of shooting of the half sky video by each of the camera devices 20a and 20b. In other words, the additional information can be synchronized with the half-sky video data captured by each of the camera devices 20a and 20b and the time code.

  The wireless communication unit 15 transmits the additional information signal-processed by the signal processing unit 14 to the playback device 30 using a predetermined wireless method (wireless LAN, Bluetooth (registered trademark), etc.). In the first embodiment, the video data is transmitted to the playback device 30. However, since it depends on the software whether the final moving image file is to be performed by the camera devices 20a and 20b or the playback device 30, the playback device 30 is used. It is not limited to. The control unit 16 controls the operation of each unit described above in an integrated manner.

  FIG. 3 is a block diagram showing the configuration of the camera devices 20a and 20b according to the first embodiment. The camera devices 20a and 20b have the same specifications. The camera devices 20a and 20b include optical systems (wide-angle lenses) 21a and 21b, imaging units 22a and 22b, image processing units 23a and 23b, compression encoding (file creation) processing units 24a and 24b, storage units 25a and 25b, and wireless units. Communication units 26a and 26b, operation units 27a and 27b, and control units 28a and 28b are provided.

  The optical systems (wide-angle lenses) 21a and 21b are lenses that can shoot at an ultra-wide angle (180 ° or more) with respect to the optical axis. The optical system 21a captures a photographer's front half sky image, and the optical system 21b photographs a photographer's back half sky image. The imaging units 22a and 22b are each composed of an imaging element or the like, and convert the front half-sky image and the back half-sky image formed through the optical systems (wide-angle lenses) 21a and 21b into electrical signals.

  Image processing units 23a and 23b respectively perform image processing (pixel interpolation processing, γ correction, generation of luminance color difference signals, white balance processing, exposure correction processing, filter processing, etc.) on the front half sky image data and the back half sky image data. I do. The compression encoding (file conversion) processing units 24a and 24b respectively perform compression / expansion (for example, compression / expansion in JPEG format, M-JPEG format, or MPEG format) of the front half-circumference image data and the back half-circumference image data. Process.

  The storage units 25a and 25b are respectively used for a buffer memory for temporarily storing the front semi-circumferential image data and the back semi-circular image data captured by the image capturing units 22a and 22b, a program necessary for controlling each unit, and a control for each unit. Used as a working memory for storing necessary data. Each of the wireless communication units 26a and 26b converts the front half-sky image data and the back half-sky image data, which are compressed files, to the playback device 30 using a predetermined wireless method (wireless LAN, Bluetooth (registered trademark), etc.). Forward. The operation units 27a and 27b each include a power switch, an imaging start switch (shutter switch), and the like. The control units 28a and 28b each integrally control the operation of each unit described above.

  FIG. 4 is a block diagram showing the configuration of the playback device 30 according to the first embodiment. The playback device 30 includes a wireless communication unit 31, an image processing unit (moving image file generating unit) 32, a moving image file storage unit 33, a playback processing unit 34, a display unit 35, an operation unit 36, and a control unit 37.

  The wireless communication unit 31 receives additional information from the head-mounted device 10 and also receives compressed front half-sky image data and back half-sky image data from each of the camera devices 20a and 20b. The image processing unit (moving image file generation) 32 performs a front half-circumference image data and a back half-circumference image data at a peripheral part (end part) of one image data and a peripheral part (end part) of the other image data. A matching image area is detected, and both pieces of image data are synthesized for each frame so as to overlap each other, thereby generating all-sky video data (moving image file). At this time, the image processing unit (moving image file generating unit) 32 embeds additional information in the generated all-sky video data (moving image file) as metadata based on the time code. The moving image file storage unit 33 stores the moving image file generated by the image processing unit (moving image file generation) 32.

  When the all-sky video data generated in this way is reproduced and displayed, it is a projection of the entire spherical space centered on the photographer, so it is displayed as it is on a display device having a planar display size. That is generally impossible. Therefore, in accordance with the size of the display unit 35 of the playback device 30 (hereinafter also referred to as a display range), a part of the whole sky video reproduced and displayed based on this whole sky video data is trimmed within a predetermined trimming range. Then, after performing distortion correction for correcting distortion due to an ultra-wide angle, the trimmed video is displayed on the display unit 35. The problem is which part of the whole sky image is trimmed and displayed at the start of playback. Usually, it is conceivable to trim an image within a predetermined trimming range from the direction the photographer is viewing at the time of photographing (generally in front of the photographer) to display an image to be displayed during reproduction.

  However, since the 360-degree video is recorded in the all-sky image, centering on the photographer, it is not seen at the time of shooting. Or something that produces a loud sound may have been recorded. Therefore, in the first embodiment, when the reproduction of the all-sky video is started, the video area (trimming position) to be reproduced and displayed in the frame of the whole-sky video data is determined based on a predetermined playback video condition. To do.

As predetermined playback video conditions, the following five conditions are assumed.
(1) Direction (direction) in which the photographer is facing
(2) Direction (direction) in which the photographer is moving
(3) Voice direction (direction)
(4) Direction (direction) in which the face of the subject facing the camera devices 20a and 20b, that is, the front face is detected
(5) Direction (direction) in which moving object (motion) is detected

  As for (1) to (3), the video to be reproduced and displayed in the frame of the all-sky video data based on any of the line-of-sight direction, the moving direction, and the sound source direction detected by the head-mounted device 10 at the time of shooting. Determine the area (trimming position). For (4) and (5), it is only necessary to perform image processing (analysis processing) on the finally obtained all-sky video data, and it is not necessary to use additional information (metadata) from the head-mounted device 10. . Therefore, (1) to (3) will be described in the first embodiment, and (4) and (5) will be described in second and third embodiments described later, respectively.

  The reproduction processing unit 34 reads the all-sky video data (moving image file) stored in the moving image file storage unit 33 and adds additional information (metadata) embedded in the first frame of the all-sky video data (moving image file). ) To obtain the direction of the photographer's line of sight (face direction), the direction of the photographer's movement, or the direction of the sound source, and the direction of the photographer's line of sight (face direction), the direction of the photographer's movement, or the direction of the sound source Based on the above, the trimming position to be displayed at the time of reproduction in the all-sky video data is determined. The reproduction processing unit 34 trims the entire sky video data in a predetermined trimming range for each frame according to the determined trimming position, and reproduces and displays the trimmed video on the display unit 35.

  The display unit 35 includes a color liquid crystal display and its driving circuit, and displays an image trimmed from the entire sky video data. The display unit 35 may include a touch panel, and may execute various commands by a touch operation on the touch panel. Specifically, in the first embodiment, the user operations include a pause operation, a movement instruction operation, a reproduction start operation, and the like. These user operations are, for example, a tap on a screen (predetermined position, predetermined icon). , Double tap, swipe or finger gesture. However, an operation using a key switch of the operation unit 36 may be used. The operation unit 36 includes operation keys such as a power switch and a volume adjustment button, and outputs an operation signal corresponding to the user's key operation to the control unit 37. The control unit 37 controls the operation of each unit described above in an integrated manner.

  FIG. 5 is a conceptual diagram for explaining an adjustment process between the head-mounted device 10 and the camera devices 20a and 20b in the video recording and reproducing system 1 according to the first embodiment. In FIG. 5, the front half sky image 40 is a shooting angle of view by the camera device 20 a, and the back half sky image 50 is a shooting angle of view by the camera device 20 b. As described above, in the first embodiment, the entire sky image is used to determine the video area (trimming position) to be reproduced in association with the face direction (line-of-sight direction), movement direction, and sound source direction of the photographer. The orientation of the photographer's face detected by the head-mounted device 10 mounted on the photographer's head at the start of data capture (that is, the “front center” of the head-mounted device 10), and the camera device 20a , 20b, that is, it is necessary to grasp the deviation from the center of the shooting angle of view. Therefore, as shown in FIG. 5, at the start of shooting, a deviation between the center 41 in front of the head-mounted device 10 and the center 42 of the shooting angle of view of the camera devices 20a and 20b is detected, and the all-sky image Record it in a video file of the data. At the time of reproduction, in order to determine the position (trimming position) of the video area to be reproduced, the position (trimming position) of the video area to be reproduced is corrected based on the detected deviation.

Next, the operation of the above-described first embodiment will be described.
FIG. 6 is a flowchart for explaining the operation of the playback device 30 according to the first embodiment. In the playback device 30, the playback processing unit 34 performs an analysis process on the first frame of the all-sky video data (moving image file) (step S10). In this analysis process, the additional information (metadata) embedded in the first frame is analyzed based on the above-described predetermined playback video condition, and is displayed during playback according to the predetermined playback video condition from all-round video data. The power trimming position is determined for each frame. A method for determining a predetermined reproduction video condition and a trimming position to be displayed at the time of reproduction for each predetermined reproduction video condition will be described later.

  Next, the reproduction processing unit 34 trims the whole sky video data for each frame based on the determined trimming position to be displayed at the time of reproduction, and executes distortion correction processing on the trimmed video. In the all-sky video data, the video is distorted and recorded especially toward the periphery (edge) of the angle of view. Therefore, the distortion of the trimmed video is corrected by executing a distortion correction process.

  Next, the playback processing unit 34 starts playback using the trimmed video as a playback start position. Next, the playback processing unit 34 sequentially executes the above-described analysis processing for subsequent frames to determine a trimming position to be displayed during playback (step S16). Based on the determined trimming position, the reproduction processing unit 34 trims the video in the display range from the whole sky video data, performs distortion correction, and reproduces and displays the video that is sequentially trimmed while updating it.

  As described above, in the first embodiment, the part of the video based on the all-round video data is determined from the predetermined playback video condition described above, and the video playback is started from that part. Thus, it becomes possible for the user to reproduce the entire sky video more effectively and more easily. In addition, when recording all-sky video data, the video is not recorded in consideration of the playback time, and all the video data of the whole sky is recorded, so that the desired part can be played back and displayed at any time. Is possible.

  Next, during the reproduction of the all-sky video, the control unit 37 determines whether or not a pause operation by the user is detected from the operation unit 36 including the touch panel (step S20). That is, after starting the reproduction of the all-sky video, the user operates (touches) the operation unit 36 including the touch panel in order to view a video other than the video being reproduced (outside the display range). If no pause operation by the user is detected (NO in step S20), the control unit 37 determines whether or not the reproduction of the all-sky video has been completed (step S22).

  If reproduction has not ended (NO in step S22), the process returns to step S16 and the above-described processing is repeated. That is, playback at a trimming position that matches a predetermined playback video condition is continued. On the other hand, when the reproduction is finished without detecting the pause operation by the user (YES in step S22), the process is finished.

  On the other hand, when a pause operation by the user is detected during playback of the all-sky video (YES in step S20), the playback processing unit 34 pauses playback (step S24), and the control unit 37 Then, it is determined whether or not a movement instruction operation by the user has been detected following the pause operation (step S26). That is, when the user wants to view a video other than the video being played back (outside the display range), the user pauses playback and then changes the trimming position in real time using, for example, a finger on the touch panel (operation unit 36). Perform an operation of sliding (called swipe).

  In the playback device 30, when a movement instruction operation by the user is detected (YES in step S26), the playback processing unit 34 changes the trimming position to be displayed at the time of playback based on the user operation and plays back (step S26). S28). At this time, the image is displayed while changing the trimming position to be displayed at the time of reproduction based on the user operation within one frame of the all-sky video data, and the reproduced video is a still image.

  On the other hand, when the movement instruction operation by the user is not detected (NO in step S26), the process proceeds to the next step S30 without changing the trimming position to be displayed at the time of reproduction.

  Next, the control unit 37 determines whether or not a reproduction start operation by the user has been detected (step S30). If the reproduction start operation by the user is not detected (NO in step S30), the process returns to step S26, and steps S26 to S26 are performed until the trimming position to be displayed at the time of new reproduction is determined and the reproduction start operation is detected. S30 is repeatedly executed.

  That is, in steps S26 to S30, for example, when the user pauses the video that has been paused (or may be a movement instruction operation using a cursor), all of the images are displayed according to the operation direction, the operation amount, and the operation speed. Trimming is performed while moving the trimming position from one frame of the celestial video data in real time from the current position and reproduced as a still image. When the video in the designated direction is displayed, the user performs a playback start operation to instruct the video playback.

  When the reproduction start operation by the user is detected (YES in step S30), the control unit 37 starts reproduction of the video trimmed at the trimming position to be displayed at the time of reproduction designated by the movement instruction operation (step S32). ). Next, the control unit 37 determines whether or not a pause operation by the user has been detected (step S34). If no pause operation by the user is detected (NO in step S34), it is determined whether or not the moving image file has ended or a reproduction end instruction operation has been performed (step S36). If the moving image file has not ended or there is no instruction to end playback (NO in step S36), the process returns to step S32, and trimming is performed at the trimming position to be displayed during playback specified in the move operation instruction. Continue playing the video.

  On the other hand, when the pause operation by the user is detected again during the video reproduction at the position specified in the movement operation instruction (YES in step S34), the process returns to step S24, and the processes in steps S24 to S34 described above are performed. repeat. That is, after the video being played back is paused, the trimming position to be displayed at the time of playback is changed again based on the movement instruction operation by the user, and the video trimmed at the changed trimming position by the user's playback start operation is played back. repeat. Therefore, the user can reproduce the all-sky video while changing the desired direction at a desired timing.

  Then, when the moving image file ends or when there is an instruction to end playback (YES in step S36), playback of the all-sky video ends.

  FIGS. 7A to 7C are flowcharts for explaining an example of frame analysis processing for determining a video area (trimming position) to be displayed at the time of playback for each predetermined playback video condition according to the first embodiment. .

  FIG. 7A shows a process of determining a video area (trimming position) to be displayed at the time of reproduction based on the gaze direction of the photographer at the time of shooting (the direction in which the face is directed). In this case, the reproduction processing unit 34 acquires the sight line direction (direction in which the face is directed) of the photographer embedded as additional information (metadata) in the first frame of the moving image file (all-sky video data) (step S1). S40). Then, the playback processing unit 34 determines a video area (trimming position) to be displayed during playback within the frame of the all-sky video data that matches the line-of-sight direction (the direction in which the face is directed) of the acquired photographer ( Step S42). As a result, when the reproduction of the whole sky image is started, the trimming process and the distortion correction process are performed so that the image in the sight line direction of the photographer (the direction in which the face is directed) is displayed, and the image is reproduced on the display unit 35. Is displayed. In this way, at the start of playback, the video in the direction of the photographer's line of sight (the direction in which the face is facing) is preferentially played back and displayed, so that it is easier for the user to view without random playback and display. Can be played.

  FIG. 7B shows a process for determining a video area (trimming position) to be displayed during playback based on the moving direction of the photographer during shooting. In this case, the reproduction processing unit 34 acquires the moving direction of the photographer embedded as additional information (metadata) in the first frame of the moving image file (all-round video data) (step S50). Then, the playback processing unit 34 determines a video area (trimming position) to be displayed during playback within the frame of the all-sky video data that matches the acquired moving direction of the photographer (step S52). Thus, when the reproduction of the all-sky video is started, the trimming process and the distortion correction process are performed so that the video in the moving direction of the photographer is displayed, and the video is reproduced and displayed on the display unit 35. As described above, since the video in the moving direction of the photographer at the time of shooting is preferentially reproduced and displayed at the start of reproduction, the image can be reproduced more easily for the user without being randomly reproduced and displayed.

  FIG. 7C shows a process of determining a video area (trimming position) to be displayed at the time of reproduction based on the sound source direction of the environmental sound at the time of shooting. In this case, the reproduction processing unit 34 analyzes several frames to several tens of frames from the beginning of the moving image file (all-sky video data), and there is a sound source whose sound volume level is greater than or equal to a predetermined threshold. Then, the sound source direction of the environmental sound embedded as metadata in the moving image file (all-round video data) is acquired (step S60).

  Then, the playback processing unit 34 determines a video area (trimming position) to be displayed during playback within the frame of the all-sky video data that matches the sound source direction of the acquired environmental sound (step S62). As a result, when the reproduction of the whole sky video is started, the trimming process and the distortion correction process are performed so that the video in the sound source direction whose volume level is equal to or higher than a predetermined threshold is displayed, and the video is reproduced and displayed on the display unit 35.

  For example, as a sound source that emits a volume level equal to or higher than a predetermined threshold, there is a possibility that something of interest such as a bird, an animal, a child, or a vehicle is photographed (recorded). According to the first embodiment, as compared with simply reproducing and displaying the traveling direction of the photographer, even in the background of the photographer, the direction of the sound source that generates some loud sound in the whole sky recorded video is displayed. Since playback starts from the video, you can't miss it.

  Note that the user can set in advance which method to use among the above-described method of determining by the photographer's line-of-sight direction, the method of determining by the photographer's moving direction, and the method of determining by the sound source direction. That's fine. Alternatively, for example, the sound source direction is first checked, and if the volume level is not equal to or greater than a predetermined threshold, the moving direction is checked. If the photographer is not moving, the line-of-sight direction is used. Priorities may be set in the order of direction, moving direction, and line-of-sight direction.

B. Second Embodiment Next, a second embodiment of the present invention will be described.
In the second embodiment, the front face of the subject included in the shot all-sky video data is detected and the front face of the subject is displayed in the display range to determine from which part of the whole sky video the playback starts. The video area (trimming position) to be displayed at the time of reproduction within the frame of the all-sky video data is determined so as to be substantially the center of Therefore, in the second embodiment, it is only necessary to perform image processing (face recognition processing) on all-sky video data, and there is no need to use additional information (metadata) from the head-mounted device 10. Therefore, the second embodiment is applicable even to all-sky video data shot without wearing the head-mounted device 10 shown in FIG.

  FIG. 8 is a block diagram showing the configuration of the playback device 30 according to the second embodiment. Note that portions corresponding to those in FIG. 4 are denoted by the same reference numerals and description thereof is omitted. The playback device 30 according to the second embodiment includes an image analysis unit 38. The image analysis unit 38 analyzes the first frame of the all-sky video data (moving image file) to detect the face of the subject (front face), and counts from the first frame of the all-sky video data (moving image file). Analyzing frames and detecting moving objects. The reproduction processing unit 34 reads out the moving image file stored in the moving image file storage unit 33, and based on the front face of the subject detected by the image analysis unit 38, the trimming position to be displayed at the time of reproduction in the all-sky image. The video trimmed from the whole sky video data according to the trimming position is displayed on the display unit 35.

  FIG. 9 is a flowchart for explaining a partial operation of the playback apparatus 30 according to the second embodiment. In the second embodiment, only the processing for determining the video region (trimming position) to be displayed at the time of reproduction in steps S10 and S16 shown in FIG. 6 is different, and the main routine shown in FIG. 6 is the same. Therefore, description of common steps is omitted.

  In the second embodiment, the video area (trimming position (direction)) is determined based on the position of the front face of the subject in the captured video. In this case, the image analysis unit 38 reads several frames to several tens of frames (or several seconds to several tens of seconds) from the beginning of the moving image file (all-round video data), and selects the face of the subject that meets a predetermined face condition. It detects (step S70). Here, as the predetermined face condition, for example, a smile, a face of a determined person, a frame occupancy rate (the face is shown larger) can be considered.

  Then, the image analysis unit 38 determines a video area (trimming position) to be displayed at the time of reproduction in the frame of the all-sky video data so that the detected subject (front face thereof) is substantially at the center of the display range ( Step S72). As a result, when the reproduction of the all-sky video is started, the trimming process and the distortion correction process are performed so that the photographed subject is displayed in the approximate center of the display range of the display unit 35, and is reproduced and displayed on the display unit 35. .

  FIG. 10 is a diagram for explaining the relationship between the all-sky image and the trimming position when determining the video region (trimming position) to be displayed at the time of reproduction based on the position of the face of the subject in the second embodiment. It is a conceptual diagram. The same applies to a case where a video area (trimming position) to be displayed at the time of reproduction is determined based on the position of a moving body to be described later. As described above, the first frame of a video file (all-round video data), or several frames to several tens of frames (or several seconds to several tens of seconds) are read from the first frame, and the face of the subject that meets a predetermined face condition Is detected.

  For example, as shown in FIG. 10, even when there is no subject (face) in the front half-sky image 40 of the whole sky video and the face 60 of the subject is detected in the back half-sky image 50, playback is performed. At the start, the trimming position 70 is determined so that the subject is displayed at substantially the center of the display range of the display unit 35, the trimming process and the distortion correction process are performed according to the trimming position 70, and the trimmed video is displayed on the display unit. 35 is reproduced and displayed.

  Therefore, even if you do not realize that someone was behind you at the time of shooting, the person who was behind you will be played and displayed preferentially at the time of playback, so there is no random scenery (front half sky) Without being reproduced and displayed, it can be reproduced more easily for the user. In addition, as described above, after the reproduction is started, the face 60 of the subject is tracked until the user pauses and the display direction is changed, so that the first time without deviating from the display range of the display unit 35 immediately. It is possible to reproduce and display the subject that has been decided on without missing.

C. Third Embodiment Next, a third embodiment of the present invention will be described.
In the third embodiment, the moving object included in the captured all-sky video data is detected as to which part of the whole-sky image is to be reproduced, and the moving object is set as the approximate center of the display range. Thus, the video area (trimming position) to be reproduced and displayed in the frame of the all-sky video data is determined. Therefore, in the third embodiment, it is only necessary to perform image processing (moving body detection processing) on the whole sky video data, and it is not necessary to use additional information from the head-mounted device 10. Therefore, as in the second embodiment, the third embodiment can be applied even to all-sky video data shot without wearing the head-mounted device 10 shown in FIG.

  Note that the configuration of the camera devices 20a and 20b and the configuration of the playback device 30 according to the third embodiment are the same as those of the above-described second embodiment, and a description thereof will be omitted. However, the image analysis unit 38 according to the third embodiment analyzes several frames from the first frame of the omnidirectional video data (moving image file), and detects the moving object based on the difference between the frames. The reproduction processing unit 34 reads out the moving image file stored in the moving image file storage unit 33, and performs trimming to be displayed at the time of reproduction in the all-sky video data based on the direction of the moving body detected by the image analysis unit 38. The position is determined, and the video trimmed from the whole sky video data according to the trimming position is displayed on the display unit 35.

  FIG. 11 is a flowchart for explaining a partial operation of the playback apparatus 30 according to the third embodiment. In the third embodiment, only the processing for determining the video area (trimming position) to be reproduced and displayed in steps S10 and S16 shown in FIG. 6 is different, and the main routine shown in FIG. 6 is the same. Since there are, description of common steps is omitted.

  In the third embodiment, a video area (trimming position) to be displayed at the time of reproduction is determined based on the position of the moving object shown in the captured video. In this case, the image analysis unit 38 analyzes several frames to several tens of frames (or several seconds to several tens of seconds) from the beginning of the moving image file (entire sky video data), and detects the moving object from the difference between the frames. (Step S80). Here, the moving body may be something that attracts some interest, such as a bird, an animal, a person, or a vehicle.

  Then, the image analysis unit 38 determines a video region (trimming position) to be reproduced and displayed within the frame of the all-sky video data so that the detected moving body is substantially at the center of the angle of view (step S82). Thus, when the reproduction of the all-sky video is started, the trimming process and the distortion correction process are performed so that the captured moving body is displayed, and the image is reproduced and displayed on the display unit 35.

  Therefore, as in the case where the presence / absence of the subject's face is set as a predetermined condition, even when it is not noticed that there was any moving object at the time of shooting, the captured moving object is given priority during playback. Since it is reproduced and displayed, an empty landscape (front half sky) can be reproduced more easily for the user without being randomly reproduced and displayed. Further, as described above, after the reproduction is started, the moving object is tracked until the user pauses and the display direction is changed, so that the first determination is made without losing the display range of the display unit 35 in an instant. It is possible to reproduce and display the mobile object without missing it.

  In the above-described embodiment, when the video being played back is paused, for example, it is reduced (zoomed out) by pinch-in (narrowing to hold two fingers on the touch screen) and pinched out (on the touch screen). (Zooming up two fingers) may be used to zoom in (zoom in). In this case, enlargement (zoom-in) and reduction (zoom-out) are realized by expanding and reducing the trimming range. When playback is resumed, playback may be performed in a zoomed-in and zoomed-out state (a state in which the trimming range is enlarged / reduced; digital zooming), or may be played back in the standard trimming range.

  Further, in the first embodiment described above, an example in which the omnidirectional microphone & directional microphone 12 and the audio signal processing unit 13 are provided in the head-mounted device 10 is shown, but the present invention is not limited thereto, and the camera devices 20a and 20b are provided. You may make it provide in. Further, not only the omnidirectional microphone & directional microphone 12 and the audio signal processing unit 13, but also an image processing unit 32 for generating a moving image file may be provided in the camera devices 20a and 20b, and various modifications may be made.

As mentioned above, although several embodiment of this invention was described, this invention is not limited to these, The invention described in the claim, and its equal range are included.
Below, the invention described in the claims of the present application is appended.

(Appendix 1)
The invention described in the supplementary note 1 is stored in the storage unit, a display unit, a storage unit that stores a predetermined range of video, and a video that is displayed in the display range of the display unit based on a predetermined reproduction video condition. A setting means for setting a position in the recorded video, and a control for trimming the video at the position set by the setting means from the video stored in the storage means and displaying the video on the display means at the start of reproduction. Display control means, detection means for detecting an external operation during display by the first display control means, and when the external operation is detected by the detection means, the first display control means And a second display control means for controlling the display means to display the video whose display is controlled by the external operation based on the operation from the outside. It is.

(Appendix 2)
According to the second aspect of the invention, information indicating a shooting direction is added to the video stored in the storage unit, and the setting unit displays the display on the display unit based on the information indicating the shooting direction. The video display device according to appendix 1, wherein a position of the video to be displayed in the range is set in the video.

(Appendix 3)
According to the third aspect of the invention, information indicating a moving direction of the photographing device at the time of photographing is added to the video stored in the storage means, and the setting means is based on the information indicating the moving direction. The video display device according to appendix 1, wherein a position of the video to be displayed in the display range of the display unit is set in the video.

(Appendix 4)
The invention according to attachment 4 obtains, from the video stored in the storage unit, a subject detection unit that detects a subject that satisfies a predetermined condition, and a position of the subject detected by the subject detection unit in the video. Position acquisition means, and the setting means sets the position in the video of the video to be displayed in the display range of the display means based on the position acquired by the position acquisition means. The video display device according to attachment 1.

(Appendix 5)
The invention according to appendix 5 is the video display device according to appendix 4, wherein the subject that satisfies the predetermined condition is a face in the front direction.

(Appendix 6)
The invention according to appendix 6 is the video display device according to appendix 4, wherein the subject that satisfies the predetermined condition is a moving object.

(Appendix 7)
According to the seventh aspect of the present invention, the video stored in the storage means is added with information indicating the direction in which the audio information generated at the time of shooting the video is generated, and the setting means The video display device according to appendix 1, wherein a position in the video of a video to be displayed in a display range of the display unit is set based on information indicating a generation direction.

(Appendix 8)
The invention according to appendix 8 is the video display device according to any one of appendices 1 to 7, wherein the video is a moving image.

(Appendix 9)
The video according to any one of appendices 1 to 8, wherein the first display control means reproduces from the trimmed portion as a reproduction start position in the moving image. It is a display device.

(Appendix 10)
The invention according to appendix 10 is a video display method for trimming a video to be displayed in a display range of a display unit from a predetermined video and displaying the trimmed video on the display unit, and based on a predetermined playback video condition, A step of setting a position of the video to be displayed in the display range of the display means in the predetermined video; and at the start of reproduction, the video at the set position is trimmed from the predetermined video and displayed on the display A step of controlling the display, a step of detecting an external operation during the video display on the display means, and an operation from the outside when the external operation is detected, And controlling to display on the display means based on the video display method.

(Appendix 11)
The invention described in appendix 11 is directed to a video display device computer that trims a video for display in a display range of a display unit from a predetermined video and displays the trimmed video on the display unit based on a predetermined playback video condition. Setting means for setting a position in the predetermined video of the video to be displayed in the display range of the display unit, and trimming the video at the position set by the setting means from the predetermined video at the start of playback, the display unit First display control means for controlling the display to be displayed, detection means for detecting an external operation during display by the first display control means, and when the external operation is detected by the detection means, Second display control means for controlling to display the video controlled by the first display control means on the display unit based on the operation from the outside. Is a program for causing to function.

DESCRIPTION OF SYMBOLS 1 Video recording / reproducing system 10 Head mounting apparatus 11 Sensors 11-1 GPS unit 11-2 Direction / barometric pressure sensor 11-3 Motion sensor 12 Omnidirectional & directional microphone 13 Audio signal processing unit 14 Signal processing unit 15 Wireless communication Unit 16 Control unit 20a, 20b Camera device 21a, 21b Optical system (wide angle lens)
22a, 22b Imaging unit 23a, 23b Image processing unit 24a, 24b Compression encoding (file formation) processing unit 25a, 25b Storage unit 26a, 26b Wireless communication unit 27a, 27b Operation unit 28a, 28b Control unit 30 Playback device 31 Wireless communication Unit 32 Image processing unit 33 Movie file storage unit 34 Playback processing unit 35 Display unit 36 Operation unit 37 Control unit 38 Image analysis unit 40 Front half-sky image 41, 42 Center 50 Back half-sky image 60 Subject face 70 Trimming position

Claims (11)

Display means;
Storage means for storing a predetermined range of video;
Setting means for setting the position in the video stored in the storage means of the video to be displayed in the display range of the display means based on a predetermined reproduction video condition;
First display control means for controlling to display the video at the position set by the setting means from the video stored in the storage means to be displayed on the display means at the start of reproduction;
Detecting means for detecting an operation from the outside during display by the first display control means;
When an operation from the outside is detected by the detection means, a second control is performed to display the image controlled by the first display control means on the display means based on the operation from the outside. Display control means,
A video display device comprising: Information indicating the shooting direction is added to the video stored in the storage means,
The setting means sets the position in the video of the video to be displayed in the display range of the display means based on the information indicating the shooting direction.
The video display device according to claim 1. Information indicating the moving direction of the photographing device at the time of photographing is added to the video stored in the storage means,
The setting means sets the position in the video of the video to be displayed in the display range of the display means based on the information indicating the moving direction.
The video display device according to claim 1. Subject detection means for detecting a subject satisfying a predetermined condition from the video stored in the storage means;
Position acquisition means for acquiring the position of the subject detected by the subject detection means in the video;
Further comprising
The setting means sets the position in the video of the video to be displayed in the display range of the display means based on the position acquired by the position acquisition means.
The video display device according to claim 1. The subject that satisfies the predetermined condition is a front face.
The video display apparatus according to claim 4. The subject that satisfies the predetermined condition is a moving object.
The video display apparatus according to claim 4. Information indicating the generation direction of audio information generated at the time of shooting the video is added to the video stored in the storage means,
The setting means sets the position in the video of the video to be displayed in the display range of the display means based on the information indicating the generation direction of the audio information.
The video display device according to claim 1. The video is a video.
The video display device according to claim 1, wherein the video display device is a video display device. The first display control means reproduces from the trimmed portion as a reproduction start position in the moving image.
The video display device according to claim 1, wherein the video display device is a video display device. A video display method for trimming a video for display in a display range of a display unit from a predetermined video and displaying the trimmed video on the display unit,
Setting a position of the video to be displayed in the display range of the display means in the predetermined video based on a predetermined playback video condition;
Controlling to trim the video at the set position from the predetermined video and display it on the display means at the start of playback;
Detecting an external operation during video display on the display means;
When the operation from the outside is detected, controlling the display-controlled video to be displayed on the display unit based on the operation from the outside;
A video display method comprising: A computer of a video display device for trimming a video for display in a display range of a display unit from a predetermined video and displaying the trimmed video on the display unit,
Setting means for setting a position of the video to be displayed in the display range of the display unit in the predetermined video based on a predetermined playback video condition;
First display control means for controlling the trimming of the video at the position set by the setting means from the predetermined video to be displayed on the display unit at the start of reproduction;
Detecting means for detecting an external operation during display by the first display control means;
When an operation from the outside is detected by the detection means, a second control is performed to display on the display section the video that is controlled by the first display control means based on the operation from the outside. Display control means,
A program characterized by functioning as

Images