JP2021129218A - Image display apparatus, image display method, and image display program - Google Patents

Abstract

To provide an image display apparatus configured to efficiently retrieve a desired scene from a landscape video captured from a moving body.SOLUTION: An image display apparatus 10 includes: an input unit 20 which acquires a landscape video captured by an in-vehicle camera CM; a video generation unit 63 which functions as a travel distance acquisition unit for acquiring information on a distance of travel after imaging is started; a thumbnail generation unit 65 which generates a first thumbnail image and a second thumbnail image; a display image generation unit 66; a display 15; and a touch panel 17. On receipt of a selection for a plurality of first thumbnail images, the apparatus displays second thumbnail images extracted from a landscape video between a selected first thumbnail image and another first thumbnail image adjacent thereto, sequentially in the order of the elapsed time or the travel distance, subsequent to the selected first thumbnail images, in a second arrangement direction which intersects with a first arrangement direction.SELECTED DRAWING: Figure 4

Description

The present invention relates to a moving image display method for displaying an image and a display method thereof.

In recent years, a playback device has become widespread that allows a landscape moving image to be captured by a photographing device provided on a moving body such as a vehicle so that an important scene photographed from the vehicle can be viewed later.

For example, in Patent Document 1, a time bar corresponding to the shooting time from the start to the end of shooting of a landscape video shot by an in-vehicle camera mounted on a moving body and a position on the time bar can be instructed by a user. An instruction display image including the slider shown and the landmark icon indicating the landmark existing along the traveling route when the landscape video was shot at the corresponding position on the time bar is displayed on the display, and the landscape video is further displayed. Among them, a display system for a vehicle that displays a landscape moving image associated with a shooting time corresponding to the position of a slider on the time bar on a display is disclosed.

Further, for example, in Patent Document 2, a still image is extracted from moving image data to generate a thumbnail image, and the thumbnail image is displayed side by side at regular time intervals on a timeline to record a desired scene in the moving image. A moving image search / viewing device that makes it possible to easily find a moving image is disclosed.

Japanese Unexamined Patent Publication No. 2013-32949 Japanese Unexamined Patent Publication No. 2002-281432

However, in the display system of Patent Document 1, when trying to find a part in which a desired landscape is recorded in a moving object in which an external landscape is photographed, the slider position is changed or a landmark icon is displayed. Since it is necessary to search for a desired scene while visually recognizing the image displayed on the display while playing back from the vicinity, it may take time to reach the desired scene if the shooting time of the moving image is long.

Further, in the moving image search / viewing device of Patent Document 2, for example, when the moving speed of the moving body is slow and the change of the landscape per predetermined time is small, the thumbnail image at regular time intervals has the same landscape or image. Since a plurality of images are lined up, it may take some time to find a thumbnail image showing a desired landscape.

The present invention has been made in view of the above points, and provides an image display device capable of efficiently searching for a desired scene from a landscape moving image taken from a moving body, and a reproduction method thereof. With the goal.

The invention according to claim 1 is a plurality of acquisition units for acquiring a moving image taken by an image pickup apparatus, and a plurality of the moving images extracted at a first predetermined interval of an index added to the moving image and indicating the progress of the moving image. A plurality of first thumbnail images of the index are generated, and a plurality of first thumbnail images of the index are extracted from the moving image between the thumbnail images adjacent to each other among the plurality of first thumbnail images. A thumbnail image generation unit that generates two thumbnail images, a display image generation unit that generates display images in which the plurality of first thumbnail images are arranged in the order of progress indicated by the index in the first arrangement direction, and the plurality of display image generation units. The display image generation unit has a selection reception unit that accepts selections for each of the first thumbnail images of the above, and the display image generation unit is the first thumbnail of any one of the plurality of first thumbnail images by the selection reception unit. When an image is selected, a thumbnail image extracted from the moving image between the first thumbnail image among the plurality of second thumbnail images and another first thumbnail image adjacent thereto is used. It is characterized in that an image arranged in a second arrangement direction intersecting with the first arrangement direction is generated following the selected first thumbnail image and in the order of progression indicated by the index.

Further, the invention according to claim 7 is a plurality of acquisition units for acquiring moving images taken from a moving body, and a plurality of acquisition units extracted from the moving body for each elapsed time of a first predetermined interval or for each moving distance of the moving body. The thumbnail image display unit has a thumbnail image display unit that displays the first thumbnail images arranged in the first arrangement direction, and a selection reception unit that accepts selections for each of the plurality of first thumbnail images. When the display unit receives a selection for each of the plurality of first thumbnail images in the selection reception unit, the display unit is between the selected first thumbnail image and another first thumbnail image adjacent thereto. The second thumbnail image extracted from the moving image is arranged in the order of the elapsed time or the moving distance, following the selected first thumbnail image in the second arrangement direction intersecting with the first arrangement direction. It is a feature that it is displayed as a thumbnail.

The invention according to claim 8 is an image generation method in which an image generation device generates a thumbnail image, in which a moving image acquisition step of acquiring a moving image taken by the imaging device and a moving image are added to the moving image. A plurality of first thumbnail images extracted at first predetermined intervals of an index indicating the progress of the moving image are generated, and among the plurality of first thumbnail images, the thumbnail images adjacent to each other are described. A thumbnail image generation step of generating a plurality of second thumbnail images extracted from the moving image at second predetermined intervals of the index, and the index moving the plurality of first thumbnail images in the first arrangement direction. A display image generation step for generating display images arranged in the order of progress shown, a selection reception step for accepting selection for each of the plurality of first thumbnail images, and the display image generation step include the selection reception unit. When any one of the first thumbnail images of the plurality of first thumbnail images is selected, the first thumbnail image of the plurality of second thumbnail images and other images adjacent thereto are selected. A second that intersects the thumbnail image extracted from the moving image with the first thumbnail image with the first arrangement direction following the selected first thumbnail image and in the order of progress indicated by the index. It is characterized by generating an image arranged in the arrangement direction of.

The invention according to claim 9 is an image generation program executed by a computer, wherein the computer has a moving image acquisition step of acquiring a moving image taken by an imaging device, and the moving image is added to the moving image. A plurality of first thumbnail images extracted at first predetermined intervals of an index indicating the progress of the moving image are generated, and among the plurality of first thumbnail images, the thumbnail images adjacent to each other are described. A thumbnail image generation step of generating a plurality of second thumbnail images extracted from the moving image at second predetermined intervals of the index, and the index moving the plurality of first thumbnail images in the first arrangement direction. A display image generation step for generating display images arranged in the order of progress shown and a selection acceptance step for accepting selections for each of the plurality of first thumbnail images are executed, and the display image generation step is the selection acceptance. When the first thumbnail image of any of the plurality of first thumbnail images is selected by the unit, the first thumbnail image of the plurality of second thumbnail images and the other adjacent to the first thumbnail image. The thumbnail image extracted from the moving image between the first thumbnail image and the first thumbnail image is crossed with the first arrangement direction in the order of progress indicated by the index, following the selected first thumbnail image. It is characterized in that an image arranged in the arrangement direction of 2 is generated.

It is a perspective view of the automobile M equipped with the image generation apparatus 10 which concerns on Example 1 of this application. It is a figure which looked at the rear door of the automobile M equipped with the image generation apparatus 10 which concerns on Example 1 of this application from the rear seat of the automobile M. It is a figure which shows an example of the operation acceptance display displayed on the touch panel 17 of the image generation apparatus 10 which concerns on Example 1 of this application. It is a functional block diagram of the image generation apparatus 10 which concerns on Example 1 of this application. This is an example of displaying thumbnail images on the display 15 and the touch panel by the image generation device 10 according to the first embodiment. In the image generation device 10 according to the first embodiment of the present application, the moving image generation unit 63 assigns the moving distance of the automobile M at the time of shooting to the landscape moving image shot by the in-vehicle camera CM, and generates a moving distance moving image. FIG. 1 is a flow chart FC1 of processing. FIG. 2 is a flowchart FC2 of a thumbnail image generation process for generating a thumbnail image of the image generation device 10 according to the first embodiment of the present application. This is an example in which the basic thumbnail image and the interpolated thumbnail image are generated at predetermined elapsed times in the image generation device 10 according to the first embodiment of the present application. This is an example in which the image generation device 10 according to the first embodiment of the present application generates a basic thumbnail image at predetermined time intervals and an interpolated thumbnail image for each predetermined movement distance between the basic thumbnail images. FIG. 3 is a flowchart FC3 of a display image generation process in which the display image generation unit 66 displays a thumbnail image group on the touch panel 17 and the display 15 at the time of reproducing a landscape moving image of the image generation device 10 according to the first embodiment of the present application. It is a figure which shows another example of the operation acceptance display displayed on the touch panel 17 of the image generation apparatus 10 which concerns on Example 1 of this application. It is a figure which looked at the rear door of the automobile M equipped with the image generator 10A which concerns on Example 2 of this application from the rear seat of the automobile M. It is a figure which shows the example of the operation acceptance display displayed on the touch panel 19 of the image generation apparatus 10A which concerns on Example 2 of this application. FIG. 4 is a flowchart FC4 of a display image generation process in which the display image generation unit 66 displays a thumbnail image group on the touch panel 19 at the time of reproducing a landscape moving image of the image generation device 10A according to the second embodiment of the present application.

Examples of the present invention will be described in detail below.

FIG. 1 is a perspective view of an automobile M equipped with the image generation device 10 according to the first embodiment of the present application.

The image generation device 10 is an image generation device that captures a landscape moving image around the automobile M by an in-vehicle camera mounted on the automobile M and reproduces the captured landscape moving image. In the following description, a configuration capable of displaying a landscape moving image in the window of the automobile M will be described.

The image generator 10 is mounted in, for example, the dashboard of the automobile M. The image generation device 10 is configured to be capable of reproducing and outputting a moving image acquired from the outside or held by itself.

The in-vehicle camera CM is mounted on, for example, a roof portion or a door panel portion of the automobile M, and is configured to be capable of photographing the scenery in front, side, or rear of the automobile M. The in-vehicle camera CM is communicably connected to the image generation device 10, and is configured to be able to transmit the captured still image or moving image to the image generation device 10.

The display 15 as a moving image display unit is, for example, a touch panel display composed of a transparent panel such as a transparent liquid crystal display or an organic EL display and a touch pad. It is a display attached to the window WD of the automobile M and capable of displaying an image visible from the inside of the automobile. The display 15 is communicably connected to the image generation device 10, and is configured to be able to display a still image or a moving image output from the image generation device 10.

For example, the display 15 is configured so that landscape moving images of the front, side, or rear of the automobile M taken by the in-vehicle camera CM can be displayed in the front window, the door window, and the rear window, respectively. The display 15 may be, for example, a display of a car navigation system installed inside the automobile M.

FIG. 2 is a view of the rear door of the automobile M equipped with the image generator 10 according to the first embodiment of the present application as viewed from the rear seat of the automobile M.

As described above, the display 15 is a touch panel display composed of, for example, a transparent panel such as a transparent liquid crystal display or an organic EL display and a touch pad, and is attached to the window WD of the automobile M. Therefore, for example, when the moving image is not displayed on the display 15, it is possible to see the outside scenery from the inside of the automobile M through the display 15.

The touch panel 17 is a touch panel display attached to the inside of the door panel. For example, the touch panel 17 is a touch panel display including a display 17A such as a liquid crystal panel and a touch pad 17B attached to the display surface of the display and accepting an operation with a human finger or the like.

The touch panel 17 is communicably connected to the image generation device 10 and can display the video output from the image generation device 10. Further, the touch panel 17 can accept the operation of playing, fast-forwarding, or fast-rewinding the moving image displayed on the display 15, and can transmit an instruction based on the acceptance of the operation to the image generation device 10.

Specifically, for example, on the touch panel 17, thumbnail images for each predetermined time of the moving image being reproduced on the display 15 or for each predetermined moving distance at the time of shooting a landscape moving image are displayed side by side in the order of reproduction time.

FIG. 3 is a diagram showing an example of an operation reception display displayed on the touch panel 17 of the image generation device 10 according to the first embodiment of the present application.

As shown in FIG. 3, the operation reception display is, for example, a basic thumbnail image sequence in which the basic thumbnail images SM1 to SM6 for each predetermined time or predetermined movement distance of the landscape video being played on the display 15 are arranged in the order of playback time. SR, a play button PL for instructing to switch between play and pause, a fast forward button FW for instructing a fast forward operation, a fast rewind button RW for instructing a fast rewind operation, and a bar BA indicating the current playback time position of a landscape video. The time display unit TM is included. Further, the thumbnail images included in the basic thumbnail image string SR are displayed while scrolling from right to left according to the passage of the playback time of the landscape moving image.

FIG. 4 is a functional block diagram of the image generation device 10 according to the first embodiment of the present application. The image generation device 10 is, for example, a device in which the input unit 20, the output unit 30, the operation input unit 40, the storage unit 50, and the moving image control unit 60 cooperate with each other via the system bus 13.

The input unit 20 is an interface for acquiring data from an external device of the image generation device 10. The input unit 20 is connected to an in-vehicle camera CM provided in the automobile M. For example, the image generation device 10 can capture a landscape image from the in-vehicle camera CM via the input unit 20.

Further, the input unit 20 is a speed sensor SS that acquires the traveling speed of the automobile and a global positioning system (GPS) that is one of the global navigation satellite systems (GNSS: Global Navigation Satellite System) that acquires the position information of the automobile. : Global Positioning System) is connected to the GPS receiver GP that can receive information. That is, the image generator 10 is configured so that the speed of the automobile M can be acquired from the speed sensor SS and GPS information can be acquired from the GPS receiver GP.

The output unit 30 is an interface that outputs data to an external device of the image generation device 10. The output unit 30 is connected to the display 17A of the display 15 and the touch panel 17 described above, and can output an image to these. Specifically, for example, the image generation device 10 outputs a moving image to the display 15 via an output unit, and outputs an operation image for a reproduction operation to the display 17A.

The operation input unit 40 is an interface for acquiring an operation input signal from an external device. The operation input unit 40 is connected to the touch pad 17B, and can acquire a signal indicating an operation received by the touch pad 17B with respect to the operation image.

The storage unit 50 is composed of, for example, a hard disk device, an SSD (solid state drive), a flash memory, or the like, and stores various programs related to the generation of thumbnail images, the reproduction of moving images, and the like in the image generation device 10. The various programs may be acquired from another server device or the like via a network (not shown), or may be recorded on a recording medium and read via various drive devices. May be good.

Various programs stored in the storage unit 50 can be transmitted via a network, and can be recorded on a computer-readable recording medium and transferred. Further, in the storage unit 50, data captured from the outside of the image generation device 10, for example, landscape moving image data taken from the camera CM, the vehicle at the time of taking the scenery moving image taken from the speed sensor SS and the GPS receiver GP. The speed and position information of M can be stored. Further, the storage unit 50 can store the data generated inside the image generation device 10.

The moving image control unit 60 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and functions as a computer. In the moving image control unit 60, the CPU realizes various functions by reading and executing various programs stored in the ROM and the storage unit 50.

The moving image control unit 60 adds information on the moving distance of the automobile M since the start of shooting the scenery moving image to the scenery moving image acquired via the input unit 20, and generates a moving distance holding video for playback. It has a generation unit 63.

The moving image generation unit 63 reads out the landscape moving image recorded in the storage unit 50 and the data of the speed and position information of the automobile M when the scenery moving image is taken. From the read speed and position information data of the automobile M, the moving distance of the automobile M from the start of shooting to each time when the landscape moving image is taken is calculated.

Further, the moving distance of the automobile M from the start of shooting to each time when each of the plurality of frames is shot is given to each of the plurality of frame images constituting the read landscape moving image. For example, the moving image generation unit 63 calculates a moving distance based on the speed or position information of the automobile M for each frame image, and assigns the moving distance to the landscape moving image. That is, the moving image generation unit 63 uses the moving distance of the automobile M from the start of shooting the moving image as the second index in addition to the time information (so-called time code) which is the first index indicating the progress of the landscape moving image. Add to video data. In other words, the moving distance of the automobile M is an in-vehicle camera CM.

The moving distance holding moving image generated by the moving distance generation unit 63 is stored in the storage unit 50.

The moving image generation unit 63 functions as an acquisition unit for acquiring a landscape moving image taken from the storage unit 50, and functions as a moving distance acquisition unit for acquiring information on the moving distance from the speed and position information of the automobile M.

The thumbnail generation unit 65 as the thumbnail image generation unit starts the landscape video, that is, the predetermined elapsed time from the start of shooting the landscape video or the predetermined elapsed time from the start of the landscape video from the plurality of frame images constituting the moving distance holding video recorded in the storage unit 50. A frame image is extracted for each predetermined movement distance, and a basic thumbnail image to be displayed in the thumbnail display area SR of the touch panel 17 is generated from the extracted frame image.

Further, the thumbnail generation unit 65 extracts the frame images between the frame images extracted for generating the basic thumbnail image for each predetermined elapsed time or for each predetermined movement distance, and uses the extracted frame images as the extracted frame images. Generate an interpolated thumbnail image based on. That is, the thumbnail generation unit 65 generates basic thumbnail images as a plurality of first thumbnail images extracted from the landscape moving image for each elapsed time or movement distance at a predetermined interval, and the basic thumbnail images adjacent to each other are generated. An interpolated thumbnail image is generated as a plurality of second thumbnail images extracted from the landscape moving image in between.

The number of thumbnail images generated can be set arbitrarily. For example, the number of thumbnail images generated is preset in relation to the size of the display area of the display destination. Alternatively, for the number of thumbnail images generated, if you want to increase the number of thumbnail images to be displayed at the display destination, reduce the size of each thumbnail image, and if you want to increase the size of each thumbnail image to be displayed, display the thumbnail. It may be set by the user so that it can be displayed in the display area by reducing the number of images.

The display image generation unit 66 supplies the operation image corresponding to the reproduced moving distance holding moving image to the display 17A via the output unit 30. As described above with respect to FIG. 3, the operation images include, for example, a basic thumbnail image sequence SR, a play button PL for instructing playback or pause switching, a fast-forward button FW for instructing a fast-forward operation, and a fast-rewind operation. It includes a fast-rewind button RW, a bar BA indicating the current playback time position of the landscape moving image, and a time display unit TM. Further, the thumbnail images included in the basic thumbnail image string SR are displayed while scrolling from right to left according to the passage of the playback time of the landscape moving image.

In other words, the display image generation unit 66 supplies the basic thumbnail image generated by the thumbnail generation unit 65 to the display 17A via the output unit 30 and displays it as the basic thumbnail image string SR. Further, when any of the basic thumbnail images in the basic thumbnail image string SR is selected on the touch panel 17, it is generated between the selected basic thumbnail image and another basic thumbnail image adjacent in the forward direction. The interpolated thumbnail image is displayed on the display 15.

Further, the display image generation unit 66 acquires the operation signal received by the touch pad 17B and issues an instruction regarding the reproduction control of the moving image based on the operation signal. The display image generation unit 66 functions as an operation reception unit as a selection reception unit in that the operation from the touch pad 17B is received.

For example, when the display image generation unit 66 receives an operation on the play button PL that gives an instruction to switch the reproduction or the pause described above, the display image generation unit 66 issues an instruction to start the reproduction or to temporarily stop the reproduction. Further, for example, when the display image generation unit 66 receives the operations of the fast-forward button FW for instructing the fast-forward operation and the fast-rewind button RW for instructing the fast-rewind operation, the display image generation unit 66 issues fast-forward and fast-rewind instructions according to the respective operations. .. Further, for example, when the display image generation unit 66 accepts the selection operation for the basic thumbnail image or the interpolated thumbnail image, the display image generation unit 66 issues an instruction for cueing reproduction from the frame image corresponding to the thumbnail image.

The reproduction control unit 67 reads out the moving distance holding video recorded in the storage unit 50, and performs playback control of the moving distance holding video in response to instructions for playback, pause, fast forward, or fast rewind. The reproduction control unit 67 supplies the reproduced moving distance holding video to the display 15 via the output unit 30. The reproduction control unit 67 controls the reproduction of the moving image based on the instruction issued from the display image generation unit 66 in response to the operation on the touch pad 17B.

FIG. 5 shows an example of displaying thumbnail images on the display 15 and the touch panel by the image generation device 10 according to the first embodiment.

In the basic thumbnail image string SR of the touch panel 17, the basic thumbnail images generated by the thumbnail generation unit 65 are displayed side by side in the order of playback time. Further, for example, the interpolated thumbnail image is displayed by selecting one of the basic thumbnail images of the basic thumbnail image string SR with a finger.

Specifically, for example, a basic thumbnail image in which an interpolated thumbnail image for a predetermined time or a predetermined movement distance between a selected basic thumbnail image and another basic thumbnail image displayed adjacent in the forward direction is selected is selected. Is displayed above. In this embodiment, the display surface of the touch panel 17 is not sufficient, so that the interpolated thumbnail image is displayed on the display 15.

For example, as shown in FIG. 5, the interpolated thumbnail image sequence ISR displayed in the order of playback time in the vertical direction is expanded and displayed at a horizontal position corresponding to the basic thumbnail image selected from the lower part of the display 15.

By selecting one of the interpolated thumbnails from the interpolated thumbnail image string ISR displayed on the display 15 again with a finger, the moving image displayed on the display 15 is cueed and reproduced from the time corresponding to the selected interpolated thumbnail image. It becomes possible to do. In order to detect this selection, a touch pad similar to that of the touch panel 17 may be provided in a portion of the display 15 where the interpolated thumbnail image sequence ISR can be displayed. The detection of the selection operation for the display of the display 15 may be performed by using another technique such as motion capture.

By selecting a part other than the interpolated thumbnail image displayed in the state where the interpolated thumbnail image string ISR is displayed, the interpolated thumbnail image string ISR is not displayed so as not to interfere with the viewing of the landscape moving image. When another basic thumbnail image is selected, the interpolated thumbnail image string ISR corresponding to the selected basic thumbnail image is newly expanded and displayed at the horizontal position corresponding to the selected basic thumbnail image. May be done. The basic thumbnail image string SR and the interpolated thumbnail image string ISR function as a graphical user interface (GUI) of the selection reception unit that accepts selections for each of the displayed thumbnail images.

When the user tries to search for a desired scene from the captured landscape video, for example, the user has an idea of where the desired scene is recorded in the landscape video, and the user has an idea. The preferred elapsed time interval or movement distance interval between the plurality of thumbnail images displayed as the basic thumbnail image string SR is different from that in the case where the image is not provided. In the former case, the thumbnail image string SR, which is a thumbnail image around the elapsed time that has already been guessed and the elapsed time interval between the thumbnail images is relatively short, is effective. In the latter case, which is the desired scene first? Since it is necessary to guess whether or not the thumbnail images are recorded around, the thumbnail image string SR having a relatively long elapsed time interval or moving distance interval between the thumbnail images is effective. In order to correspond to this, the image generation device 10 has an elapsed time interval or a moving distance interval between adjacent thumbnail images of a plurality of thumbnail images displayed on the thumbnail image string SR as a part of the operation reception display, that is, a basic thumbnail image string. The time scale or distance scale of SR can be set by the user.

As described above, the image generation device 10 acquires a moving image taken by the in-vehicle camera CM, and a plurality of basic thumbnails obtained from the moving image and extracted from the moving image at first predetermined intervals of an index indicating the progress of the moving image. A thumbnail generation unit 65 that generates an image and generates a plurality of interpolated thumbnail images extracted at a second predetermined interval of an index from a moving image between thumbnail images adjacent to each other among a plurality of basic thumbnail images. A display image generation unit 66 that generates a display image in which a plurality of basic thumbnail images are arranged in the order of progress indicated by an index in the first arrangement direction, and a touch panel 17 and a display 15 that accept selection for each of the plurality of basic thumbnail images. When the basic thumbnail image of any of the plurality of basic thumbnail images is selected by the touch panel 17 and the display 15, the display image generation unit 66 together with the basic thumbnail image of the plurality of interpolated thumbnail images. A second array in which the thumbnail image extracted from the moving image between the other basic thumbnail images adjacent thereto intersects the first arrangement direction in the order of progress indicated by the index, following the selected basic thumbnail image. Generate images arranged in a direction.

Further, the thumbnail generation unit 65 divides the moving image into a plurality of moving image portions at the first predetermined interval, extracts the first frame of each of the plurality of moving image parts, generates a plurality of basic thumbnail images, and generates a plurality of basic thumbnail images. A frame image is extracted from each of the moving image portions at each of the second predetermined intervals smaller than the first predetermined interval to generate a plurality of interpolated thumbnail images, and the first predetermined interval and the second predetermined interval are generated. Is an interval based on either the elapsed time in the moving image or the moving distance of the in-vehicle camera CM as an index indicating the progress of the moving image.

Further, the image generation device 10 includes a display image generation unit 66 that acquires information on the moving distance of the vehicle-mounted camera CM since the start of video recording, and is either a first predetermined interval or a second predetermined interval. One is an interval based on the moving distance as an index indicating the progress of the moving image.

Further, the display image generation unit 66 displays a plurality of basic thumbnail images side by side in the order of elapsed time in the moving image or the moving distance of the in-vehicle camera CM as an index indicating the progress of the moving image along the first arrangement direction. Above or below each of the plurality of basic thumbnail images, each of the interpolated thumbnail images contained in the plurality of moving image portions corresponding to each of the plurality of basic thumbnail images elapses in the second arrangement direction perpendicular to the first arrangement direction. Generate display images arranged in chronological order or travel distance order.

Further, the image generation device 10 has a reproduction control unit 67 that controls the reproduction of moving images and outputs an image, and the touch panel 17 and the display 15 have a plurality of basic thumbnails of the display image generated by the display image generation unit 66. The image selection for each of the image or the plurality of interpolated thumbnail images is accepted, and the reproduction control unit reproduces the moving image from the position corresponding to the selected basic thumbnail image or the interpolated thumbnail image.

Further, the image generation device 10 acquires a moving image taken from the automobile M, and first obtains a plurality of basic thumbnail images extracted from the moving image for each elapsed time of the first predetermined interval or the moving distance of the automobile M. The display 15 has a touch panel 17 that is arranged and displayed in the arrangement direction and accepts selections for each of the plurality of basic thumbnail images, and the display 15 receives selections for each of the plurality of basic thumbnail images on the touch panel 17. An interpolated thumbnail image extracted from a moving image between the selected basic thumbnail image and another basic thumbnail image adjacent thereto is selected as a basic thumbnail image in a second arrangement direction that intersects the first arrangement direction. Followed by, they are arranged and displayed in order of elapsed time or travel distance.

The basic thumbnail image and the interpolated thumbnail image may be the extracted frame image itself or a reduced version of the extracted frame image. Hereinafter, both the process of extracting a frame image from a landscape video and using it as it is as a thumbnail image and the process of reducing the extracted frame image to generate a thumbnail image will be described as simply extracting a thumbnail image. ..

FIG. 6 shows, in the image generation device 10 according to the first embodiment of the present application, the moving distance generation unit 63 assigns the moving distance of the automobile M at the time of shooting to the landscape moving image shot by the in-vehicle camera CM to obtain the moving distance holding video. The flowchart FC1 of the moving image generation processing to be generated is shown.

In the first embodiment, the basic thumbnail image and the interpolated thumbnail image extracted from the landscape video are extracted for each predetermined elapsed time from the start of the landscape video, that is, the start of shooting the landscape video, or a predetermined movement distance of the automobile M. Therefore, the moving image generation unit 63 adds information on the moving distance of the automobile M at the time of shooting the scenery moving image to the scenery moving image, and generates the moving distance holding moving image.

The moving image generation process may be automatically performed at the end of shooting the landscape movie or at the start of playing the landscape movie. Further, the moving image generation process may be arbitrarily performed by the user after the shooting of the landscape moving image is completed and the scenery moving image is saved. Further, the moving image generation process may be automatically executed at an arbitrary timing when a landscape moving image to which information on the moving distance is not added is saved.

When the generation of the travel distance video is started, as a landscape video acquisition step, the video generation unit 63 records the landscape video in the storage unit 50, the speed sensor SS and the GPS receiver GP provide the traveling speed and position information of the automobile M. Is read (step S101).

Next, as a moving distance acquisition step, the moving distance generation unit 63 calculates the moving distance of the automobile M when the landscape moving image is taken from the read traveling speed and position information of the automobile M (step S102).

Next, the moving image generation unit 63 assigns each of the plurality of frame images constituting the read landscape moving image a moving distance from the start of taking the scenery moving image of the automobile M at the time when the frame is taken (step). S103).

After the end of step S103, the moving image generation unit 63 records the landscape moving image to which the moving distance is given in the storage unit 50 as the moving distance holding movie (step S104).

By performing the above processing, from at least one of the landscape video taken by the in-vehicle camera CM, the traveling speed of the automobile M acquired by the speed sensor SS, and the position information of the automobile M acquired by the GPS receiver GP. , Generates a moving distance holding video to which the moving distance when the landscape video was shot is given.

In the first embodiment, it has been described that the moving distance of the automobile M is given to the landscape moving image after the scenery moving image is shot.

However, the moving distance of the automobile M may be given to the landscape moving image while shooting the landscape moving image. In the case of performing while shooting, each time a frame image of a landscape moving image is supplied, the moving distance is calculated and given from the traveling speed and position information of the automobile M at the time of acquiring the frame image.

FIG. 7 shows a flowchart FC2 of a thumbnail image generation process for generating a thumbnail image of the image generation device 10 according to the first embodiment of the present application.

The thumbnail image generation process for generating the thumbnail image may be executed in parallel with the flowchart FC1 for generating the moving distance holding moving image. Further, the flow of the thumbnail image generation process may be automatically started when the playback of the landscape moving image is started. Further, the thumbnail image generation process may be performed at an arbitrary timing according to an instruction from the user after the shooting is completed.

When the generation of the thumbnail image is started, the thumbnail generation unit 65 reads out the moving distance holding video recorded in the storage unit 50 (step S201).

The thumbnail generation unit 65 is the first extraction interval for extracting the basic thumbnail image from the set time scale or distance scale of the basic thumbnail image string SR and the set quantity of the basic thumbnail images to be generated. Is determined (step S202).

The thumbnail generation unit 65 extracts a first frame image group having information for each first extraction interval from a plurality of frame images constituting the landscape moving image based on the determined first extraction interval (step S203). ).

The thumbnail generation unit 65 generates a basic thumbnail image group from the first frame image group (step S204).

Next, the thumbnail generation unit 65 determines the second extraction interval for extracting the interpolated thumbnail image from the elapsed time or the moving distance between each of the frame images extracted as the basic thumbnail image (step S205).

The thumbnail generation unit 65 has a second frame having information for each second extraction interval from a plurality of frame images between each of the frame images extracted as the basic thumbnail image based on the determined second extraction interval. The image group is extracted (step S206).

The thumbnail generation unit 65 generates an interpolated thumbnail image group from the second frame image group (step S207).

After that, the thumbnail generation unit 65 records the generated basic thumbnail image group and the interpolated thumbnail image group in the storage unit 50 (step S208).

By the above processing, it is possible to generate a basic thumbnail image group and an interpolated thumbnail image group according to the elapsed time or the moving distance at the time of shooting from the shot landscape moving image.

In addition, steps S204 and S207 function as thumbnail image generation steps.

Next, the basic thumbnail image and the interpolated thumbnail image generated from the moving distance holding video will be described with reference to FIGS. 8 and 9.

FIG. 8 shows an example in which the image generation device 10 according to the first embodiment of the present application generates a basic thumbnail image and an interpolated thumbnail image at predetermined elapsed times.

In the first embodiment, a case where the extraction interval of the basic thumbnail image is 5 minutes and the extraction interval of the interpolated thumbnail image is 1 minute in the 30-minute landscape moving image taken by the in-vehicle camera CM will be described.

Since the extraction interval of the basic thumbnail image is set to 5 minutes, the thumbnail generation unit 65 uses 0:00, 5:00, 10:00, 15:00, and 20 as the basic thumbnail image in the 30-minute landscape video. The frame images corresponding to the time codes at 0:00 and 25:00 are extracted from the moving distance holding video, and the six basic thumbnail images SM1 to SM6 generate the basic thumbnail image group BSM.

Further, since the extraction interval of the interpolated thumbnail image is set to 1 minute, the thumbnail generation unit 65 is placed between the basic thumbnail image SM1 and the basic thumbnail image SM2 at 1: 1 of every minute from 0:00 of the basic thumbnail image SM1. Four interpolated thumbnail images SMi11 to SMi14 at 0:00, 2:00, 3:00, and 4:00 are generated.

Further, the thumbnail generation unit 65 between the basic thumbnail image SM2 and the basic thumbnail image SM3 is set to 6:00, 7:00, and 8:00 every minute from 5:00 of the basic thumbnail image SM2. Four interpolated thumbnail images SMi21 to SMi24 at 9:00 are generated.

In addition, the thumbnail generation unit 65 between the basic thumbnail image SM3 and the basic thumbnail image SM4 has the basic thumbnail image SM3 at 11:00, 12:00, and 13:00 every minute from 10:00. Four interpolated thumbnail images SMi31 to SMi34 at 14:00 are generated.

In addition, the thumbnail generation unit 65, between the basic thumbnail image SM4 and the basic thumbnail image SM5, is set every minute from 15:00 to 16:00, 17:00, and 18:00 of the basic thumbnail image SM4. Four interpolated thumbnail images SMi41 to SMi44 at 19:00 are generated.

Further, the thumbnail generation unit 65 between the basic thumbnail image SM5 and the basic thumbnail image SM6 is set every minute from 20:00 to 21:00, 22:00, and 23:00 of the basic thumbnail image SM5. Four interpolated thumbnail images SMi51 to SMi54 at 24:00 are generated.

In addition, the thumbnail generation unit 65 is between the basic thumbnail image SM6 and the shooting end time of 30:00, every minute from 25:00 to 26:00, 27:00, and 28:00 of the basic thumbnail image SM6. At the hour, 4 interpolated thumbnail images SMi61 to SMi64 at 29:00 are generated.

Further, the 24 interpolated thumbnail images of the interpolated thumbnail images SMi11 to SMi64 generated by the thumbnail generation unit 65 are generated as the interpolated thumbnail image group ISM.

According to the first embodiment, the basic thumbnail image group BSM and the interpolated thumbnail image group ISM are generated at predetermined time intervals, respectively. As a result, the thumbnail images can be confirmed in detail by the interpolated thumbnail image group, and the desired scene in the landscape moving image can be efficiently searched.

FIG. 9 shows an example in which the image generation device 10 according to the first embodiment of the present application generates a basic thumbnail image at predetermined time intervals and an interpolated thumbnail image for each predetermined movement distance between the basic thumbnail images.

In the first embodiment, the extraction interval of the basic thumbnail image is set to 5 minutes and the extraction interval of the interpolated thumbnail image is set to 1 km in the landscape moving image in which the time length taken by the vehicle-mounted camera CM is 30 minutes and the moving distance is 15 km. The case will be described.

Since the extraction interval of the basic thumbnail image is set to 5 minutes, the thumbnail generation unit 65 uses 0:00, 5:00, 10:00, 15:00, and 20 as the basic thumbnail image in the 30-minute landscape video. The frame images corresponding to the time codes at 0:00 and 25:00 are extracted from the moving distance holding video, and the six basic thumbnail images SM1 to SM6 generate the basic thumbnail image group BSM.

Further, since the extraction interval of the interpolated thumbnail image is set to 1 km, the thumbnail generation unit 65 includes the basic thumbnail image SM1 to which the movement distance information of 0.0 km is given and the basic thumbnail image to which the movement distance information of 1.2 km is given. An interpolated thumbnail image SMi11, which is a frame image at 4:30, corresponding to a movement distance of 1.0 km after 1 km from a movement distance of 0.0 km of the basic thumbnail image SM1 is generated with SM2.

Further, the thumbnail generation unit 65 is 1 of the basic thumbnail image SM2 between the basic thumbnail image SM2 to which the movement distance information of 1.2 km is given and the basic thumbnail image SM3 to which the movement distance information of 5.6 km is given. Information on the travel distances of 2.2 km, 3.2 km, 4.2 km, and 5.2 km per 1 km was added between the travel distance of .2 km and the travel distance of 5.6 km of the basic thumbnail image SM3 at 6:30. Four interpolated thumbnail images SMi21 to SMi24, which are frame images at hour, 8:10, 8:40, and 9:15, are generated.

Further, the thumbnail generation unit 65 is 5 of the basic thumbnail image SM3 between the basic thumbnail image SM3 to which the movement distance information of 5.6 km is given and the basic thumbnail image SM3 to which the movement distance information of 7.2 km is given. An interpolated thumbnail image SMi31, which is a frame image at 13:00, is generated with information on a moving distance of 6.6 km after 1 km from a moving distance of .6.6 km.

Further, the thumbnail generation unit 65 7 of the basic thumbnail image SM4 between the basic thumbnail image SM4 to which the movement distance information of 7.2 km is given and the basic thumbnail image SM5 to which the movement distance information of 10.7 km is given. Information on the travel distances of 8.2 km, 9.2 km, and 10.2 km per 1 km between the travel distance of .2 km and the travel distance of 10.7 km of the basic thumbnail image SM3 was added at 15:30, 16 :. Three interpolated thumbnail images SMi41 to SMi43, which are frame images at 20:00 and 17:15, are generated.

Further, the thumbnail generation unit 65 is 10 of the basic thumbnail image SM5 between the basic thumbnail image SM5 to which the movement distance information of 10.7 km is given and the basic thumbnail image SM6 to which the movement distance information of 14.8 km is given. Information on the travel distances of 11.7 km, 12.7 km, 13.7 km, and 14.7 km per 1 km was added between the travel distance of .7 km and the travel distance of 14.8 km of the basic thumbnail image SM6 at 21:30. Four interpolated thumbnail images SMi51 to SMi54, which are frame images at the hour, 22:05, 23:30, and 24:45, are generated.

Further, the thumbnail generation unit 65 does not have a predetermined movement distance for generating the interpolated thumbnail image between the basic thumbnail image SM6 to which the movement distance information of 14.8 km is added and the shooting end time of 15.0 km. No interpolated thumbnail image is generated.

Further, the 13 interpolated thumbnail images of the interpolated thumbnail images SMi11 to SMi54 generated by the thumbnail generation unit 65 are generated as the interpolated thumbnail image group ISM.

According to the first embodiment, the basic thumbnail image group BSM is generated at predetermined time intervals, and the interpolated thumbnail image group ISM is generated at predetermined movement distances. As a result, when a landscape video is taken while the automobile M is running, the thumbnail image can be confirmed in detail by the interpolated thumbnail image group in the section where the amount of change in the landscape video is large and the running speed is high. It is possible to efficiently search for a desired scene in a landscape moving image.

Although the extraction interval of the interpolated thumbnail image has been described here as 1 km, the extraction interval is not limited to this. For example, the extraction interval of the interpolated thumbnail image is appropriately determined according to the difference in moving distance between the basic thumbnail images adjacent to each other in the basic thumbnail image group BSM and the display destination space of the interpolated thumbnail image. It may be set automatically based on the maximum number of displayed images, and the extraction interval of the interpolated thumbnail image may be arbitrarily specified by the user.

Further, as shown in FIG. 9, the interpolated thumbnail image corresponding to the basic thumbnail image may not be generated. In that case, for example, each of the basic thumbnail groups displayed in the basic thumbnail image string SR may be provided with an identification display indicating whether or not an interpolated thumbnail is generated or an identification display indicating the number of interpolated thumbnails generated. ..

That is, the display image generation unit 66 displays a basic thumbnail image in which at least a part of the plurality of interpolated thumbnail images is displayed when the basic thumbnail image of one of the plurality of basic thumbnail images is selected by the touch panel 17. Is generated in a manner that can be identified from other basic thumbnail images.

FIG. 10 is a flowchart FC3 of a display image generation process in which the display image generation unit 66 displays a thumbnail image group on the touch panel 17 and the display 15 when the landscape moving image of the image generation device 10 according to the first embodiment of the present application is reproduced. ..

The flow of the display image generation process for displaying the thumbnail image group and instructing the cueing playback according to the selection of the thumbnail image is started when the playback control unit 67 starts playing the moving distance holding video, and the playback control unit 67 It is repeatedly executed until the playback of the moving distance holding video is completed. For example, the reproduction of the moving image by the reproduction control unit 67 can be started by the touch operation to the reproduction button PL as described above.

When the playback control unit 67 starts playing the moving distance holding video, the display image generation unit 66 reads out the basic thumbnail image group of the landscape video recorded in the storage unit 50 as a display image generation step, and displays the basic thumbnail image group. It is displayed on the basic thumbnail image string SR of the touch panel 17 (step S301).

Next, as a selection acceptance step, the display image generation unit 66 determines whether any of the basic thumbnail images has been selected on the touch panel 17 (step S302).

When it is determined on the touch panel 17 that any of the basic thumbnail images has been selected (step S302: Yes), the display image generation unit 66 is adjacent to the selected basic thumbnail image in the forward direction as a display image generation step. Read the interpolated thumbnail image group generated with the displayed basic thumbnail image. Further, the read interpolated thumbnail image group is expanded and displayed as an interpolated thumbnail image string ISR at a horizontal position corresponding to the selected basic thumbnail image on the display 15 (step S303).

If it is not determined on the touch panel 17 that any of the basic thumbnail images has been selected (step S302: No), the display image generation unit 66 repeatedly executes the process of step S302.

Next, the display image generation unit 66 receives either the basic thumbnail image selected in step S302 or the interpolated thumbnail image displayed in the interpolated thumbnail image string ISR while the interpolated thumbnail image string ISR is displayed in step S303. Is selected (step S304).

When it is determined that either the basic thumbnail image or the interpolated thumbnail image is selected (step S304: Yes), the display image generation unit 66 starts from the playback position corresponding to the basic thumbnail image or the interpolated thumbnail image selected in step S304. An instruction to perform cue reproduction is issued (step S305). When this cueing instruction is issued, the reproduction control unit 67 performs cueing reproduction from the frame image corresponding to the selected thumbnail image.

After that, the reproduction control unit 67 hides and hides the interpolated thumbnail image sequence ISR displayed on the display 15 in step S303 (step S306).

Further, in step S304, when it is not determined that either the basic thumbnail image selected in step S302 or the interpolated thumbnail image displayed in the interpolated thumbnail image string ISR is selected (step S304: No), the playback control unit 67 Determines whether a portion other than the basic thumbnail image or the interpolated thumbnail image is selected (step S307).

When it is determined that a part other than the basic thumbnail image or the interpolated thumbnail image is selected (step S307: Yes), the playback control unit 67 executes the process of step S306 to hide and display the interpolated thumbnail image string ISR. I will not let you.

After executing the process of step S306, a series of flows for displaying the thumbnail image group of the reproduction control unit 67 ends, and when the moving distance holding video is being reproduced, the process of step S301 is repeatedly executed.

By the above processing, by displaying the interpolated thumbnail image string ISR during the reproduction of the captured landscape moving image, it is possible to efficiently search for a desired scene from the landscape moving image.

In addition, steps S301 and S303 function as display image generation steps.

In other words, it is an image generation method in which the image generation device 10 generates a thumbnail image, in which a moving image taken by an in-vehicle camera CM is acquired, and the moving image is given to the moving image and is given to the moving image at a first predetermined interval of an index indicating the progress of the moving image. Multiple basic thumbnail images extracted for each are generated, and a plurality of interpolated thumbnails extracted at the second predetermined interval of the index from the moving images between the thumbnail images adjacent to each other among the plurality of basic thumbnail images. A thumbnail image generation step for generating an image, a display image generation step for generating a display image in which a plurality of basic thumbnail images are arranged in the order of progress indicated by the index in the first arrangement direction, and selection for each of the plurality of basic thumbnail images. The display image generation step has a selection acceptance step of accepting the above, and the display image generation step is a basic thumbnail of the plurality of interpolated thumbnail images when any of the basic thumbnail images of the plurality of basic thumbnail images is selected by the touch panel 17. A second that crosses the thumbnail image extracted from the moving image between the image and the other basic thumbnail images adjacent thereto with the first arrangement direction in the order of progression indicated by the index, following the selected basic thumbnail image. Generates images arranged in the arrangement direction of.

In the first embodiment, the method of displaying the interpolated thumbnail image group on the display 15 as the interpolated thumbnail image string ISR has been described, but the method of displaying the interpolated thumbnail image group is not limited to this.

FIG. 11 is a diagram showing another example of the operation reception display displayed on the touch panel 17 of the image generation device 10 according to the first embodiment of the present application.

At the start of playback of the moving distance holding video, the basic thumbnail images generated by the thumbnail generation unit 65 are displayed side by side in the order of playback time on the basic thumbnail image string SR of the touch panel 17.

Further, the display at the lower left of each of the basic thumbnail image and the interpolated thumbnail image indicates the reproduction time and the moving distance indicated by each thumbnail image. The display on the upper right of each of the basic thumbnail images indicates the number of interpolated thumbnail images corresponding to each of the basic thumbnail images.

Further, for example, the basic thumbnail image SM3 of the basic thumbnail image string SR may be selected with a finger and scrolled downward to display the interpolated thumbnail image SMi31 in order in the upward direction of the basic thumbnail image SM3.

As a result, it is not necessary to configure the display 15 with a touch panel display, and it is possible to obtain the same effect as that of the first embodiment.

In the first embodiment, up to this point, an example in which a basic thumbnail image and an interpolated thumbnail image are generated at predetermined elapsed times, and a basic thumbnail image is generated at predetermined time intervals, and an interpolated thumbnail image is used as a basic thumbnail image. Although an example of generating for each predetermined movement distance between the two is shown, the combination of the extraction criteria of the basic thumbnail image and the interpolated thumbnail image is not limited to these. For example, a basic thumbnail image and an interpolated thumbnail image may be generated for each predetermined movement distance, a basic thumbnail image may be generated for each predetermined movement distance, and an interpolated thumbnail image may be generated for each predetermined time between the basic thumbnail images. May be generated. Further, any combination may be specified by the user.

In the first embodiment, a method of displaying an interpolated thumbnail image group corresponding to the selected basic thumbnail image by selecting one of the basic thumbnail images to be displayed in the basic thumbnail image string SR has been described. However, the display of the interpolated thumbnail image group may be automatically displayed.

FIG. 12 is a view of the rear door of the automobile M equipped with the image generator 10A according to the second embodiment of the present application as viewed from the rear seat of the automobile M.

The configuration of the second embodiment is the same as that of the first embodiment except that the interpolated thumbnail image string ISR on which the interpolated thumbnail image group is displayed is displayed on the touch panel 19. Further, the method of adding the moving distance information to the landscape moving image and the method of generating the basic thumbnail image group and the interpolated thumbnail image group are the same as those in the first embodiment. Further, regarding the method of displaying the basic thumbnail image group and the interpolated thumbnail image group, the second embodiment is different in that the interpolated thumbnail image group is automatically displayed.

The touch panel 19 is a touch panel display composed of a display 19A such as a liquid crystal panel mounted inside the door panel and a touch pad 19B, and a basic thumbnail image group of a moving image being played on the display 15 is displayed. The thumbnail image string SR and the interpolated thumbnail image group are displayed. The interpolated thumbnail image string ISR is displayed.

FIG. 13 is a diagram showing an example of an operation reception display displayed on the touch panel 19 of the image generation device 10A according to the second embodiment of the present application.

As shown in FIG. 13, in the operation reception display, for example, the basic thumbnail image sequence SR in which the basic thumbnail images SM1 to SM6 generated by the thumbnail generation unit 65 from the landscape video being played on the display 15 are arranged in the order of playback time. Is displayed. Further, an interpolated thumbnail image string ISR in which the interpolated thumbnail images generated by the thumbnail generation unit 65 are arranged vertically in the order of playback time at positions corresponding to the basic thumbnail images SM1 to SM6 is displayed above the basic thumbnail image string SR. NS.

Further, the interpolated thumbnail image group included in the interpolated thumbnail image string ISR is scrolled in the horizontal direction according to the scrolling of the basic thumbnail image string SR due to the passage of the reproduction time of the landscape moving image.

Further, for example, as shown in FIG. 13, when the reproduction position is the reproduction time corresponding to the interpolated thumbnail image, the interpolated thumbnail image may be highlighted.

As described above, the display image generation unit 66 displays a plurality of basic thumbnail images side by side in the order of elapsed time or moving distance along the first arrangement direction, and is above or below each of the plurality of basic thumbnail images. A display image in which each of the plurality of interpolated thumbnail images included in the plurality of moving image portions corresponding to each of the plurality of basic thumbnail images is arranged in the order of elapsed time or moving distance in the second arrangement direction perpendicular to the first arrangement direction. To generate.

Further, as described above, the image generation device 10 acquires the landscape moving image taken by the in-vehicle camera CM, and from the landscape moving image, the first predetermined interval of either the elapsed time in the landscape moving image or the moving distance of the in-vehicle camera CM. A plurality of basic thumbnail images extracted for each are generated, and from a landscape video between thumbnail images adjacent to each other among the plurality of basic thumbnail images, at every second predetermined interval of either elapsed time or travel distance. A thumbnail generation unit 65 that generates a plurality of interpolated thumbnail images extracted in the above, and a plurality of basic thumbnail images are arranged and displayed in the first arrangement direction in the order of elapsed time or moving distance, and the plurality of interpolated thumbnail images are displayed. , A display image generation unit 66 that generates display images arranged in the order of elapsed time or moving distance following a plurality of basic thumbnail images in a second arrangement direction intersecting the first arrangement direction.

Further, the image generation device 10 has a reproduction control unit 67 that controls reproduction of a landscape moving image and outputs an image, and the plurality of basic thumbnail images or a plurality of interpolated thumbnail images in the display image generated by the display image generation unit 66. The reproduction control unit 67 further includes an operation reception unit that receives an instruction operation including a selection operation for each of the above, and the reproduction control unit 67 performs reproduction control based on the instruction operation.

Further, the image generation device 10 acquires a landscape moving image taken by the in-vehicle camera CM, and first arranges a plurality of basic thumbnail images extracted from the landscape moving image for each elapsed time or moving distance of the first predetermined interval. A plurality of interpolated thumbnail images extracted from each of the landscape videos between the basic thumbnail images adjacent to each other among the plurality of basic thumbnail images are arranged and displayed in the order of elapsed time or moving distance in the direction, and are displayed in the first array. A touch panel that accepts selection for each of the plurality of basic thumbnail images and the plurality of interpolated thumbnail images by arranging and displaying each of the plurality of basic thumbnail images in the second arrangement direction intersecting the direction in the order of elapsed time or moving distance. An image display device including a display 15 and a playback control unit 67 that reproduces a landscape moving image based on selections for each of a plurality of basic thumbnail images and a plurality of interpolated thumbnail images by the touch panel 17 and the display 15.

FIG. 14 is a flowchart FC4 of a display image generation process in which the display image generation unit 66 displays a thumbnail image group on the touch panel 19 when the landscape moving image of the image generation device 10A according to the second embodiment of the present application is reproduced.

The display image generation process of the second embodiment is basically the same process as the display process of the thumbnail image group of the first embodiment, but since the thumbnail image group is automatically displayed, the basic thumbnail image is selected and then interpolated. Processing to display the thumbnail image string ISR (steps S302 and S303 in FIG. 10), processing to select one of the thumbnail images again after the interpolated thumbnail image string ISR is displayed (steps S304 and S307 in FIG. 10) and display. The process of hiding the interpolated thumbnail image string ISR (step S306 in FIG. 10) is not executed. Further, in step S403, it is different in that cueing reproduction is executed when any of the thumbnail images is selected. The other processes of steps S401, S402 and S404 correspond to steps S301, S303 and S305 shown in FIG. 10 of the first embodiment, respectively, and the processing contents are the same, so the description thereof will be omitted.

Further, the flow of the display image generation processing in the second embodiment is started when the playback control unit 67 starts playing the moving distance holding video, and the playback control unit 67 finishes playing the moving distance holding video, as in the first embodiment. It is executed repeatedly until it is done. For example, the reproduction of the moving image by the reproduction control unit 67 can be started by the touch operation to the reproduction button PL as described above.

After the thumbnail image group is displayed in the basic thumbnail image string SR and the interpolated thumbnail image string ISR in steps S401 and S402, the display image generation unit 66 is displayed in the basic thumbnail image string SR or the interpolated thumbnail image string ISR. It is determined whether any of the thumbnail images is selected (step S403).

When it is determined that either the basic thumbnail image string SR or the thumbnail image displayed in the interpolated thumbnail image string ISR is selected (step S403: Yes), the display image generation unit 66 executes the process of step S404. An instruction is issued to perform cue playback from the playback position of the selected basic thumbnail image or interpolated thumbnail image.

When it is not determined that either the basic thumbnail image string SR or the thumbnail image displayed in the interpolated thumbnail image string ISR is selected (step S403: No), the display image generation unit 66 executes playback control of the moving distance holding video. The process of steps S401 to S403 is repeatedly executed while issuing the instruction of.

By the above processing, the basic thumbnail image sequence SR and the interpolated thumbnail image sequence ISR are displayed during the reproduction of the captured landscape moving image, so that a desired scene can be efficiently searched from the landscape moving image.

In other words, it is an image generation method for generating a thumbnail image by the image generation device 10A, which is a landscape video acquisition step of acquiring a landscape video taken by an in-vehicle camera CM, and an elapsed time or in-vehicle in the landscape video from the landscape video. A plurality of basic thumbnail images extracted at any first predetermined interval of the movement distance of the camera CM are generated, and the transition is made from the landscape video between the thumbnail images adjacent to each other among the plurality of basic thumbnail images. A thumbnail image generation step that generates a plurality of interpolated thumbnail images extracted at a second predetermined interval of either time or travel distance, and a first arrangement direction of a plurality of basic thumbnail images in the order of elapsed time or travel distance. Generates a display image in which a plurality of interpolated thumbnail images are arranged in the order of elapsed time or moving distance following a plurality of basic thumbnail images in a second arrangement direction that intersects the first arrangement direction. Includes a display image generation step to be performed.

In the above-described first and second embodiments, the moving distance is calculated based on the traveling speed or the shooting position of the automobile M with respect to the landscape moving image taken from the in-vehicle camera CM mounted on the automobile M, and the moving distance or the landscape is calculated. The image generation device in which the thumbnail generation unit 65 generates the basic thumbnail image group and the interpolated thumbnail image according to the elapsed time in the moving image has been described. However, the image generator according to the present invention is not limited to landscape moving images from automobiles.

For example, it may be possible to generate a moving image in which information on the moving distance is added to a moving image taken by a smartphone, a handy camera, or the like. Further, the captured moving distance holding moving image may be recorded in an external recording device such as a USB memory or HDD, and processed and played back by a device such as a PC.

Further, in the first and second embodiments, an image generation device that generates one moving image data in which a moving distance at the time of shooting the scenery moving image of the automobile M is added to the scenery moving image and generates a thumbnail image group of the moving image data. Was explained. However, the method of generating the thumbnail image group is not limited to generating one moving image data. For example, a landscape moving image to which information on the moving distance is not given and information data indicating the elapsed time and the moving distance with respect to the elapsed time are generated from the landscape moving image. The thumbnail generation unit 65 may determine a frame image to be extracted as a thumbnail image from the information data, and extract the frame image as a thumbnail image from the landscape moving image. This makes it possible to reduce the file data capacity of landscape moving images.

Further, in the first and second embodiments, as the display format of the thumbnail image group, an example in which both the basic thumbnail image group and the interpolated thumbnail image group are generated at a predetermined time interval and the basic thumbnail image group are generated and interpolated at a predetermined time interval. An example in which thumbnail image groups are generated at predetermined movement distance intervals has been described. However, the display format of the thumbnail image group according to the present invention is not limited to this. For example, both the basic thumbnail image group and the interpolated thumbnail image group may be generated at a predetermined movement distance interval, or the basic thumbnail image group may be generated at a predetermined movement distance interval and the interpolated thumbnail image group may be generated at a predetermined elapsed time interval. You may.

Further, in the first and second embodiments, the method of calculating the moving distance from the speed and position information of the automobile M has been described, but the method of calculating the moving distance is not limited to this. For example, the moving distance may be calculated from the position information by the GPS receiver GP of the automobile M and the clock or the like built in the automobile M or the image generator. This makes it possible to calculate the moving distance without using the speed sensor SS.

That is, the movement distance from the start of shooting is given to each of the plurality of frame images constituting the shot image, and the basic thumbnail image group and the interpolated thumbnail image group are variably generated according to the movement distance or the elapsed time. I hope I can.

10 Image generator 15 Display 17, 19 Touch panel 20 Input unit 30 Output unit 40 Operation input unit 50 Storage unit 60 Video control unit 63 Video generation unit 65 Thumbnail generation unit 66 Display image generation unit 67 Playback control unit

Claims (10)

The acquisition unit that acquires the moving image taken by the image pickup device, and
From the moving image, a plurality of first thumbnail images added to the moving image and extracted at first predetermined intervals of an index indicating the progress of the moving image are generated, and among the plurality of first thumbnail images, each other. A thumbnail image generation unit that generates a plurality of second thumbnail images extracted at second predetermined intervals of the index from the moving image between adjacent thumbnail images.
A display image generation unit that generates a display image in which the plurality of first thumbnail images are arranged in the first arrangement direction in the order of progression indicated by the index.
A selection reception unit that accepts selections for each of the plurality of first thumbnail images, and
Have,
The display image generation unit is the first of the plurality of second thumbnail images when the first thumbnail image of any of the plurality of first thumbnail images is selected by the selection reception unit. The thumbnail images extracted from the moving image between the thumbnail image of 1 and the other first thumbnail image adjacent thereto are followed by the selected first thumbnail image and in the order of progress indicated by the index. An image generation device characterized in that an image arranged in a second arrangement direction intersecting with the first arrangement direction is generated. The thumbnail image generation unit divides the moving image into a plurality of moving image portions at each first predetermined interval, extracts the first frame of each of the plurality of moving image parts, and obtains the plurality of first thumbnail images. A frame image is extracted from each of the plurality of moving image portions at each of the second predetermined intervals smaller than the first predetermined interval to generate the plurality of second thumbnail images.
The first predetermined interval and the second predetermined interval are characterized in that they are intervals based on either the elapsed time in the moving image or the moving distance of the imaging device as an index indicating the progress of the moving image. The image generator according to claim 1. A moving distance acquisition unit for acquiring information on the moving distance of the imaging device since the start of shooting the moving image is provided.
The image generation according to claim 1, wherein either the first predetermined interval or the second predetermined interval is an interval based on the moving distance as an index indicating the progress of the moving image. Device. When one of the plurality of first thumbnail images is selected by the selection reception unit, the display image generation unit displays at least a part of the plurality of second thumbnail images. The image generation device according to any one of claims 1 to 3, wherein the first thumbnail image to be created is generated in a manner that can be identified from the other first thumbnail image. The display image generation unit arranges the plurality of first thumbnail images in the order of elapsed time in the moving image or the moving distance of the imaging device as an index indicating the progress of the moving image along the first arrangement direction. While displaying, each of the second thumbnail images included in the plurality of moving image portions corresponding to each of the plurality of first thumbnail images is displayed above or below each of the plurality of first thumbnail images. The invention according to any one of claims 1 to 4, wherein the display images are arranged in the second arrangement direction perpendicular to the first arrangement direction in the order of elapsed time or the movement distance. Image generator. It has a playback control unit that controls the playback of the moving image and outputs the video.
The selection reception unit receives image selection for each of the plurality of first thumbnail images or the plurality of second thumbnail images in the display image generated by the display image generation unit, and the reproduction control unit receives the image selection. The image generation device according to any one of claims 1 to 5, wherein the moving image is reproduced from a position corresponding to the selected first thumbnail image or the second thumbnail image. An acquisition unit that acquires videos taken from a moving body,
A thumbnail image display unit that arranges and displays a plurality of first thumbnail images extracted from the moving image for each elapsed time of a first predetermined interval or for each moving distance of the moving body in the first array direction.
It has a selection reception unit that accepts selections for each of the plurality of first thumbnail images.
When the selection reception unit receives a selection for each of the plurality of first thumbnail images, the thumbnail image display unit receives the selected first thumbnail image and another first thumbnail image adjacent thereto. The second thumbnail image extracted from the moving image between and the first thumbnail image selected in the second arrangement direction intersecting with the first arrangement direction is followed by the elapsed time or the movement. An image display device characterized by arranging and displaying in order of distance. An image generation method in which an image generator generates thumbnail images.
The video acquisition step to acquire the video captured by the image pickup device, and
From the moving image, a plurality of first thumbnail images added to the moving image and extracted at first predetermined intervals of an index indicating the progress of the moving image are generated, and among the plurality of first thumbnail images, each other. A thumbnail image generation step of generating a plurality of second thumbnail images extracted at second predetermined intervals of the index from the moving image between adjacent thumbnail images.
A display image generation step of generating a display image in which the plurality of first thumbnail images are arranged in the first arrangement direction in the order of progression indicated by the index, and a display image generation step.
A selection acceptance step that accepts selections for each of the plurality of first thumbnail images, and
Have,
In the display image generation step, when the first thumbnail image of any of the plurality of first thumbnail images is selected by the selection reception unit, the first of the plurality of second thumbnail images is selected. The thumbnail images extracted from the moving image between the thumbnail image of 1 and the other first thumbnail image adjacent thereto are followed by the selected first thumbnail image and in the order of progress indicated by the index. An image generation method characterized in that an image arranged in a second arrangement direction intersecting with the first arrangement direction is generated. An image generation program executed by a computer, which is executed by the computer.
The video acquisition step to acquire the video captured by the image pickup device, and
From the moving image, a plurality of first thumbnail images added to the moving image and extracted at first predetermined intervals of an index indicating the progress of the moving image are generated, and among the plurality of first thumbnail images, each other. A thumbnail image generation step of generating a plurality of second thumbnail images extracted at second predetermined intervals of the index from the moving image between adjacent thumbnail images.
A display image generation step of generating a display image in which the plurality of first thumbnail images are arranged in the first arrangement direction in the order of progression indicated by the index, and a display image generation step.
A selection acceptance step that accepts selections for each of the plurality of first thumbnail images, and
To run,
In the display image generation step, when the first thumbnail image of any of the plurality of first thumbnail images is selected by the selection reception unit, the first of the plurality of second thumbnail images is selected. The thumbnail images extracted from the moving image between the thumbnail image of 1 and the other first thumbnail image adjacent thereto are followed by the selected first thumbnail image and in the order of progress indicated by the index. An image display program characterized by generating images arranged in a second arrangement direction that intersects the first arrangement direction. A computer-readable recording medium comprising storing the image display program according to claim 9.

Images