JP2021129215A - Video reproduction apparatus, video generation apparatus, video generation method, and video generation program - Google Patents

Abstract

To provide a video reproduction apparatus which can efficiently retrieve a desired scene from a landscape video captured from a moving body, and a reproduction method.SOLUTION: A video reproduction apparatus 10 includes: an acquisition unit which acquires a landscape video captured from a moving body; a video generation unit 63 which generates an information-added video formed by adding importance information based on information on an imaging status in each point of time in an imaging period in the landscape video, to the landscape video; a video recording unit which records the information-added video generated in the video generation unit 63; and a reproduction control unit 67 which reads the information-added video from the video recording unit and performs reproduction control including reproduction and fast-forward/fast-rewind control of the landscape video for video output. The reproduction control unit 67 performs reproduction control including reproduction and fast-forward/fast-rewind of the landscape video at a speed based on the importance information.SELECTED DRAWING: Figure 4

Description

The present invention relates to a moving image reproducing device for reproducing moving images and a method for reproducing the moving image.

In recent years, a playback device has become widespread in which a landscape moving image can be taken with a shooting device provided on a moving body such as a vehicle, and a scene shot from the vehicle can be viewed later.

For example, in Patent Document 1, a time bar corresponding to the shooting time from the start to the end of shooting of a landscape video shot by an in-vehicle camera mounted on a moving body and a position on the time bar can be instructed by a user. An instruction display image including the slider shown and the landmark icon indicating the landmark existing along the traveling route when the landscape video was shot at the corresponding position on the time bar is displayed on the display, and the landscape video is further displayed. Among them, a display system for a vehicle that displays a landscape moving image associated with a shooting time corresponding to the position of a slider on the time bar on a display is disclosed.

Further, for example, in Patent Document 2, a still image is extracted from moving image data to generate a thumbnail image, and the thumbnail image is displayed side by side at regular time intervals on a timeline to record a desired scene in the moving image. A moving image search / viewing device that makes it possible to easily find a moving image is disclosed.

Japanese Unexamined Patent Publication No. 2013-32949 Japanese Unexamined Patent Publication No. 2002-281432

However, in the display system of Patent Document 1, when trying to find a part in which a desired landscape is recorded in a moving object in which an external landscape is photographed, the slider position is changed or a landmark icon is displayed. Since it is necessary to search for a desired scene while visually recognizing the image displayed on the display while playing back from the vicinity, it may take time to reach the desired scene if the shooting time of the moving image is long.

Further, in the moving image search / viewing device of Patent Document 2, for example, when the moving speed of the moving body is slow and the change of the landscape per predetermined time is small, the thumbnail image at regular time intervals has the same landscape or image. Since a plurality of images are lined up, it may take some time to find a thumbnail image showing a desired landscape.

The present invention has been made in view of the above points, and provides a moving image reproduction device capable of efficiently searching for a desired scene from a landscape moving image taken from a moving body, and a reproduction method thereof. With the goal.

The invention according to claim 1 provides an acquisition unit that acquires a landscape moving image shot from a moving body, and importance information based on information on a shooting situation at each time point during the shooting period of the scenery moving object. A video generation unit that generates the information-holding video assigned to the video generator, a video recording unit that records the information-holding video generated by the video generation unit, and a landscape video that reads the information-holding video from the video recording unit. It has a playback control unit that performs playback, fast-forward, or fast-rewind control and outputs an image, and the playback control unit converts playback control including playback, fast-forward, or fast-rewind of the landscape moving image into the importance information. It is characterized by performing at a speed based on.

The invention according to claim 10 is an acquisition unit for acquiring a landscape moving image shot from a moving body, and an importance of acquiring importance information based on information on a shooting situation at each time point during the shooting period of the scenery moving body. It is characterized by having an information acquisition unit and a reproduction control unit that reproduces the landscape moving image at a speed based on the importance information.

The invention according to claim 11 provides an acquisition unit that acquires a landscape moving image shot from a moving body, and importance information based on information on a shooting situation at each point during the shooting period of the scenery moving object. It is characterized by having a moving image generation unit that generates an information-holding moving image assigned to the above.

The invention according to claim 12 is a moving image generation method in which a moving image reproducing device gives importance information to a landscape moving image, an acquisition step in which an information acquisition unit acquires information on a shooting status of the landscape moving image, and moving image generation. The unit is characterized in that it has an information-holding video generation step of generating an information-holding video attached to the landscape video with importance information based on information on the shooting status at each time point during the shooting period of the landscape video. There is.

The invention according to claim 13 is a moving image generation program executed by a moving image generating device provided in a computer, wherein an information acquisition unit acquires information on a shooting status of a landscape moving image on the computer, and moving image generation. The feature is that the department executes the information-holding video generation step of generating the information-holding video added to the landscape video with importance information based on the information about the shooting situation at each time point during the shooting period of the landscape video. It is supposed to be.

It is a perspective view of the automobile M equipped with the moving image reproduction apparatus 10 which concerns on embodiment of this application. It is a figure seen from the inside of the automobile M equipped with the moving image reproduction apparatus 10 which concerns on embodiment of this application. It is a figure which shows an example of the operation reception part 17 of the moving image reproduction apparatus 10 which concerns on embodiment of this application. It is a figure which shows the structure of the moving image reproduction apparatus 10 which concerns on embodiment of this application. FIG. 2 is a flowchart FC2 for reproducing an information-holding moving image as a reproduction control step of the moving image reproducing device 10 according to the embodiment of the present application. This is an example of the elapsed time from the start of shooting of the landscape moving image of the moving image playback device 10 according to the embodiment of the present application, the frame numbers of a plurality of frames at predetermined times, the basic time codes of the plurality of frames, and the virtual time code. .. It is a figure which shows the relationship between the frame number of a plurality of frames constituting the information-holding moving image in the example of the information-holding moving image shown in FIG. 6 and the virtual time code given to each frame number. FIG. 1 is a flowchart FC1 for generating an information-holding moving image by adding a virtual time code to a landscape moving image taken by an in-vehicle camera CM as an information-holding moving image generation step of the moving image reproducing device 10 according to the embodiment of the present application. It is a figure which showed the display example of the touch panel 17 at the time of fast-forwarding execution of the information holding moving image shown in FIGS. 6 and 7. It is a figure which showed the display example of the touch panel 17 at the time of fast-forwarding execution of the information holding moving image shown in FIGS. 6 and 7.

Examples of the present invention will be described in detail below.

FIG. 1 is a perspective view of an automobile M equipped with the moving image reproducing device 10 according to the embodiment of the present application.

The moving image playback device 10 is a moving image playback device that shoots a landscape movie around the car M with an in-vehicle camera mounted on the car M and reproduces the shot scenery movie. In the following description, a configuration capable of displaying a landscape moving image in the window of the automobile M will be described.

The moving image reproducing device 10 is mounted in, for example, the dashboard of the automobile M. The moving image reproduction device 10 is configured to be capable of reproducing and outputting a moving image acquired from the outside or held by itself.

The in-vehicle camera CM as the acquisition unit is mounted on, for example, the roof portion or the door panel portion of the automobile M, and is configured to be capable of photographing the scenery in front, side, or rear of the automobile M. The in-vehicle camera CM is communicably connected to the moving image playback device 10, and is configured to be able to transmit the captured still image or moving image to the moving image playback device.

The display 15 as a moving image display unit is, for example, a display composed of a transparent panel such as a transparent liquid crystal display or an organic EL display, attached to the window WD of the automobile M, and capable of displaying an image visible from the inside of the automobile. be. The display 15 is communicatively connected to the moving image reproducing device 10, and is configured to be able to display a still image or a moving image output from the moving image reproducing device 10.

For example, the display 15 is configured so that landscape moving images of the front, side, or rear of the automobile M taken by the in-vehicle camera CM can be displayed in the front window, the door window, and the rear window, respectively. The display 15 may be, for example, a display of a car navigation system installed inside the automobile M.

FIG. 2 is a view of the rear door of the automobile M equipped with the moving image reproducing device 10 according to the embodiment of the present application as viewed from the rear seat of the automobile M.

As described above, the display 15 is composed of a transparent panel such as a transparent liquid crystal display or an organic EL display, and is attached to the window WD of the automobile M. Therefore, for example, when the moving image is not displayed on the display 15, it is possible to see the outside scenery from the inside of the automobile M through the display 15.

The touch panel 17 as an operation receiving unit is a touch panel display attached to the inside of the door panel. For example, the touch panel 17 is a touch panel display including a display 17a such as a liquid crystal panel and a touch pad 17b attached to the display surface of the display and accepting an operation with a human finger or the like.

The touch panel 17 is communicably connected to the moving image playback device 10, and can display the video output from the moving image playback device 10. Further, the touch panel 17 can accept the operation of playing, fast-forwarding, or fast-rewinding the moving image displayed on the display 15, and can transmit an instruction based on the acceptance of the operation to the moving image reproducing device 10.

Specifically, for example, on the touch panel 17, thumbnail images of the moving image being reproduced on the display 15 at predetermined time intervals are displayed side by side in the order of reproduction time.

FIG. 3 is a diagram showing an example of an operation reception display displayed on the touch panel 17 of the moving image playback device 10 according to the embodiment of the present application.

The operation reception display is, for example, a thumbnail image string SR in which thumbnail images SM1 to SM6 of a landscape video being played on the display 15 at predetermined time intervals are arranged in the order of playback time, and a button for instructing to switch playback or pause. It includes a PL, a button FW for instructing a fast-forward operation, a button RW for instructing a fast-rewind operation, a bar BA indicating the current playback time position of a landscape moving image, and a time display unit TM. Further, the thumbnail images included in the thumbnail image string SR are displayed while scrolling from right to left according to the passage of the playback time of the landscape moving image.

In this embodiment, the reproduction means that the captured landscape moving image is played back in the forward direction at a basic speed determined based on the time code given to the moving image. Further, fast-forwarding refers to playing back a captured landscape moving image in a forward direction at a speed faster than the above-mentioned basic speed. In addition, fast-rewinding refers to playing back a captured landscape movie in the opposite direction at a speed faster than the basic speed.

FIG. 4 is a functional block diagram of the moving image playback device 10 according to the embodiment of the present application. The moving image playback device 10 is, for example, a device in which the input unit 20, the output unit 30, the operation input unit 40, the storage unit 50, and the moving image control unit 60 cooperate with each other via the system bus 13.

The input unit 20 is an interface for acquiring data from an external device of the moving image playback device 10. The input unit 20 is connected to an in-vehicle camera CM provided in the automobile M. For example, the moving image reproduction device 10 can capture a landscape image from the in-vehicle camera CM via the input unit 20.

Further, the input unit 20 is a speed sensor SS that acquires the traveling speed of the automobile and a global positioning system (GPS) that is one of the global navigation satellite systems (GNSS: Global Navigation Satellite System) that acquires the position information of the automobile. : Global Positioning System) is connected to the GPS receiver GP that can receive information. That is, the moving image reproducing device 10 is configured so that the speed of the automobile M can be acquired from the speed sensor SS and GPS information can be acquired from the GPS receiver GP.

The output unit 30 is an interface that outputs data to an external device of the moving image playback device 10. The output unit 30 is connected to the display 17a of the display 15 and the touch panel 17 described above, and can output an image to these. Specifically, for example, the moving image reproduction device 10 outputs a moving image to the display 15 via an output unit, and outputs an operation image as an operation reception display for a reproduction operation to the display 17a.

The operation input unit 40 is an interface for acquiring an operation input signal from an external device. The operation input unit 40 is connected to the touch pad 17b, and can acquire a signal indicating an operation received by the touch pad 17b with respect to the operation image.

The storage unit 50 is composed of, for example, a hard disk device, an SSD (solid state drive), a flash memory, or the like, and stores various programs related to the generation of thumbnail images, the reproduction of moving images, and the like in the moving image reproduction device 10. The various programs may be acquired from another server device or the like via a network (not shown), or may be recorded on a recording medium and read via various drive devices. May be good.

Various programs stored in the storage unit 50 can be transmitted via a network, and can be recorded on a computer-readable recording medium and transferred. Further, in the storage unit 50, data captured from the outside of the moving image playback device 10, for example, landscape moving image data taken from the in-vehicle camera CM, landscape moving image captured from the speed sensor SS and the GPS receiver GP, is taken. The speed and position information of the automobile M can be stored. Further, the storage unit 50 can store the data generated inside the moving image reproducing device 10.

The moving image control unit 60 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and functions as a computer. In the moving image control unit 60, the CPU realizes various functions by reading and executing various programs stored in the ROM and the storage unit 50.

The moving image control unit 60 has a moving image generation unit 63 that adds information to the landscape moving image acquired via the input unit 20 to generate an information-holding moving image for playback. The moving image generation unit 63 reads out the landscape moving image recorded in the storage unit 50 and the speed or position information data of the automobile M at each time when the scenery moving image is taken. Information based on the speed or position information of the automobile M at each time when each of the plurality of frames is photographed is added to each of the plurality of frame images constituting the read landscape moving image. For example, the moving image generation unit 63 determines the importance of each frame image according to the traveling speed of the automobile M, and adds importance information, which is information based on the importance, to the landscape moving image.

The importance is taken by, for example, the traveling speed of the automobile M, the shooting position, the appearance of the landscape associated with the position on the map information corresponding to the shooting position, or the in-vehicle camera CM based on the traveling direction of the automobile M. Judgment is made based on the shooting direction.

For example, when the automobile M is traveling at a low speed or is stopped, the surrounding landscape seen from the automobile hardly changes. Therefore, when determining the importance with respect to the traveling speed of the automobile, the moving image generation unit 63 determines that the importance of the landscape while the automobile M is traveling at a low speed or is stopped is low. On the contrary, when the automobile is traveling at high speed, the landscape moving image changes significantly in a short time, so that the importance is judged to be high.

Further, the moving image generation unit 63 determines that the importance is high when, for example, the position information of the automobile by GPS and the map database recorded in advance are linked and the automobile is in a position having a high highlight rank on the map database. On the contrary, in the landscape moving image in which the automobile M is at a position having a low highlight rank on the map database, the moving image generation unit 63 determines that the importance is low.

Further, the importance based on the location information may be determined without interlocking with the map database. For example, in a mountainous area with a high altitude, the scenery and the like may be highly spectacular. Therefore, it may be determined that the importance is high when the positioning in the range of a predetermined position is detected from the positioning information such as GPS.

Further, in the landscape moving image in which the automobile M is traveling, the landscape moving image in which the side in the traveling direction is photographed changes greatly in a short time with respect to the landscape moving image in which the front or the rear of the traveling direction of the automobile M is photographed. Therefore, the moving image generation unit 63 determines that the landscape moving image on the side in the traveling direction is more important than the scenery moving image in front of or behind the traveling direction.

Further, if the GPS receiver GP cannot acquire GPS information for a predetermined time or longer, it is highly possible that the automobile M is traveling in a structure such as a tunnel. In such a case, the landscape video is displayed. The monotonous content will continue. Therefore, the moving image generation unit 63 determines that the importance of the scenery moving image when the GPS information is not received is low with respect to the scenery moving image when the GPS information is received.

The above-mentioned importance determination conditions can be set arbitrarily. Further, the determination condition of the importance is the traveling speed of the automobile M, the shooting position, the appearance of the landscape associated with the position on the map information corresponding to the shooting position, or the shooting taken by the photographing unit with respect to the traveling direction of the automobile M. The importance is determined not only in one direction but also by a plurality of determination conditions.

In other words, the moving image generation unit 63 uses the importance as the moving speed of the automobile M, the shooting position, the appearance of the landscape associated with the position on the map information corresponding to the shooting position, and the moving direction of the automobile M. Judgment is made based on the shooting direction or the radio wave reception status of the satellite system.

The importance is not limited to the speed and position when shooting a moving image. For example, each of the plurality of frame images constituting the moving image may be compared, and the frame having a large change in the image between the frame images may be determined as having a high importance.

Further, for example, the moving image generation unit 63 assigns a virtual time code indicating a time passage different from a time code (hereinafter, referred to as a basic time code) indicating the actual time passage at the time of shooting the landscape moving image based on the importance. Generate the information-holding video.

The virtual time code is calculated from the average value of the importance for each predetermined time, and is the importance information in the form of reflecting the importance in the virtual passage of time.

Specifically, the moving image generation unit 63 has a time elapsed different from the basic time code for each of the plurality of frame images at each predetermined time of the landscape moving image according to the average value of the importance calculated for each predetermined time. Give a virtual time code including. That is, the moving image generation unit 63 functions as an importance information acquisition unit that generates an importance and a virtual time code from the data of the landscape moving image and the speed or position information of the automobile M when the scenery moving image is taken. In addition, it functions as a moving image generation unit that generates an information-holding moving image by adding a virtual time code as the generated importance information to the landscape moving image.

The virtual time code is set so that the higher the importance, the closer to the time lapse of the basic time code, and the lower the importance, the earlier the time lapse of the basic time code.

Further, the moving image generation unit 63 can arbitrarily set a predetermined time for calculating the average value of the importance.

The information-holding moving image generated by the moving image generation unit 63 is stored in the storage unit 50. The importance information given to the moving image is based on the moving speed of the automobile M, the shooting position, the appearance of the landscape associated with the position on the map information corresponding to the shooting position, and the moving direction of the automobile M. It may include one or more of directions, importance based on satellite system radio reception, and virtual timecode.

The thumbnail generation unit 65 extracts frame images at predetermined time intervals from a plurality of frame images constituting the information-holding moving image recorded in the storage unit 50, and generates thumbnail images to be displayed in the thumbnail display area SR of the touch panel 17. .. Further, it is preferable that the predetermined time is synchronized with the predetermined time when the moving image generation unit 63 calculates the average value of the importance.

The thumbnail image may be the extracted frame image itself or a reduced version of the extracted frame image. Hereinafter, both the process of extracting a frame image from a landscape video and using it as it is as a thumbnail image and the process of reducing the extracted frame image to generate a thumbnail image will be described as simply extracting a thumbnail image. ..

The reproduction control unit 67 reads out the information-retaining moving image recorded in the storage unit 50, and controls the reproduction of the information-retaining moving image in response to an instruction to play, pause, fast-forward, or fast-rewind. For example, in this reproduction control, the reproduction control is performed based on the basic time code, and the fast forward and fast rewind are performed based on the virtual time code.

The reproduction control unit 67 supplies the reproduced information-holding moving image to the display 15 via the output unit 30. Further, the reproduction control unit 67 supplies the operation image corresponding to the reproduced information-holding moving image to the display 17a via the output unit 30. The reproduction control unit 67 acquires the operation signal received by the touch pad 17b and controls the reproduction of the moving image based on the operation signal.

In other words, the video playback device 10 provides the in-vehicle camera CM that acquires the landscape video shot from the automobile M and the importance information based on the information on the shooting status at each time point during the shooting period of the landscape video. A video generation unit 63 that generates the information-holding video assigned to the image, a storage unit 50 that records the information-holding video generated by the video generation unit 63, and a storage unit 50 that reads the information-holding video from the storage unit 50 to play and fast-forward the landscape video. Alternatively, it has a playback control unit 67 that performs fast-rewind control and outputs an image, and the playback control unit 67 performs playback control including playback, fast-forward, or fast-rewind of a landscape moving image at a speed based on importance information. conduct.

Further, the moving image playback device 10 includes a thumbnail generation unit 65 that extracts frame images at predetermined time intervals from a plurality of frame images constituting the information holding video and generates thumbnail images, and thumbnail images in which the thumbnail images are arranged in the order of playback time. The playback control unit 67 further includes a touch panel 17 that displays the column SR and accepts an instruction operation including a selection operation of a thumbnail image included in the thumbnail image column SR or a scroll operation for the thumbnail image column SR, and the playback control unit 67 is based on the instruction operation. Playback control is performed.

In addition, the moving image generation unit 63 sets a series of virtual images based on information on the shooting situation when each of the plurality of frame images is shot for each of the plurality of frame images constituting the shot landscape moving image. A virtual time code, which is a different time code, is added as importance information to generate an information-holding moving image, and the reproduction control unit 67 performs reproduction control based on the virtual time code.

Further, the moving image generation unit 63 determines the importance of each of the plurality of frame images based on the information regarding the shooting situation, and assigns a virtual time code corresponding to the importance to the plurality of frame images.

Further, the moving image generation unit 63 generates a virtual time code so that the lower the importance, the faster the passage of time indicated by the virtual time code in the group of frame images.

FIG. 5 shows a flowchart FC1 for generating an information-holding moving image by adding a virtual time code to a landscape moving image taken by an in-vehicle camera CM in the moving image reproducing device 10 according to the embodiment of the present application. Here, an example of generating an information-holding moving image after capturing the entire landscape moving image will be described. Further, here, an example of setting the importance of the landscape image according to the traveling speed of the automobile M will be described.

First, the moving image generation unit 63 determines in step S101 whether or not the shooting of the landscape moving image is completed (step S101). This determination may be made, for example, by accessing the in-vehicle camera CM and confirming whether or not the moving image shooting is completed. Further, this determination may be made, for example, by whether or not the reception of the moving image data from the in-vehicle camera CM has been completed.

When it is determined that the shooting of the landscape moving image is completed (step S101: Yes), the moving image generation unit 63 is acquired by the speed sensor SS during the shooting period and the scenery moving image recorded in the storage unit 50 as an acquisition step. The traveling speed of the automobile M is read out (step S102).

Next, the moving image generation unit 63 adds the traveling speed information of the automobile M at the time when the frame is taken to each of the plurality of frame images constituting the read landscape moving image (step S103).

The moving image generation unit 63 determines the importance according to the traveling speed of the automobile M in each of the plurality of frames constituting the landscape moving image to which the traveling speed of the automobile M is given, and further assigns the importance to the landscape moving image. (Step S104).

Next, the moving image generation unit 63 divides the landscape moving image in which the importance is given to each of the frames at predetermined time intervals, and calculates the average value of the importance for each predetermined time (step S105).

After that, as an information holding moving image generation step, the moving image generation unit 63 assigns a virtual time code including a time passage different from the basic time code and the time passage according to the average value of the importance for each predetermined time. (Step S106).

After the end of step S106, the moving image generation unit 63 records the landscape moving image to which the virtual time code is attached in the storage unit 50 as an information holding moving image (step S107).

By performing the above processing, an information holding video to which the importance and the virtual time code are given is generated from the landscape video taken by the in-vehicle camera CM and the traveling speed of the automobile M acquired by the speed sensor SS.

In other words, the video generation method of the present application is a video generation method in which the video playback device 10 assigns a virtual time code to the landscape video, and the video generation unit 63 acquires information on the shooting status of the landscape video. The moving image generation unit has an information-holding moving image generation step of generating an information-holding moving image in which a virtual time code based on information about a shooting situation at each time point during the shooting period of the scenery moving image is assigned to the scenery moving image.

In this embodiment, it has been described that the traveling speed of the automobile M, the importance, and the virtual time code are given to the landscape video after the landscape video is shot.

However, the traveling speed of the automobile M, the importance, and the virtual time code may be assigned to the landscape video while shooting the landscape video. When performing while shooting, the processing of steps S102 and S103 is repeated each time the frame image of the landscape video and the traveling speed and position information of the automobile M are supplied from the input unit 20, and the traveling speed of the automobile M at the time of the frame image is given. , Judgment and grant of importance. Further, each time the recording time of the landscape moving image exceeds a predetermined time, the processes of steps S105 and S106 are repeated, the average value of the importance is calculated, and the virtual time code is assigned.

Next, an example of a virtual time code assigned to the information holding video will be described with reference to FIG.

FIG. 6 shows an example of the elapsed time from the start of shooting of the moving image reproduction device 10 according to the embodiment of the present application, the frame numbers of a plurality of frames at predetermined time intervals, and the basic time codes of the plurality of frames. Further, the average value of the importance for each predetermined time calculated by the moving image generation unit 63, the magnification of the virtual time code with respect to the basic time code, and the assigned virtual time code are shown.

In this embodiment, in the 600-second landscape video shot by the in-vehicle camera CM, the shooting frame rate is set to 30 fps, the predetermined time for calculating the average value of importance is set to 60 seconds, and the importance is divided according to the traveling speed. A case where the degree is set in four stages of 0 to 3 will be described.

In addition, the magnification of the virtual time code is the same as the basic time code when the importance is 3, 1/2 of the basic time code when the importance is 2, and 1/3 of the basic time code when the importance is 1. In the case of degree 0, the setting will be described as a setting in which a virtual time code is assigned at a time code interval of 1/5 of the basic time code.

Since the basic time code and the virtual time code progression in the case of importance 3 proceed according to the frame rate of the captured landscape video, the progression speed is 1/30 second (about 0.033 seconds) per frame. Have. Further, the virtual time code progression in the case of importance 2 has a progress speed of 1/60 second (about 0.017 seconds) per frame, and the virtual time code progression in the case of importance 1 has a progress speed of 1/60 second (about 0.017 seconds) per frame. It has a progress speed of 1/90 second (about 0.011 second), and the virtual time code progress at the importance of 0 has a progress speed of 1/150 second (about 0.007 second) per frame.

That is, importance 3 is the same time lapse speed as the basic time code, importance 2 is about twice the time lapse speed of the basic time code, and importance 1 is about 3 times the time lapse speed of the basic time code. At 0, a virtual time code having a time elapsed speed about 5 times that of the basic time code is given.

In FIG. 6, the average value of the calculated importance is 3 in the elapsed time from the start of shooting to 180 seconds. Therefore, the virtual time code is given at a progress speed of 1/30 second per frame, which is equivalent to the basic time code.

In addition, the average value of the calculated importance is 1 during the elapsed time of 180 to 300 seconds from the start of shooting. Therefore, the virtual time code is given at a progress speed of 1/90 second per frame, which is a progress speed of 1/3 of the basic time code. That is, in 120 seconds of the elapsed time of 180 to 300 seconds from the start of shooting, the basic time code is the same progress speed as the actual shooting time of this period, but the virtual time code is the actual shooting time of 120 seconds in this period. On the other hand, it is given at a progress rate of 40 seconds.

In addition, the average value of the calculated importance is 2 in the elapsed time of 300 to 420 seconds from the start of shooting. Therefore, the virtual time code is given at a progress speed of 1/60 second per frame, which is a progress speed of 1/2 of the basic time code. That is, in 120 seconds of the elapsed time of 300 to 420 seconds from the start of shooting, the virtual time code is given at a progress speed of 60 seconds with respect to the actual shooting time of 120 seconds in this period.

Further, the average value of the calculated importance is 0 during the elapsed time of 420 to 480 seconds from the start of shooting. Therefore, the virtual time code is given at a progress speed of 1/150 second per frame, which is a progress speed of 1/5 of the basic time code. That is, in 60 seconds of the elapsed time of 420 to 480 seconds from the start of shooting, the virtual time code is given at a progress speed of 12 seconds with respect to the actual shooting time of 60 seconds in this period.

In addition, the average value of the calculated importance is 2 during the elapsed time of 480 to 600 seconds from the start of shooting. Since the progress speed of the virtual time code during this period is the same as that of the above-mentioned case of importance 2, the description thereof will be omitted.

FIG. 7 is a diagram showing the relationship between the frame numbers of a plurality of frames constituting the information-holding moving image in the example of the information-holding moving image shown in FIG. 6 and the virtual time code assigned to each frame number.

The horizontal axis shows the frame numbers of a plurality of frames constituting the information-holding moving image. In addition, the elapsed time from the start of shooting corresponding to each frame number is shown. The vertical axis represents the time code, and the horizontal axis represents the time code corresponding to the frame number.

The broken line in the figure indicates the basic time code, and the solid line indicates the virtual time code.

Since the basic time code is the same as the time elapsed when the landscape video was shot and it does not depend on the importance, the time elapsed per frame is constant at 1/30 second in any frame. As shown by the broken line in the figure, the basic time code is assigned as a straight line to the frame number.

Hereinafter, the virtual time code shown by the solid line in the figure will be described.

Since the calculated importance of the section of frame numbers 1 to 5400 (elapsed time from the start of shooting to 0 to 180 seconds) is 3, the virtual time code per frame is given with the same elapsed time as the basic time code. Will be done.

Since the calculated importance of the section of frame numbers 5401 to 9000 (elapsed time 180 to 300 seconds from the start of shooting) is 1, the virtual time code per frame is 1/3 of the basic time code. Granted by.

Since the calculated importance of the section of frame numbers 9001 to 12600 (elapsed time from the start of shooting 300 to 420 seconds) is 2, the virtual time code per frame is half the elapsed time of the basic time code. Granted by.

Since the calculated importance of the section of frame numbers 12601 to 14400 (elapsed time 420 to 480 seconds from the start of shooting) is 0, the virtual time code per frame is 1/5 of the basic time code. Granted by.

Since the calculated importance of the section of frame numbers 14401 to 18000 (elapsed time from the start of shooting 480 to 600 seconds) is 2, the virtual time code per frame is half the elapsed time of the basic time code. Granted by.

Next, the operation when the information-holding moving image recorded in the storage unit 50 is reproduced in the moving image reproducing device 10 will be described in detail.

When a reproduction instruction of the information-holding moving image recorded in the storage unit 50 on the touch panel 17 is input to the operation input unit 40 by the touch pad 17b, the operation input unit 40 supplies the reproduction instruction to the moving image control unit 60.

The moving image control unit 60 reads out the information-holding moving image instructed to be played back from the storage unit 50.

The thumbnail generation unit 65 is used for each predetermined time from a plurality of frame images constituting the information-holding moving image read from the storage unit 50 at each predetermined time when the average value of importance is calculated by the moving image generation unit 63. A frame image is extracted, a thumbnail image to be displayed on the thumbnail display unit SR of the touch panel 17 is generated, and the thumbnail image is supplied to the output unit 30.

The reproduction control unit 67 reproduces the information-holding moving image read from the storage unit 50 based on the basic time code, and supplies the reproduced image to the output unit 30.

The output unit 30 displays the thumbnail images supplied from the thumbnail generation unit 65 on the thumbnail image string SR displayed on the display 17a of the touch panel 17 in order of playback time. Further, the image supplied from the reproduction control unit 67 is displayed on the display 15.

When the user tries to search for a desired scene from the captured landscape video, for example, the user has an idea of where the desired scene is recorded in the landscape video, and the user has an idea. The suitable shooting time interval between the plurality of thumbnail images displayed as the thumbnail image string SR is different from that in the case where the thumbnail image string SR is not used. In the former case, the thumbnail image string SR, which is a thumbnail image around the shooting time that has already been guessed and the shooting time interval between the thumbnail images is relatively short, is effective. In the latter case, which is the desired scene first? Since it is necessary to guess whether or not the images are recorded around, the thumbnail image string SR having a relatively long shooting time interval between the thumbnail images is effective.

In order to cope with this, the moving image playback device 10 is configured so that the shooting time interval between adjacent thumbnail images of a plurality of thumbnail images displayed in the thumbnail image string SR as a part of the operation reception display can be changed by the user. It is preferable to have.

Specifically, the moving image playback device 10 includes an operation icon for designating the time scale of the thumbnail image string SR by the user in the operation reception display of the touch panel 17, and the thumbnail generation unit 65 displays the operation reception display of the user. The extraction interval of the virtual time code for extracting the thumbnail image to be displayed as the thumbnail image string SR is determined according to the time scale specified by the operation. This makes it possible to arbitrarily change the time scale of the thumbnail image sequence to search for a desired scene from a limited time range, or to search for a desired scene from a relatively long time range. It is possible to generate a valid thumbnail image string in.

Next, a case where a fast-forward or fast-rewind instruction is input to the operation input unit 40 from the touch panel 17 during playback of a landscape moving image of an information-holding moving image will be described.

The operation input unit 40 to which the fast-forward or fast-rewind instruction is input from the touch panel 17 supplies the fast-forward or fast-rewind instruction to the reproduction control unit 67 of the moving image control unit 60.

When the fast-forward or fast-rewind instruction is supplied, the playback control unit 67 performs fast-forward or fast-rewind playback control of the information-holding moving image at a predetermined playback speed magnification based on the virtual time code. That is, as shown in FIGS. 6 and 7, the lower the importance, the faster the speed of fast forward or fast rewind.

For example, when a double-speed fast-forward instruction is given in the example of the information-holding moving images of FIGS. 6 and 7, the reproduction control unit 67 performs double-speed fast-forward control in the section of importance 3.

Further, in the section of importance 2, since the virtual time code is assigned with the elapsed time of 1/2 of the basic time code per frame, the reproduction control unit 67 has four times the fast-forward speed. Fast forward control is performed with.

Further, in the section of importance 1, the virtual time code is assigned with the elapsed time of 1/3 of the basic time code per frame, so that the reproduction control unit 67 has a fast forward speed of 6 times. Fast forward control is performed with.

Further, in the section where the importance is 0, since the virtual time code per frame is assigned with the elapsed time of 1/5 of the basic time code, the reproduction control unit 67 has a fast forward speed of 10 times. Fast forward control is performed with.

In this embodiment, the playback control unit 67 has been described as performing fast-forward or fast-rewind control based on the virtual time code only during fast-forward or fast-rewind execution, but it is also variable based on the virtual time code during playback. It may be provided with a mode in which reproduction control is performed. In this case, in the above moving image example, the reproduction is performed in the section of importance 3 with the lapse of time equivalent to the basic time code. In addition, reproduction is performed at twice the speed in the section of importance 2, reproduction is performed at three times the speed in the section of importance 1, and reproduction is performed at five times the speed in the section of importance 0. Is done.

FIG. 8 shows a flowchart FC2 when playing back an information-holding moving image in the moving image reproducing device 10 according to the embodiment of the present application.

First, the reproduction control unit 67 determines whether or not the reproduction of the landscape moving image is instructed. This determination may be made, for example, depending on whether or not the operation for instructing the reproduction of the landscape moving image is accepted on the touch panel 17 (step S201).

Next, the reproduction control unit 67 reads out the information-holding moving image recorded in the storage unit 50 and instructed to reproduce (step S202).

The thumbnail generation unit 65 extracts a frame image for each predetermined time based on a predetermined time obtained by calculating an average value of importance from the read information-holding video, reduces the extracted frame image, and thumbnails the image. To create. Then, the operation display screen including the thumbnail image is supplied to the display 17a via the output unit 30 (step S203).

Next, the reproduction control unit 67 executes reproduction of the landscape moving image of the read information-holding moving image based on the basic time code (step S204).

Next, the moving image control unit 60 determines whether the fast-forwarding or fast-rewinding of the landscape moving image is instructed. This determination may be made on the touch panel 17, depending on whether or not the fast-forward or fast-rewind instruction operation of the landscape moving image is accepted from the touch panel 17 (step S205).

If it is not determined that the moving image control unit 60 has received the fast forward or fast rewind instruction (step S205: No), the reproduction control unit 67 continues the process of step S204 and executes the reproduction of the information-holding moving image.

Further, when it is determined that the moving image control unit 60 has received the fast forward or fast rewind instruction (step S205: Yes), the playback control unit 67 executes the fast forward or fast rewind process at a predetermined magnification based on the virtual time code (step S205: Yes). S206).

At the time of fast-forwarding or fast-rewinding of the information-holding moving image, the reproduction control unit 67 determines whether the fast-forwarding or fast-rewinding process has reached the first or last frame of the information-holding moving image (step S207).

If it is determined by the fast forward or fast rewind process that the first or last frame of the information holding video has been reached during the execution of the fast forward or fast rewind processing (step S207: Yes), the playback control unit 67 reproduces the information holding video. It ends (step S210).

If it is not determined by the fast-forward or fast-rewind process that the first or last frame of the information-holding video has been reached during the fast-forward or fast-rewind process (step S207: No), the playback control unit 67 continues the process of step S206. At the same time, it is determined whether the fast-forwarding or fast-rewinding instruction of the information-holding video is accepted from the touch panel 17 via the operation input unit 40 (step S208).

When it is determined in step 208 that the moving image control unit 60 has received the instruction to cancel the fast forward or fast rewind (step S208: Yes), the playback control unit 67 re-executes the process of step S204 to perform fast forward or fast rewind. The information-holding video is played again based on the basic time code from the frame number for which the cancellation instruction is received.

In step 208, when it is not determined that the moving image control unit 60 has received the instruction to cancel the fast forward or fast rewind (step S208: No), the reproduction control unit 67 repeats steps S207 and S208 to repeat the first or last frame of the information holding video. It is determined whether the user has reached the limit or has been instructed to cancel the fast forward or fast rewind.

Further, in step S205, when it is not determined that the operation input unit 40 has received the fast forward or fast rewind instruction during the reproduction of the information-holding moving image of step S204 (step S205: No), the reproduction control unit 67 processes the step S204. It is determined whether or not the final frame of the information-holding moving image has been reached by the reproduction process while continuing the above (step S209).

When the reproduction control unit 67 determines that the final frame of the information-holding moving image has been reached by the reproduction process (step S209: Yes), the reproduction control unit 67 ends the reproduction of the information-holding moving image (step S210).

When the first frame of the information-holding moving image is reached by the fast-rewinding process in step S207, the reproduction control unit 67 may shift to the processing of step S204 and execute the reproduction processing from the first frame of the information-holding moving image.

9A and 9B are diagrams showing a display example of the touch panel 17 when the information-holding moving image shown in FIGS. 6 and 7 is fast-forwarded.

FIG. 9A shows the operation reception display of the touch panel 17 at the time of fast-forwarding in the section of importance 3, and FIG. 9B shows the operation reception display of the touch panel 17 at the time of fast-forwarding in the section of importance 1. The length of the arrow in the figure indicates the scrolling speed.

For example, when a double-speed fast-forward instruction is given, the scroll speed in the section of importance 3 in FIG. 9A is 2 during playback based on a virtual time code in which the elapsed time per frame is equivalent to the basic time code. Scrolls at twice the scroll speed.

Further, the scroll speed in the section of importance 1 in FIG. 9B is scrolled at a scroll side speed 6 times that at the time of playback based on the virtual time code in which the elapsed time per frame is 1/3 of the basic time code. NS.

As a result, fast-forwarding or fast-rewinding can be executed at a normal speed in a high-importance section determined to have high highlights in the captured landscape video, and low-importance determined to have low highlights. In the section, fast forward or fast rewind can be executed at a speed faster than usual.

According to the moving image reproducing device 10 of the present invention, importance information according to the traveling speed or position information of the automobile M is added to the landscape moving image taken by the in-vehicle camera CM mounted on the automobile M, and the importance information is added to the importance information. By assigning a virtual time code based on the above, it becomes possible to efficiently search for a desired scene in the landscape moving image.

In this embodiment, a method of selecting a button FW for instructing a fast-forward operation and a button RW for instructing a fast-rewind operation displayed on the touch panel 17 for a fast-forward or fast-rewind instruction of an information-holding video has been described. Fast forward or fast rewind operations are not limited to this.

The fast-forward or fast-rewind operation may be performed by, for example, flicking the thumbnail image sequence SR of the touch panel 17 with a finger to perform an inertial scrolling operation. At this time, the inertial scroll speed may be variably changed in each section according to the importance.

In other words, the speed of fast-forwarding or fast-rewinding other than the reproduction of the information-holding moving image may be variably changed according to the importance.

Further, in the moving image playback device 10 of the above-described embodiment, a method of generating an information-holding moving image by assigning an importance and a virtual time code when the in-vehicle camera CM finishes shooting a landscape moving image has been described. However, the method of generating the information-holding video is not limited to the execution when the shooting of the landscape video is finished. For example, after it is determined by the touch panel 17 that the playback instruction of the landscape moving image has been given (step S201: Yes in FIG. 8), steps S102 to S106 of FIG. 5 are executed to generate the information-holding moving image. After that, the information-holding moving image may be supplied to the thumbnail generation unit 65 and the reproduction control unit 67, and the processes after step S202 in FIG. 8 may be executed.

By the above processing step, the moving image reproduction device 10 assigns the importance and the virtual time code after receiving the reproduction instruction of the landscape moving image. This makes it possible to reflect the setting without reconverting the video data even if the setting for changing the judgment condition of importance or the predetermined time for calculating the average value of importance is performed after shooting the landscape video. Become.

Further, in the present embodiment, the importance of the landscape video taken from the in-vehicle camera CM mounted on the automobile M is determined according to the traveling speed and the shooting position of the automobile M, and fast forward or fast according to the importance. The moving image playback device 10 for variably changing the return speed has been described. However, the moving image playback device according to the present invention is not limited to landscape moving images from automobiles.

For example, a moving image in which importance is given to a moving image taken by a smartphone, a handy camera, or the like may be generated. Further, the captured information-holding moving image may be recorded in an external recording device such as a USB memory or HDD and played back in a device such as a PC.

Further, in this embodiment, the method of calculating the average value of importance for each predetermined section of the basic time code and assigning the virtual time code for each predetermined section has been described, but the method of assigning the virtual time code is described. Not limited to this. For example, the passage of time for each frame may be assigned according to the importance to each of the plurality of frames constituting the landscape moving image to which the importance is given. As a result, even if the importance of a predetermined section of the basic time code changes significantly, it is possible to efficiently search the section of high importance.

Further, in this embodiment, a method of displaying thumbnail images of information-holding moving images side by side on a scroll bar for each predetermined time and variably controlling the speed of fast forward or fast rewind for each predetermined time has been described. However, the moving image playback device according to the present invention is not limited to the scroll bar in which thumbnail images are displayed side by side, and may be used for a seek bar type moving image playback device in which thumbnail images are not displayed.

Further, in this embodiment, the moving image reproduction device 10 that generates one moving image data in which a virtual time code is added to a landscape moving image and controls the reproduction of the moving image data has been described. However, the playback control method is not limited to generating one moving image data. For example, a landscape video to which the importance and the virtual time code are not assigned, and information data indicating the frame number and the virtual time code for the frame number are generated from the landscape video. By controlling the playback of the landscape video and the information data in parallel by the playback control unit 67, the playback is executed based on the basic time code assigned to the landscape video during playback, and the information data is framed during fast forward or fast rewind. The virtual time code corresponding to the number and the frame number may be read and the fast forward or fast rewind control may be performed variably.

Further, in the present embodiment, the method of determining the importance from the traveling speed of the automobile M and generating the virtual time code from the importance has been described, but the method of generating the virtual time code is not limited to this. For example, a plurality of threshold values may be set for the running average speed of the automobile M for each predetermined time, and a virtual time code corresponding to the plurality of thresholds of the running average speed may be set. This makes it possible to directly generate a virtual time code from the traveling speed of the automobile M at predetermined time intervals.

That is, it is sufficient that a virtual time code indicating a specific highlight can be generated for each of the plurality of frame images constituting the captured image, and the speed at the time of fast-forwarding or fast-rewinding the moving image can be changed.

10 Video playback device 15 Display 17 Touch panel 20 Input unit 30 Output unit 40 Operation input unit 50 Storage unit 60 Video control unit 63 Video generation unit 65 Thumbnail generation unit 67 Playback control unit

Claims (14)

An acquisition unit that acquires landscape videos taken from a moving body,
A video generation unit that generates an information-holding video that is attached to the landscape video with importance information based on information about the shooting status at each time point during the shooting period of the landscape video.
A video recording unit that records the information-holding video generated by the video generation unit, and a video recording unit.
It has a playback control unit that reads the information-holding video from the video recording unit, performs playback, fast-forward or fast-rewind control of the landscape video, and outputs the video.
The reproduction control unit is a moving image reproduction device characterized in that reproduction control including reproduction, fast forward, and fast rewind of the landscape moving image is performed at a speed based on the importance information. The moving image generation unit sets a series of virtual images based on information on the shooting situation when each of the plurality of frame images is shot for each of the plurality of frame images constituting the shot landscape movie. A virtual time code, which is a time code, is added as the importance information to generate the information holding video.
The moving image playback device according to claim 1, wherein the playback control unit performs the playback control based on the virtual time code. The moving image generation unit determines the importance of each of the plurality of frame images based on the information regarding the shooting situation, and assigns the virtual time code corresponding to the importance to the plurality of frame images. The moving image playback device according to claim 2. A second aspect of the present invention, wherein the moving image generation unit assigns the virtual time code so that the lower the importance, the faster the passage of time indicated by the virtual time code in the group of frame images. The moving image playback device according to any one of 3. The moving image reproducing device according to any one of claims 2 to 4, wherein the moving image generating unit determines the importance based on the moving speed of the moving body. The moving image reproducing device according to any one of claims 2 to 4, wherein the moving image generating unit determines the importance based on the GNSS positioning information of the moving body or the shooting position. The moving image generating unit according to any one of claims 2 to 4, wherein the moving image generating unit determines the importance based on the GNSS positioning information of the moving body or the position on the map in comparison with the shooting position. Video playback device. The moving image reproducing device according to any one of claims 2 to 4, wherein the moving image generating unit determines the importance based on a shooting direction based on a moving direction of the moving body. The moving image playback device includes a thumbnail image generation unit that extracts a frame image from the plurality of frame images constituting the information-holding moving image at predetermined time intervals and generates a thumbnail image.
An operation reception unit that displays a thumbnail image string in which the thumbnail images are arranged in the order of playback time and accepts an instruction operation including a selection operation of the thumbnail image included in the thumbnail image string or a scroll operation for the thumbnail image string.
Including
The moving image reproduction device according to any one of claims 1 to 8, wherein the reproduction control unit performs the reproduction control based on the instruction operation. An acquisition unit that acquires landscape videos taken from a moving body,
The importance information acquisition unit that acquires importance information based on the information on the shooting situation at each time point during the shooting period of the landscape video, and the importance information acquisition unit.
A moving image reproduction device comprising a reproduction control unit for reproducing the landscape moving image at a speed based on the importance information. An acquisition unit that acquires landscape videos taken from a moving body,
A moving image generation device characterized by having a moving image generating unit for generating an information-holding moving image added to the scenery moving image, based on importance information based on information on a shooting situation at each point during the shooting period of the landscape moving image. .. This is a video generation method in which a video playback device adds importance information to a landscape video.
The acquisition process in which the information acquisition department acquires information on the shooting status of the landscape video, and
The video generation unit has an information-holding video generation step of generating the information-holding video attached to the landscape video with importance information based on the information about the shooting status at each time point during the shooting period of the landscape video. A featured video generation method. A video generation program executed by a video generation device provided in a computer, which is installed in the computer.
The acquisition process in which the information acquisition department acquires information on the shooting status of landscape videos,
The video generation unit executes the information-holding video generation step of generating the information-holding video added to the landscape video with importance information based on the information on the shooting status at each time point during the shooting period of the landscape video. A video generation program featuring. A computer-readable recording medium comprising storing the moving image generation program according to claim 13.

Images