JP6451521B2 - Digest creation device, digest creation system, digest creation method and program - Google Patents

Description

  The present invention relates to a digest creation device, a digest creation system, a digest creation method, and a program.

  Digest video is created from video data. For example, Patent Document 1 describes that in a highlight still image generation / distribution device, a highlight scene moving image is cut out from video captured by a camera, and an important scene is extracted from the moving image as a highlight still image. Has been.

JP 2005-191770 A

  An important scene can be extracted by the highlight still image generation / distribution device according to the background art. However, the apparatus designates a portion to be extracted as a highlight in a moving image being shot by a sensor signal or manually, and a range important for the user cannot be extracted from the moving image data with high accuracy.

  The present invention has been made to solve such a problem, and a digest creation device, a digest creation system, a digest that can create a digest video by accurately extracting a range important for a user from video data. An object is to provide a creation method and a program.

  Therefore, the present invention sets in advance the data communication unit that continuously receives the vital data of the user who records or reproduces the moving image data and receives the timing data of the tag input of the user, and the vital data received by the data communication unit. When the time range corresponding to the timing data is included in the time range that is greater than or equal to the value set, based on the tag range processing unit that sets the time range as a tag range and the tag range set by the tag range processing unit, The digest processing unit creates a digest video of the video data. When there are multiple tag ranges, the digest processing unit creates digest video from multiple tag ranges based on the importance of vital data in the tag range. Provided is a digest creation device for selecting a tag range to be used.

  The present invention also provides a data communication step for continuously receiving vital data of a user who performs recording or reproduction of moving image data and receiving timing data of tag input of the user, and vital data received in the data communication step is set in advance. When the time range corresponding to the timing data is included in the time range that is greater than or equal to the value that has been set, based on the tag range processing step that sets the time range as a tag range, and the tag range set in the tag range processing step, A digest processing step that creates a digest video of the video data. When there are multiple tag ranges, the digest processing step creates digest video from multiple tag ranges based on the importance of vital data in the tag range. Eject to select a tag range for To provide a deposition method.

  In addition, the present invention provides a data communication procedure for continuously receiving vital data of a user who performs recording or reproduction of moving image data to a computer and receiving timing data of tag input of the user, and vital data received by the data communication procedure. If the time range corresponding to the timing data is included in the time range that is greater than or equal to the preset value, the tag range processing procedure for setting the time range as a tag range and the tag range set in the tag range processing procedure A digest processing procedure for creating a digest video of the video data based on the importance of the vital data in the tag range when there are a plurality of tag ranges. Tag to create a digest video from multiple tag ranges To provide a program to select the circumference.

  According to the present invention, it is possible to provide a digest creation device, a digest creation system, a digest creation method, and a program that accurately extract a range important for a user from moving image data and create a digest video.

It is a block diagram which shows schematic structure of the digest production system 1 which concerns on embodiment. It is a figure which shows the relationship between the time change of the heart rate which concerns on embodiment, and the range of the tag F. FIG. It is a flowchart which shows the process sequence of the digest process part 52 which concerns on embodiment. It is a figure for demonstrating the processing content of the digest process part 52 which concerns on embodiment. It is a figure for demonstrating the processing content of the digest process part 52 which concerns on embodiment. It is a figure for demonstrating the digest video data which concerns on embodiment.

  Hereinafter, a digest creation system according to the present embodiment will be described with reference to the drawings. The digest creation system according to the present embodiment includes the vital data (biological information) of the user and the tag timing data input by the user in an important scene when the user is shooting a sporting game or the like. Is used to create a digest video by accurately setting a tag range in an important range of shooting data.

First, the configuration of the digest creation system according to the present embodiment will be described.
FIG. 1 is a block diagram showing a schematic configuration of a digest creation system 1 according to the present embodiment.
The digest creation system 1 includes a sensor device 2, a tag input device 3, an imaging device 4, and the like. Communication between the sensor device 2 and the imaging device 4 and between the tag input device 3 and the imaging device 4 can be performed wirelessly or by wire.

The sensor device 2 measures a heart rate, which is vital data of the user, when a user (not shown) is photographing with the imaging device 4. The sensor device 2 includes data communication that transmits the measured heart rate to the imaging device 4 in real time in addition to the configuration of a general sensor device such as the sensor main body 21, the sensor control unit 22, the sensor data processing unit 23, and the CPU 24. The unit 25 is provided.
When the tag input device 3 thinks that it is better for the user to shoot with the image pickup device 4 and tag the photographic data, for example, in the case of a soccer game, the tag input device 3 is pressed by pressing a button or the like. Input. The tag input device 3 includes a key input unit 31 such as a button and a screen, a key control unit 32, an input tag data processing unit 33, a configuration provided in a general tag input device such as a CPU 34, and a tag input by a user. A data communication unit 35 that transmits input tag data, which is data, to the imaging device 4 in real time is provided. The input tag data includes, for example, information on the time when the user inputs a tag.

  The imaging device (digest creation device) 4 can capture a moving image such as a video camera or a digital camera. The imaging device 4 includes a data communication unit 47, sensor data, in addition to a configuration provided in a general imaging device such as an image sensor 41, an imaging unit 42, an image data processing unit 43, a data output unit 44, a recording medium 45, and a CPU 46. A recording unit 48, a tag control unit 49, a tag range processing unit 50, a tag recording unit 51, a digest processing unit 52, a digest recording unit 53, and the like are provided.

The data communication unit 47 receives sensor data transmitted by the data communication unit 25 of the sensor device 2, input tag data transmitted by the data communication unit 35 of the tag input device 3, and the like. In addition, the data communication unit 47 may transmit a signal for controlling the operation start timing to the sensor device 2, the tag input device 3, and the like. That is, the data communication unit 25 of the sensor device 2 and the data communication unit 35 of the tag input device 3 communicate with the data communication unit 47 of the imaging device 4 in one direction or in both directions such as a control signal and a status notification signal. Also good.
The sensor data recording unit 48 records sensor data received from the sensor device 2.
The tag control unit 49 controls the entire operation of adding a tag in the imaging device 4.
The tag range processing unit 50 uses the received sensor data and input tag data to set a tag in an appropriate range of the moving image data captured by the imaging device 4.

The tag recording unit 51 stores the received sensor data, input tag data, set tag range data, and the like in association with moving image data.
The received sensor data, input tag data, set tag range data, and moving image data associated therewith may be recorded on the recording medium 45.
The digest processing unit 52 creates a digest video based on the tag range set by the tag range processing unit 50.
The digest recording unit 53 stores the data related to the digest video created by the digest processing unit 52 in association with the moving image data.

The input tag data includes, for example, the time when the user inputs the tag and the time when the imaging device 4 receives the tag input. Even if it is not the time, if the sensor device 2, the tag input device 3, and the imaging device 4 can share some synchronized timing such as the start of recording, the information of the elapsed time from that point can be substituted.
If the sensor device 2, the tag input device 3 and the imaging device 4 have a built-in clock, the sensor device 2 and the tag input device 3 transmit the measured heart rate and input tag data to the imaging device 4 in real time. do not have to. Later, it is possible to synchronize the timing of the moving image data, vital data, and input tag data based on the respective clock information.

  Each component realized by the imaging device 4 can be realized by executing a program under the control of the CPU 46. More specifically, the imaging device 4 is realized by loading a program stored in a storage unit (not shown) into a main storage device (not shown) and executing the program under the control of the CPU 46. Each component is not limited to being realized by software by a program, and may be realized by any combination of hardware, firmware, and software.

  The above-described program can be stored using various types of non-transitory computer readable media and supplied to the imaging device 4. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W and semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)) are included.

  Further, the program may be supplied to the imaging device 4 by various types of temporary computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the imaging device 4 via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

Subsequently, an operation of the digest creation system 1 according to the present embodiment, that is, a digest creation method will be described.
First, an outline of the operation of the tag range processing unit 50 will be described. As described above, the tag range processing unit 50 uses the received sensor data and input tag data to set a tag in an appropriate range of the moving image data captured by the imaging device 4.

FIG. 2 is a diagram showing the relationship between the time change of the heart rate and the tag range set using the vital data (heart rate) and the tag data input by the user according to the present embodiment.
The graph showing the heart rate and the tag F that is the input tag data (in this embodiment, tags F1 to F4) are shown at the top, and the tag F range associated with the video data (tag range) is shown at the bottom. ing. In the heart rate graph, the horizontal axis represents time, and the vertical axis represents heart rate. A horizontal line near the heart rate 75 indicates a threshold for setting the tag range of the tag F. Further, horizontal arrows and characters such as “normal” and “tension” written in the vicinity of the heart rate 40 are for explaining the state of the user at that time.

In the example of the heart rate shown in FIG. 2, the state of the user changes from a change in heart rate from the start of the game to the end of the game to a normal state, a tension state or an excitement state, and a normal (relaxed) state. Four peaks (1 to 4) of the heart rate occur in the tension state or the excitement state, and the user inputs the tags F1 to F4 at each peak.
When the heart rate is a time range in which the heart rate is equal to or greater than a predetermined value (here, 75 as a threshold) and the user inputs the tag F within the time range, the tag range processing unit 50 Is set as the tag range of the tag F inputted. A tag may be attached to a frame in the time range.

For example, for the first peak, since the heart rate exceeds a predetermined value in the time range t1 to t2, and the user inputs the tag F1, the time range t1 to t2 is set as the tag range of the tag F1. To do. For the second peak, since the heart rate exceeds a predetermined value in the time range t3 to t4 and the user inputs the tag F2, the time range t3 to t4 is set as the tag range of the tag F2. To do. The same applies to the third and fourth peaks.
Then, the tag range processing unit 50 records the tag F, the tag range of the tag F, and the time range (tag time) in the tag recording unit 51 as a recording tag list.

Note that the tag range processing unit 50 does not set the tag range when the user does not input the tag F within the time range even if the heart rate has a time range equal to or greater than the predetermined value.
Moreover, even if the user inputs the tag F, if the heart rate is not equal to or greater than the predetermined value, the tag range processing unit 50 sets the tag range by default or the time or the number of frames set by the user.
In this way, the tag range processing unit 50 can accurately set the tag range in the important range of the moving image data.

Next, the operation of the digest processing unit 52 will be described.
FIG. 3 is a flowchart showing a processing procedure of the digest processing unit 52 according to the present embodiment.
4 and 5 are diagrams for explaining the processing contents of the digest processing unit 52 according to the present embodiment. 4 and 5 respectively show a recording tag list, a digest candidate list, and a creation digest from the left side.

  The record tag list is created by the tag range processing unit 50 as described above and recorded in the tag recording unit 51. Information such as the tag ID, tag start time, and tag time of the tag set by the tag range processing unit 50 is recorded. including. The digest candidate list is created by the digest processing unit 52 and recorded in the storage unit or the digest recording unit 53 inside the digest processing unit 52. The tag ID of a tag that may be used in the digest video, and the tag Includes information such as vital data integration values and tag times for each tag.

The created digest is created by the digest processing unit 52 and recorded in the digest recording unit 53, and includes tag ID information of a tag used in the digest video.
FIG. 4 shows a state in which the processing of the tag F1 and the tag F2 is completed in the processing procedure described below. FIG. 5 shows a state in which the processing is completed in the processing procedure.
Further, the storage tag list, digest candidate list, and created digest can be recorded on the recording medium 45 together with the moving image data.

  The processing procedure of the digest processing unit 52 will be described with reference to FIGS. It is assumed that the digest processing unit 52 creates a digest video within 10 minutes with reference to the setting of the digest video setting time. The digest video setting time is for the user to set the length of the digest video to be created in advance.

First, when processing is started, the digest processing unit 52 determines whether there is a tag F to be added to the recording tag list (step S10). The digest processing unit 52 refers to the recording tag list in order from the top, and determines that there is a tag F1 (Yes in step S10).
Then, it is determined whether the tag F is a tag set using the vital data and the tag data input by the user (step S20). The tag F1 is determined to be a tag using vital data and tag data input by the user (Yes in step S20).

Then, the vital data values within the tag range are integrated (step S30). Here, the heart rate in the time range t1 to t2, which is the tag range of the tag F1, is integrated to calculate the integrated value 300. The integrated value of vital data is an index of the importance of the tag F.
Then, the tag ID in the candidate list is compared with the vital data integrated value (step S40). In the case of the tag F1, since no other tag ID is recorded in the candidate list, the process proceeds to the next.

Then, the tags F corresponding to the tag IDs in the candidate list are arranged in descending order of the integrated value of the vital data (step S50). In the case of the tag F1, since no other tag F is recorded in the candidate list, the tag F1 is recorded and the process proceeds to the next.
Then, it is determined whether the digest video set time has been exceeded (step S60). The tag time of the tag F1 is 3 minutes (180 seconds), and it is determined that it does not exceed the above-described digest video setting time of 10 minutes (No in step S60), and the process returns to step S10.

  Next, the digest processing unit 52 similarly determines that there is a tag F2 (Yes in step S10), determines that the tag F2 is a tag using vital data (Yes in step S20), and determines the tag F2 The integrated value 500 of the heart rate in the time range t3 to t4 that is the tag range is calculated (step S30).

Then, the tag F1 in the candidate list is compared with the vital data integrated value (step S40), and the tag IDs of the candidate list are arranged in descending order of the vital data integrated value (step S50). Here, the integrated value of the tag F1 is 300, the integrated value of the tag F2 is 500, and the tags F2 are arranged so as to be higher than the tag F1.
The tag time of the tag F2 is 4 minutes (240 seconds), the tag time of the tag F1 is 3 minutes, and it is 7 minutes in total, and does not exceed the digest video setting time of 10 minutes. Judgment is made (No in step S60). As described above, FIG. 4 shows the state at this point, that is, the processing of the tags F1 and F2. Then, the process returns to step S10.

Next, the digest processing unit 52 similarly calculates the integrated value 400 in the heart rate in the time range t5 to t6 that is the tag range of the tag F3 (steps S10 to S30).
Then, the tag F2 and tag F1 in the candidate list are compared with the vital data integrated value (step S40), and the tag F in the candidate list is sorted in descending order of the vital data integrated value, that is, the tag F2, the tag F3, and the tag F1. They are arranged in order (step S50).

The tag time of the tag F3 is 4 minutes (240 seconds), the tag times of the tag F2 and the tag F1 are 4 minutes and 3 minutes, respectively, and the total time is 11 minutes. It is determined that it exceeds 10 minutes (Yes in step S60).
Then, the tag ID having a small vital data integrated value is deleted from the candidate list until the set time is reached (step S70). Here, first, the tag F1 having a small vital data integrated value is deleted. As a result, the total tag time of the tags F2 and F3 becomes 8 minutes and falls within the set time, and the process returns to step S10.

Next, the digest processing unit 52 similarly calculates the integrated value 140 in the heart rate in the time range t7 to t8 that is the tag time of the tag F4 (steps S10 to S30), and sets the tag ID of the candidate list as the vital data. The integrated values are arranged in descending order, tag F2, tag F3, and tag F4 in this order (steps S40 and S50).
The tag time of tag F4 is 2 minutes and 20 seconds (140 seconds), and the total tag time of tag F2 and tag F3 is 10 minutes and 20 seconds, so it is determined that the digest video set time has been exceeded. Then, the tag F4 having a small vital data integrated value is deleted, the total tag time is set to the set time (steps S60 and S70), and the process returns to step S10.

When the digest processing unit 52 determines that there is no next tag F (No in step S10), the tag ID of the tag F remaining in the digest candidate list is recorded in the creation digest, and the process ends. FIG. 5 shows the time point, that is, the state where the processing is completed. The tags F2 and F3 remaining in the digest candidate list are recorded as they are in the created digest. In FIG. 5, the deleted tag F1 and tag F4 are also shown with a strikethrough (double line) for reference.
If the digest processing unit 52 determines that the tag F is not a tag using vital data (No in step S20), the digest processing unit 52 returns to step S10.

FIG. 6 is a diagram for explaining digest video data according to the present embodiment. The original image data is shown on the upper side, and the digest video data is shown on the lower side. Here, the digest video setting time is infinite.
In the digest video data, the tag times of the tags F1 to F4 are different, and it can be seen that a digest video is created by accurately extracting a range important for the user.
As described above, the digest creation system 1 according to the present embodiment can accurately extract a range important for a user from video data and create a digest video. Also, for the extracted range, By determining their importance, a digest video can be created within a set time.

  In the digest creation system 1 according to the present embodiment, the heart rate is used as vital data. However, data other than the heart rate, for example, data indicating the degree of excitement of the user may be used as vital data. In this case, when the vital data is a time range such as the predetermined value or less or less than the predetermined value, and the user inputs the tag F within the time range, the time range is set as the tag F tag range. May be set.

  Further, in the digest creation system 1 according to the present embodiment, the importance of the tag F is determined using the integrated value of the heart rate (vital data), but the integrated value of the heart rate above the threshold, that is, The importance of the tag F may be determined using an integrated value of “a value obtained by subtracting a threshold value from the heart rate”. In addition to the integral value of vital data, the magnitude of the absolute value of the peak of vital data, the sharpness of the peak of vital data (the magnitude of change rate), the number of tags F per unit time (tag density), The importance of the tag F may be determined using the length of the tag range (length of tag time) or the like.

In the digest creation system 1 according to the present embodiment, the threshold value of the heart rate for setting the tag range of the tag F is fixed, but the threshold value is set according to the situation such as for each user or for each scene. It may be settable. Further, the threshold value may be changed at any time so that the tag range of the tag F can be adjusted even after moving image data is recorded.
Also, in the digest creation system 1 according to the present embodiment, when the heart rate is in a time range equal to or greater than a predetermined value and the user inputs the tag F within the time range, the time range is set to the tag F. However, when the heart rate is a time range equal to or greater than a predetermined value and the user inputs the tag F within the time range, a predetermined time before and after the tag F input by the user, For example, three seconds before and after the tag F may be set as the tag F tag range. Of course, the user may be allowed to select a time range in which the heart rate is equal to or greater than a predetermined value or a predetermined time before and after the tag F.

Moreover, although the digest production system 1 which concerns on this Embodiment was set as the structure provided with the sensor apparatus 2, the tag input device 3, the imaging device 4, etc., the structure by which an imaging device serves as a sensor apparatus and a tag input device, or imaging The device may be configured to serve as either a sensor device or a tag input device. That is, the digest creation system may be configured as a digest creation device.
In addition, the digest creation system 1 according to the present embodiment is configured so that the user's vital data and input tag when the user is shooting using the imaging device 4, that is, when the user is recording moving image data. The digest video was created based on the data, but when the user is watching or editing the video using an image processing device or the like, that is, when the user is playing the video data, A digest video may be created based on vital data and input tag data. For this reason, the digest creation apparatus according to the present embodiment is not limited to the imaging apparatus, and may be an image processing apparatus or the like.

  As described above, the digest creation device 4 according to this embodiment includes the data communication unit 47 that continuously receives the vital data of the user who records or reproduces the moving image data and receives the timing data of the user's tag input. The tag range for setting the time range tm to tn to the tag range tm to tn when the time range corresponding to the timing data is included in the time range in which the vital data received by the data communication unit 47 is greater than or equal to a preset value The processing unit 50 includes a digest processing unit 52 that creates a digest video of moving image data based on the tag range set by the tag range processing unit 50. The digest processing unit 52 includes a plurality of tag ranges. , Digest from multiple tag ranges based on the importance of vital data in the tag range It is to select a tag range to create.

The digest creation device 4 according to the present embodiment calculates the importance of vital data in the digest processing unit 52 by integrating vital data values within the tag range.
The digest creation system 1 according to the present embodiment also includes a sensor device 2 that measures user vital data and outputs it to the data communication unit 47, and a tag that outputs timing data of tag input by the user to the data communication unit 47. An input device 3 and a digest creation device 4 are provided.

  In addition, the digest creation method according to the present embodiment includes a data communication step of continuously receiving vital data of a user who performs recording or reproduction of moving image data and receiving timing data of tag input of the user, and a data communication step. When the time range corresponding to the timing data is included in the time range in which the received vital data is greater than or equal to a preset value, the tag range processing step sets the time range as the tag range and the tag range processing step. A digest processing step that creates a digest video of the video data based on the tag range, and the digest processing step includes a plurality of tag ranges based on the importance of vital data in the tag range. To create a digest video from the tag range It is for selecting the tag range.

  Further, the program according to the present embodiment includes a data communication procedure for continuously receiving vital data of a user who performs recording or reproduction of moving image data and receiving timing data of tag input of the user, and a data communication procedure. If the time range corresponding to the timing data is included in the time range in which the vital data received in step 1 is equal to or greater than the preset value, set the tag range processing procedure to set the time range as the tag range and the tag range processing procedure The digest processing procedure for creating a digest video of the video data based on the tag range that has been created, and the digest processing procedure is important for vital data in the tag range when there are multiple tag ranges. Create digest video from multiple tag ranges based on degree And selects tags range for.

DESCRIPTION OF SYMBOLS 1 Digest creation system 2 Sensor apparatus 3 Tag input device 4 Imaging device (digest creation apparatus)
50 Tag range processing unit 52 Digest processing unit

Claims (5)

A data communication unit for continuously receiving vital data of a user who records or reproduces video data and receives timing data of the user's tag input;
A tag range processing unit that sets the time range as a tag range when the time range corresponding to the timing data is included in a time range in which the vital data received by the data communication unit is greater than or equal to a preset value;
A digest processing unit that creates a digest video of the video data based on the tag range set by the tag range processing unit;
The digest processing unit selects the tag range for creating the digest video from a plurality of the tag ranges based on the importance of the vital data in the tag range when there are a plurality of the tag ranges. A digest creation device characterized by that. The digest creation apparatus according to claim 1, wherein the importance of the vital data in the digest processing unit is calculated by integrating vital data values within the tag range. A sensor device for measuring the vital data of the user and outputting it to the data communication unit;
A tag input device that outputs the timing data of the tag input by the user to the data communication unit;
A digest creation system comprising: the digest creation device according to claim 1. A data communication step of continuously receiving vital data of a user who performs recording or reproduction of moving image data and receiving timing data of the tag input of the user;
A tag range processing step of setting the time range as a tag range when the time range corresponding to the timing data is included in a time range in which the vital data received in the data communication step is greater than or equal to a preset value;
A digest processing step of creating a digest video of the video data based on the tag range set in the tag range processing step;
The digest processing step selects the tag range for creating the digest video from a plurality of the tag ranges based on the importance of the vital data in the tag range when there are a plurality of the tag ranges. ,
A digest creation method characterized by this. On the computer,
A data communication procedure for continuously receiving vital data of a user who performs recording or reproduction of moving image data and receiving timing data of the tag input of the user;
A tag range processing procedure for setting the time range as a tag range when the time range corresponding to the timing data is included in a time range in which the vital data received in the data communication procedure is equal to or greater than a preset value;
A digest processing procedure for creating a digest video of the video data based on the tag range set in the tag range processing procedure;
A program for executing
The digest processing procedure selects the tag range for creating the digest video from a plurality of the tag ranges based on the importance of the vital data in the tag range when there are a plurality of the tag ranges. A program characterized by that.

Images