JP6464925B2 - Tagging device, tagging system, tagging method and tagging program - Google Patents

Description

  The present invention relates to a tagging device, a tagging system, a tagging method, and a tagging program.

  In recent years, a necessary scene is extracted from moving image data shot by a digital camera or digital video, and a digest moving image is edited and created. For example, there is a technique for adding a tag (or an index or a flag) to a scene necessary for shooting a moving image, and editing and creating a moving image scene of a user's preference according to the tag in moving image editing and creation. .

  For example, Patent Document 1 describes an image recording apparatus that sets a flag for extracting and recording a frame at a start timing when biometric information changes, and records a frame at the timing (and before and after the timing) at which the flag is set. Yes.

JP 2014-192755 A

  However, in the image recording apparatus described in Patent Document 1, the tag is struck even when the tag is not desired because the tag is struck as the timing for recording all the changes in the biological information. There was a problem.

  A tagging device according to the present invention includes a tag input data that is a base for attaching a tag to data including at least one of audio and moving images, and a data input unit that receives vital data that is biometric information, the tag input data, and the vital data Based on the data, a range setting unit that determines that the level of the vital data is equal to or greater than a preset threshold value as a taggable time range, and a peak value of the vital data can be tagged in the taggable time setting range A tag control unit for making time timing.

The tagging system of the present invention includes an imaging device that captures an image of a subject and outputs moving image data, a tag input device that accepts tag tagging instructions from a tag grantor and outputs tag input data, and measures vital data of the tag grantor. A sensor device for outputting, a tagging device for attaching a tag to the moving image data based on the tag input data and the vital data, a data input unit for receiving the tag input data and the vital data, and the tag input Based on the data and the vital data, a range setting unit that determines a level of the vital data equal to or higher than a preset threshold as a taggable time range, and a peak value of the vital data in the taggable time range. A tag control unit for setting a tag assignable time.

  The tagging method of the present invention accepts tag input data and vital data that is biometric information when a tag is added to video data, and the level of the vital data is based on the tag input data and the vital data. A predetermined threshold value or more is determined as a tag assignable time range, a peak value of the vital data is detected in the tag assignable time range, and the peak value is set as a tag assignable time timing.

  The tagging program of the present invention includes a data input step for receiving tag input data and vital data that is biometric information when a tag is attached to video data, and the vital data based on the tag input data and the vital data. A range setting step for determining a tag level time range that is equal to or higher than a predetermined threshold as a data level, a step for detecting a peak value of the vital data in the tagging time range, and tagging the peak value A tag control step for setting a possible time timing is executed by a computer.

  According to the tagging device, the tagging system, the tagging method, and the tagging program of the present invention, it is possible to generate a tag with an appropriate timing, and to tag a moving image with an appropriate timing.

1 is a block diagram showing a configuration of a tagging system according to a first exemplary embodiment. 3 is a flowchart illustrating an example of an operation of the tagging device according to the first exemplary embodiment; It is a figure which shows the example of the relationship between vital data and tag hitting timing. 6 is a flowchart illustrating an example of an operation of a tagging device according to a second exemplary embodiment;

(Embodiment 1)
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram illustrating a tagging system including a sensor device, a tag input device, a tagging device, and an imaging device according to the first embodiment.

  In FIG. 1, moving image data acquired by the imaging device 200 continuously capturing an image of a subject, the sensor device 300 receives a person's vital data, and the tag input device 400 accepts a tagging instruction from a person to perform tagging. The tag input data that is the source of the instruction is transmitted to the tagging device 100. The tagging device 100 determines the timing for tagging the moving image data based on the input tag input data and the vital data.

  In FIG. 1, the tagging device 100 processes the moving image data captured by the imaging device 200, the vital data measured by the sensor device 300, the tag input instruction received by the tag input device 400 and the time in synchronization. When delay is considered in the transmission of data between devices, the data of each device may be transmitted in association with the time, and the data at the same time may be processed in synchronization.

  First, the tagging apparatus 100 includes a data input unit 101, a sensor data recording unit 102, a tag control unit 103, a range setting unit 104, and a CPU 105.

  The data input unit 101 communicates with a sensor device 300 (to be described later) and a tag input device 400 (to be described later) wirelessly or by wire. Specifically, the data input unit 101 receives vital data transmitted from the sensor device 300, receives tag input data transmitted from the tag input device 400, and receives moving image data captured from the imaging device 200. Receive. The tag input data includes time information at which the tag is given.

  In the case of wireless communication, the data input unit 101 is preferably configured by a wireless communication circuit including an antenna, an amplifier circuit, a frequency conversion circuit, a modulation circuit, and a demodulation circuit (not shown). In addition, when communicating by wire, the data input unit 101 is preferably configured by a wired communication circuit including an amplifier circuit, a modulation circuit, and a demodulation circuit (not shown). In addition, when communicating using light, the data input unit 101 preferably includes a light emitting element, a light modulation circuit, a light receiving element, and a light demodulation circuit (not shown).

  In particular, the data input unit 101 receives vital data measured by the sensor device 300, receives a tag hitting instruction transmitted by the tag input device 400, and receives moving image data captured from the imaging device 200.

  The sensor data recording unit 102 records vital data transmitted from the sensor device 300. For example, the sensor data recording unit 102 is preferably a memory such as a RAM (Random Access Memory). The moving image data from the imaging device 200 is recorded in synchronization with the time information of the vital data.

  The range setting unit 104 determines the tag assignable time range from the vital data based on the time information of the tag input data. Specifically, the range setting unit 104 sets a continuous time range in which vital data including a tagging time is equal to or greater than a predetermined threshold as a first tag assignable time range. When a heart rate is used as vital data, a time range in which the heart rate at the time of tagging is equal to or greater than a predetermined threshold (for example, heart rate 75) is set as a first tag assignable time range. The user may arbitrarily set the first tag assignable time range.

  The tag control unit 103 sets the closest peak of at least one vital data within the first tag assignable time range as the tag assignable time timing and the timing for tagging the moving image data. Further, when there are a plurality of peaks in the vital data, the timing of the peak closest to the timing of tag placement instruction is set as the timing of tag placement.

  The CPU 105 is a central processing unit, and connects the data input unit 101, the sensor data recording unit 102, the tag control unit 103, and the range setting unit 104 via a data bus, and executes program instructions. Information processing.

  Next, the imaging apparatus 200 includes a CPU 201, a camera 202, an imaging unit 203, an image data processing unit 204, a recording medium 205, and a data input / output unit 206.

  A CPU 201 is a central processing unit, and performs information processing by connecting an imaging unit 203, an image data processing unit 204, and a data input / output unit 206 via a data bus and executing program instructions.

  The camera 202 optically captures an object, converts it into an electrical signal, generates image data, and outputs the captured image data to the imaging unit 203. For example, the camera 202 is preferably a combination of a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary metal-oxide-semiconductor) image sensor and an optical lens.

  The imaging unit 203 buffers the image data captured by the camera 202 and outputs it as moving image data having a predetermined frame rate. Specifically, the imaging unit 203 sets the image data captured by the camera 202 as continuous image data at a predetermined time interval, that is, moving image data.

  The image data processing unit 204 performs image processing on the moving image data output from the imaging unit 203 and stores the moving image data after the image processing in the storage medium 205 via the data input / output unit 206. Specifically, the image data processing unit 204 converts the frame rate instructed by the CPU 201. Further, the image data processing unit 204 may convert the moving image data to an image size designated by the CPU 201. When the moving image data output from the image capturing unit 203 is stored in the recording medium 205 without performing image processing, the image data processing unit 204 reads out the moving image data captured from the recording medium 205, performs image processing, and performs processing. Later moving image data may be stored in the recording medium 205.

  The recording medium 205 is a medium for recording moving image data. For example, the recording medium 205 is preferably a nonvolatile memory such as a flash memory. Further, the recording medium 205 may be removable from the imaging apparatus 200.

  The data input / output unit 206 stores data in the recording medium 205 or reads data from the recording medium 205. For example, the data input / output unit 206 is preferably configured by an interface circuit with the recording medium 205.

  The sensor device 300 includes a CPU 301, a sensor 302, a sensor control unit 303, a sensor data processing unit 304, and a data communication unit 305. The sensor device 300 continuously measures vital data, and continuously transmits the measured vital data to the tagging device 100 and the imaging device 200.

  The CPU 301 is a central processing unit, and is connected to the sensor 302, the sensor control unit 303, the sensor data processing unit 304, and the data communication unit 305 via a data bus, and performs information processing by executing program instructions.

  The sensor 302 measures a vital sign, converts the measurement data into an electrical signal, and outputs the electrical signal to the sensor control unit 303 as vital data. The vital sign to be measured is preferably one of heart rate, respiratory rate, blood pressure, body temperature, brain wave, sweat volume, body movement of the subject, and oxygen saturation of arterial blood, or a combination thereof. Therefore, the sensor 302 is preferably a heart rate monitor, a respirometer, a sphygmomanometer, a thermometer and a pulse oximeter, or an acceleration sensor, or a combination thereof.

  The sensor control unit 303 controls the sensor 302 and amplifies, converts and outputs vital data output from the sensor 302. For example, the sensor control unit 303 is preferably configured by an amplifier circuit, an analog-digital conversion circuit, and the like.

  The sensor data processing unit 304 buffers the vital data output from the sensor control unit 303, converts it into a predetermined format, and outputs it to the data communication unit 305.

  The data communication unit 305 transmits the vital data processed from the sensor data processing unit 304 to the data input unit 101 wirelessly or by wire. The data communication unit 305 is preferably configured by a wireless communication circuit, a wired communication circuit, or an optical communication circuit similar to the data input unit 101.

  When a communication method in which a delay is assumed between the sensor device 300 and the tagging device 100 and the imaging device 200 is used, the sensor data processing unit 304 combines the measured vital data and the time when the vital data is measured. Thus, it is possible to suppress the influence of delay.

  The tag input device 400 includes a CPU 401, a key input unit 402, a key control unit 403, a tag data processing unit 404, and a data communication unit 405.

  The CPU 401 is a central processing unit, and is connected to the key control unit 403, the tag data processing unit 404, and the data communication unit 405 via a data bus, and performs information processing by executing program instructions.

  The key input unit 402 receives an operation for hitting a tag. For example, a switch that accepts a tag hitting instruction signal generated by pressing a button or the like and connects an electric circuit is suitable.

  The key control unit 403 performs various processes on the signal generated by operating the key input unit 402 and outputs a tag input instruction signal. For example, the key control unit 403 performs processing such as electrical signal amplification and noise removal by chattering.

  When the tag data processing unit 404 is a tag hitting operation, the tag data processing unit 404 transmits a tag hitting instruction to the tagging device 100 in the imaging device 200 via the data communication unit 405.

  When a communication method in which a delay is assumed between the tag input device 400 and the tagging device 100 and the imaging device 200 is used, the tag data processing unit 404 accepts a tag hitting operation instead of a tag hitting instruction. You may send the time. Further, the tag data processing unit 404 may transmit the tag hitting instruction in combination with the time when the tag hitting operation is accepted.

  The data communication unit 405 transmits the tag input instruction output from the tag data processing unit 404 to the data input unit 101 wirelessly or by wire. The data communication unit 405 is preferably configured by a wireless communication circuit, a wired communication circuit, or an optical communication circuit similar to the data input unit 101.

  Next, the operation of the tagging device 100 will be described. FIG. 2 is a flowchart showing an example of the operation of the tagging apparatus according to the first embodiment.

  In step S <b> 501, the tag control unit 103 determines whether the data input unit 101 has received a tag input instruction signal from the tag input device 400. If the tag input instruction signal from the tag input device 400 is received, the process proceeds to step S502. If the tag input instruction signal from the tag input apparatus 400 is not received, the process proceeds to step S509.

  In step S502, the range setting unit 104 determines that the level of vital data received from the sensor device 300 at the time corresponding to the input timing of the tag input instruction signal received in step S501 is greater than or equal to a predetermined threshold set in advance. Determine whether or not. If it is equal to or greater than the predetermined threshold, the process proceeds to step S503, and if it is less than the predetermined threshold, the process proceeds to step S506.

  In step S503, the range setting unit 104 extracts, from the vital data received from the sensor device 300, a continuous region whose level is equal to or higher than a predetermined threshold value including the timing of the tag entry instruction received in step S501. The first tag assignable time range is set, and the process proceeds to step S504.

  In step S504, the tag control unit 103 selects the vital data closest to the input timing of the tag input instruction signal received in step S501 from the first tag assignable time range of the vital data set in step S503. A peak is extracted, and the process proceeds to step S505.

  In step S505, the tag control unit 103 sets the peak time extracted in step S504 as the first tag assignable time timing, and adds the first tag to the image at the same time of the moving image data received from the imaging device 200. And proceed to step S507.

  In step S506, the tag control unit 103 assigns a second tag to the image at the same time of the moving image data received from the imaging device 200 at the input timing of the tag hitting instruction signal received in step S501, and in step S507. Proceed to

  In step S507, the tag control unit 103 confirms whether imaging has been completed. If the imaging has not ended, the process returns to step S501. If the imaging has ended, the processing related to tagging ends.

  With the above operation, the tagging apparatus 100 can place a tag on a moving image at an appropriate timing when tagging is desired.

  Next, an example of actual tagging will be described. FIG. 3 is a diagram showing an example of the relationship between vital data and tagging timing. In FIG. 3, the horizontal axis indicates time, and the vertical axis indicates heart rate, which is an example of vital data. FIG. 3 shows an example of a heart rate when watching a game such as sports.

  First, before and after the game is started, spectators experience an increase in heart rate due to tension. Although the heart rate exceeds the predetermined threshold in the range of time t1 to t2, the tag control unit 103 does not hit the tag during t1 to t2 because there is no tag hitting instruction from the tag input device 400. .

  Next, if the tag input instruction timing from the tag input device 400 is A2 in the range from time t3 to time t4, the spectator is excited by an increase in the score of the game being watched and the heart rate increases. And the tag control unit 103 determines that the range of times t3 to t4 when the heart rate exceeds a predetermined threshold is the tag assignable time range.

  Further, the timing A2 is delayed from the timing A2 'which becomes a peak between the times t3 and t4. For example, it may be considered that the spectator is unable to follow a sudden change in the situation, and the button press operation or the like is delayed. In this case, the tag control unit 103 changes the timing at which the tag is actually shot from the timing A2 at which the tag is instructed to the timing A2 'that is the peak of the heart rate between the tag grantable times t3 to t4.

  Next, if the tag input instruction timing from the tag input device 400 is A3 between times t5 and t6, the tag control unit 103 determines that the heart rate exceeds the predetermined threshold at times t5 to t6. The range is determined as a tag assignable time range.

  This timing A3 is ahead of timing A3 'which becomes a peak between times t5 and t6. For example, there may be a case where a spectator prefetches a change in the situation. In this case, the tag control unit 103 changes the timing at which the tag is actually shot from the timing A3 at which tagging is instructed to the timing A3 'that is the peak of the heart rate between the tag assignable times t5 to t6.

  When the game ends at time t7, the heart rate increases due to the reverberation, and the heart rate exceeds a predetermined threshold in the range from time t7 to t8. However, since there is no tag placement instruction from the tag input device 400, the tag control unit 103 does not place a tag between t7 and t8.

  As described above, according to the tagging device of the first embodiment, if the peak value of the vital data within a predetermined period is equal to or greater than a predetermined threshold value before or after the tagging instruction, By setting this timing as the tag hitting timing, the tag can be hit at an appropriate timing even when the tag hitting timing deviates early or late.

(Embodiment 2)
Next, a second embodiment will be described. FIG. 4 is a flowchart illustrating an example of the operation of the tagging apparatus according to the second embodiment.

  In step S <b> 601, the tag control unit 103 determines whether the data input unit 101 has received a tag hitting instruction signal from the tag input device 400. If the tag input instruction signal from the tag input device 400 is received, the process proceeds to step S602. If the tag input instruction signal from the tag input apparatus 400 is not received, the process proceeds to step S608.

  In step S602, the range setting unit 104 has a level of vital data received from the sensor device 300 at a time corresponding to the input timing of the tag input instruction signal received in step S601 equal to or higher than a predetermined threshold value set in advance. Determine whether or not. If it is equal to or greater than the predetermined threshold, the process proceeds to step S603, and if it is less than the predetermined threshold, the process proceeds to step S607.

  In step S603, the range setting unit 104 extracts, from the vital data received from the sensor device 300, a continuous region whose level is equal to or higher than a predetermined threshold value including the timing of the tag hitting instruction received in step S601. The first tag assignable time range is set, and the process proceeds to step S604.

  In step S604, the tag control unit 103 selects the vital data closest to the input timing of the tag input instruction signal received in step S601 from the first tag assignable time range of the vital data set in step S603. The level peak is extracted, and the process proceeds to step S605.

  In step S605, the tag control unit 103 determines the importance level from the peak level extracted in step S604, and proceeds to step S606. Specifically, when the vital data is a heart rate, the threshold is set as a reference 0, and when the peak level range is 0 or more and less than 10, the degree of excitement is low, and the peak level range is 10 or more and less than 20 In this case, the importance is determined according to the peak level range of the vital data, such that the excitement level is high and the excitement level is high when the peak level range is 20 or more. The peak level range is not limited to three levels, and can be arbitrarily set by the user including the range of each peak value.

  In step S606, the tag control unit 103 sets the peak time extracted in step S604 as the first tag assignable time timing, and adds the first tag to the image at the same time of the moving image data received from the imaging device 200. Further, the importance determined in step S605 is added to the first tag as auxiliary information, and the process proceeds to step S608. Here, the importance calculated from the vital data, the importance determined by the tag grantor, tag granter information, and the like can be added to the tag as auxiliary information.

  In step S607, the tag control unit 103 assigns a second tag to the image at the same time of the moving image data received from the imaging device 200 at the input timing of the tag hitting instruction signal received in step S601, and in step S608. Proceed to

  In step S608, the tag control unit 103 confirms whether imaging has been completed. If the imaging has not ended, the process returns to step S601. If the imaging has ended, the processing related to tagging ends.

  With the above operation, the tagging apparatus 100 can place a tag on a moving image at an appropriate timing when tagging is desired.

  Further, according to the tagging device of the second embodiment, when the amount of change in vital data within a predetermined period is equal to or greater than a predetermined threshold, the tag control unit Auxiliary information can be divided into levels and auxiliary information can be added to the tag.

  Note that the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention. For example, the tag input device, the sensor device, the tagging device, and the imaging device may be separate devices or may be integrated devices.

  Further, the tag input device, the sensor device, and the imaging device are not limited to one, and communication may be performed in a one-to-multiple relationship or a multiple-to-multiple relationship. For example, one tag input device and one sensor device may communicate with a plurality of imaging devices. In this case, tag input timing and vital data are shared by a plurality of imaging devices.

  Further, a plurality of tag input devices and a plurality of sensor devices may communicate with one imaging device. In this case, a plurality of tag input timing and vital data may be used in a single imaging device, or tag input may be accepted when a certain number of tag inputs are met within a certain period. good.

  Furthermore, a plurality of tag input devices and a plurality of sensor devices may communicate with a plurality of imaging devices. In this case, the tag input timing and vital data may be shared by a plurality of imaging devices, or a plurality of tag input timings and vital data may all be used by the imaging device. If a certain number or more is met, tag input may be accepted.

  The data associated with the tag is not limited to moving image data, and any data can be applied as long as it is audio or moving image, or data including audio and moving image.

  In the above embodiment, the time range in which the heart rate, which is vital data, exceeds a predetermined threshold is determined as a tag-applicable time range. However, the slope, the amount of change, the time exceeding the threshold, the integrated value, etc. The range may be determined using as an index of changes in vital data.

  In the above embodiment, when the heart rate, which is vital data, is less than a predetermined threshold, the second tag is assigned at the tag input timing time, but is closest to the tag input timing time. You may make it give a 2nd tag to the peak of vital data.

  Moreover, in the said embodiment, although the range setting part 104 determined the tag possible time range from vital data based on the time information of tag input data, it can tag the time range where vital data is more than a threshold value. The presence or absence of tag input data may be detected after setting the time range.

  In the above embodiment, the sensor data recording unit 102, the tag control unit 103, the range setting unit 104, the imaging unit 203, the image data processing unit 204, the data input / output unit 206, the sensor control unit 303, and the sensor data processing unit. 304, the key control unit 403, and the tag data processing unit 404 can be implemented by hardware or software such as ASIC (Application Specific Integrated Circuit). Further, part of the processing may be performed by software, and the rest may be performed by hardware. When implemented by software, a computer system having one or a plurality of CPUs (Central Processing Units) such as a microprocessor may be caused to execute a program related to processing of functional blocks. These programs can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Compact Disc Read Only Memory), CD- R, CD-R / W, DVD-ROM (Digital Versatile Disc Read Only Memory), DVD-R (DVD Recordable), DVD-R DL (DVD-R Dual Layer), DVD-RW (DVD ReWritable), DVD- RAM, DVD + R, DVR + R DL, DVD + RW, BD-R (Blu-ray (registered trademark) Disc Rewritable), BD-RE (Blu-ray (registered trademark) Disc Rewritable), BD-ROM, semiconductor memory (for example, mask ROM) , PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)). The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

DESCRIPTION OF SYMBOLS 100 Tagging apparatus 101 Data input part 102 Sensor data recording part 103 Tag control part 104 Range setting part 200 Imaging device 105, 201, 301, 401 CPU
202 Camera 203 Imaging unit 204 Image data processing unit 205 Recording medium 206 Data input / output unit 300 Sensor device 302 Sensor 303 Sensor control unit 304 Sensor data processing unit 305, 405 Data communication unit 400 Tag input device 402 Key input unit 403 Key control unit 404 Tag data processing unit 500 Tagging system

Claims (5)

A data input unit that accepts tag input data and vital data that is biometric information when adding a tag to video data;
Based on the tag input data and the vital data, a range setting unit that determines a level of the vital data as a first tag grantable time range that is equal to or higher than a preset threshold;
In the first tag assignable time range, a tag control unit having a peak value of the vital data as a first tag assignable time timing;
A tagging device comprising: The tag control unit includes the first tag assignable time timing in the moving image data and
The first tag assigned to the same time of the image, tagging device according to claim 1, characterized in that to add a supplementary information to the first tag in accordance with the level of the peak value of the vital data . An imaging device for imaging a subject and outputting moving image data;
A tag input device that accepts a tag granting instruction of a tag grantor and outputs tag input data; and
A sensor device that measures and outputs the vital data of the tag giver;
A tagging device that adds a tag to the video data based on the tag input data and the vital data;
A data input unit for receiving the tag input data and the vital data;
Based on the tag input data and the vital data, a range setting unit that determines a level of the vital data as a first tag grantable time range that is equal to or higher than a preset threshold;
In the first tag assignable time range, a tag control unit having a peak value of the vital data as a first tag assignable time timing;
A tagging system comprising: Accepts tag input data and vital data that is biometric information when tagging video data,
Based on the tag input data and the vital data, the level of the vital data is determined as a first tag assignable time range above a preset threshold,
In the first tag assignable time range, a peak value of the vital data is detected, and the peak value is set as a first tag assignable time timing.
Tagging method. A data input step for receiving tag input data and vital data which is biometric information, which is a source for attaching a tag to video data;
Based on the tag input data and the vital data, a range setting step for determining a level of the vital data that is equal to or higher than a preset threshold as a first tag grantable time range;
Detecting a peak value of the vital data in the first tag assignable time range;
A tag control step in which the peak value is a first tag assignable time timing;
A tagging program that runs a computer.

Images