JP2021101525A - Recording device, moving image system, recording method and program - Google Patents

Abstract

To provide a recording device, capable of improving convenience for viewing to enjoy in-vehicle camera videos.SOLUTION: A recording device 10 includes: a camera shooting a moving image inside or outside a car; a microphone getting in-vehicle sound while the camera is shooting; a song information acquisition section 100 acquiring song information on a song that is sung while the camera is shooting and a start timing and an end timing in a moving image of the song, from the audio data of the sound acquired by the microphone; and a recorded data generation section 101 generating recorded data obtained by adding the song information, the start timing and the end timing to the video data of a moving image shot by the camera.SELECTED DRAWING: Figure 4

Description

The present invention relates to a recording device, a moving image system, a recording method, and a program.

Various research and development are being carried out on the functions related to the drive recorder. For example, Patent Document 1 discloses a technique of adding BGM (background music) at a tempo according to the traveling speed of a vehicle to a digest video summarizing a drive video.

Further, in recent years, there is a need to use a drive recorder not only for recording an accident but also for enjoying a drive image. As a technique for enjoying an image taken by a camera, for example, there is a technique described in Patent Document 2. In this technique, the image recording device detects the music flowing from the car stereo with a microphone, acquires the music data of the music, and associates the music data with the image data. By doing so, it is possible to associate the music that was playing at the time of shooting with the shot image. Therefore, according to this technique, it is possible to provide an electronic album that can reproduce a photographed image while remembering the atmosphere at the time of photographing.

Japanese Unexamined Patent Publication No. 2016-167771 Japanese Unexamined Patent Publication No. 2006-109151

As mentioned above, in recent years there is a need to enjoy recorded drive images. Therefore, it is required to provide various technologies to meet such needs. In particular, it is required to provide a technology for improving the convenience of viewing the image of the vehicle camera.

An object of the present disclosure is to provide a recording device, a moving image system, a recording method, and a program capable of improving convenience for viewing and viewing images taken by a vehicle camera.

The recording device according to the first aspect is
With a camera that shoots videos inside or outside the car,
A microphone that acquires the sound inside the vehicle during shooting by the camera, and
A song information acquisition unit that acquires song information of a song sung during shooting by the camera and start timing and end timing of the song in the moving image from the audio data of the sound acquired by the microphone.
It has a recording data generation unit that generates recording data in which the song information, the start timing, and the end timing are added to the video data of the moving image taken by the camera.

The moving image system according to the second aspect is
A recording device that generates recording data and
A playback device that acquires the recorded data from the recording device and reproduces the recorded data is provided.
The recording device is
With a camera that shoots videos inside or outside the car,
A microphone that acquires the sound inside the vehicle during shooting by the camera, and
A song information acquisition unit that acquires song information of a song sung during shooting by the camera and start timing and end timing of the song in the moving image from the audio data of the sound acquired by the microphone.
It has a recording data generation unit that generates recording data in which the song information, the start timing, and the end timing are added to the video data of the moving image taken by the camera.
The playback device is
It has a reproduction control unit that controls to output a song based on the song information during reproduction of a section from the start timing to the end timing when the recorded data is reproduced.

In the recording method according to the third aspect,
Acquires the sound inside the vehicle while the camera is shooting the video inside or outside the vehicle. From the audio data of the sound acquired by the microphone, the song information of the song sung during the shooting by the camera and the video of the song. Get the start timing and end timing in
Recording data is generated in which the song information, the start timing, and the end timing are added to the video data of the moving image taken by the camera.

The program according to the fourth aspect is
Acquires the sound inside the vehicle while the camera is shooting the video inside or outside the vehicle. From the audio data of the sound acquired by the microphone, the song information of the song sung during the shooting by the camera and the video of the song. The song information acquisition step to acquire the start timing and end timing in
The computer is made to execute the recording data generation step of generating the recording data in which the song information, the start timing, and the end timing are added to the video data of the moving image taken by the camera.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide a recording device, a moving image system, a recording method, and a program capable of improving convenience for viewing and viewing images taken by a vehicle camera.

It is a schematic diagram which shows an example of the structure of the moving image system which concerns on embodiment. It is a block diagram which shows an example of the hardware composition of the recording apparatus which concerns on embodiment. It is a block diagram which shows an example of the hardware composition of the reproduction apparatus which concerns on embodiment. It is a block diagram which shows an example of the functional structure of the recording apparatus which concerns on embodiment. It is a block diagram which shows an example of the functional structure of the reproduction apparatus which concerns on embodiment. It is a flowchart which shows an example of the tag information setting operation by the recording apparatus which concerns on embodiment. It is a schematic diagram which shows the tag information set in the recorded data. It is a flowchart which shows an example of the reproduction operation by the reproduction apparatus which concerns on embodiment. It is a block diagram which shows an example of the functional structure of the reproduction apparatus which concerns on embodiment. It is a flowchart which shows an example of the setting operation of the tag information about the music information by the recording apparatus which concerns on embodiment. It is a flowchart which shows an example of the setting process of music information by a user. It is a flowchart which shows an example of the process flow about determination of the music to be output. It is a block diagram which shows an example of the hardware composition of the recording apparatus which concerns on embodiment. It is a flowchart which shows an example of the flow of the process of moving the recorded data of the section in which a song is sung to another storage device. It is a block diagram which shows an example of the hardware composition of the reproduction apparatus which concerns on embodiment. It is a block diagram which shows an example of the functional structure of the reproduction apparatus which concerns on embodiment. It is a schematic diagram which shows the reproduction start position of the recorded data. It is a schematic diagram which shows the reproduction start position of the recorded data. It is a flowchart which shows an example of the flow of the operation of determining the reproduction start position of the recorded data. It is a block diagram which shows an example of the functional structure of the recording apparatus which concerns on embodiment. It is a flowchart which shows an example of the tag information setting operation by the recording apparatus which concerns on embodiment.

Hereinafter, embodiments will be described with reference to the drawings. In order to clarify the explanation, the following description and drawings have been omitted or simplified as appropriate.

<Embodiment 1>
FIG. 1 is a schematic diagram showing an example of the configuration of the moving image system 1 according to the first embodiment. The moving image system 1 includes a recording device 10 that generates recorded data, and a playback device 20 that acquires the recorded data from the recording device 10 and reproduces the recorded data. The playback device 20 may acquire the recorded data directly from the recording device 10 by wired or wireless communication, or may acquire the recorded data from the recording device 10 via another device or the like. ..

Hereinafter, the recording device 10 and the playback device 20 will be specifically described. FIG. 2 is a block diagram showing an example of the hardware configuration of the recording device 10. Further, FIG. 3 is a block diagram showing an example of the hardware configuration of the playback device 20. Further, FIG. 4 is a block diagram showing an example of the functional configuration of the recording device 10. FIG. 5 is a block diagram showing an example of the functional configuration of the reproduction device 20.

The recording device 10 is a device provided in a vehicle such as a four-wheeled vehicle, and is a so-called drive recorder. As shown in FIG. 2, the recording device 10 includes a camera 11, a microphone 12, a network interface 13, a storage device 14, and a processor 15.

The camera 11 is a digital camera, and captures a moving image inside or outside the vehicle in which the recording device 10 is mounted. The microphone 12 is a device that collects the sound inside the vehicle, and acquires the sound inside the vehicle during shooting by the camera 11.

The network interface 13 is used to communicate with other devices. The other device may be, for example, a server that acts on behalf of a part of the processing of the recording device 10, or a playback device 20 that receives the recorded data. The network interface 13 may include, for example, a network interface card (NIC).

The storage device 14 includes a memory group including either or both of a volatile memory and a non-volatile memory, and an internal storage such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). In the present embodiment, the memory and the internal storage are collectively referred to as a storage device 14, but the memory and the internal storage do not have to be built in the recording device 10 as one device, but as separate devices. It may be built in the recording device 10.

The internal storage is used, for example, for storing recorded data, various data required for processing of the recording device 10, and the like. The memory is used, for example, to store software (computer program) containing one or more instructions executed by the processor 15.

This program can be stored and supplied to a computer using various types of non-transitory computer readable medium. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media (eg flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg magneto-optical disks), Compact Disc Read Only Memory (CD-ROM), CD- Includes R, CD-R / W, semiconductor memory (eg, mask ROM, Programmable ROM (PROM), Erasable PROM (EPROM), flash ROM, Random Access Memory (RAM)). The program may also be supplied to the computer by various types of transient computer readable medium. Examples of temporary computer-readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

The processor 15 reads software (computer program) from the memory and executes it to process, for example, the components shown in FIG. That is, each process of the song information acquisition unit 100, the recording data generation unit 101, and the storage control unit 102 shown in FIG. 4 may be realized by executing the program. As described above, the recording device 10 has a function as a computer. The processor 15 may be, for example, a microprocessor, an MPU (Micro Processor Unit), a CPU (Central Processing Unit), or the like. The processor 15 may include a plurality of processors.

The playback device 20 is a device that reproduces the recorded data recorded by the recording device 10, and specifically, for example, a smartphone, a tablet terminal, a PC (Personal Computer), or the like. As shown in FIG. 3, the playback device 20 includes a display 21 for displaying a video of recorded data and a speaker 22 for outputting audio of recorded data. The recording device 10 also includes an input device 23, a network interface 24, a storage device 25, and a processor 26.

The input device 23 is a device that receives an operation input of a user who uses the playback device 20, and is, for example, a pointing device.
The network interface 24 is used to communicate with other devices such as the recording device 10 that transmits the recorded data. The network interface 24 may include, for example, a network interface card (NIC).

The storage device 25 includes a memory group and internal storage, like the storage device 14 in the recording device 10. The internal storage is used, for example, to store recorded data acquired from the recording device 10, various data required for processing of the playback device 20, and the like. The memory is used, for example, to store software (computer program) containing one or more instructions executed by the processor 26. This program can also be supplied to the computer using various types of non-transitory computer-readable media and can also be supplied to the computer by various types of temporary computer-readable media.

The processor 26 reads software (computer program) from the memory and executes it to process, for example, the components shown in FIG. That is, each process of the reproduction control unit 200, the instruction reception unit 201, and the song data acquisition unit 202 shown in FIG. 5 may be realized by executing the program. As described above, the playback device 20 also has a function as a computer. The processor 26 may be, for example, a microprocessor, an MPU, a CPU, or the like. The processor 26 may include a plurality of processors.

Next, the functional configuration of the recording device 10 will be described. As shown in FIG. 4, the recording device 10 includes a song information acquisition unit 100, a recording data generation unit 101, and a storage control unit 102. When the user sings a song while driving, the recording device 10 acquires the song information of the song and associates the song information with the video data of the camera 11. At this time, it is specified from which time point to which time point in the video data is the period during which the user actually sang, and the video data and the song information are associated with each other. It should be noted that the song information when the melody of the song is sung without the lyrics, such as a humming song, may be acquired, or the song information when the lyrics are sung may be acquired.

The song information acquisition unit 100 acquires song information of a song sung during shooting by the camera 11 and start timing and end timing in the moving image of the song from the audio data of the sound acquired by the microphone 12. Here, the song information is song identification information. That is, it is information for uniquely identifying a song. In addition, the song information acquisition unit 100 may further acquire position information that specifies the position of a part of the song acquired by the microphone 12 at the start timing in the entire song. That is, when a certain song is sung, the song information acquisition unit 100 starts singing from where in the song (for example, from the beginning of the song, from the chorus, from No. 2 or the like, etc. ) Is acquired. For example, this position information is time information indicating an elapsed time from the start of a song.

Specifically, the song information acquisition unit 100 acquires these in the following manner, for example. The song information acquisition unit 100 continuously monitors the audio data of the sound acquired by the microphone 12 and determines whether or not the user's voice is detected. When the voice is detected, the song information acquisition unit 100 starts accumulating the detected voice audio data in the buffer. When the audio data for a predetermined unit time is accumulated, the song information is analyzed for the audio data for the unit time. In the present embodiment, this analysis process is performed by, for example, a server communicatively connected to the recording device 10. Note that this server performs analysis processing by, for example, executing a program stored in the memory of the server by the processor of the server.

The song information acquisition unit 100 transmits audio data for a unit time to the server. This server uses a known music search technology such as a humming search to search for a song from voice, and stores a song having a music feature similar to the music feature of the received audio data, and stores the music feature for each song. Search by referring to the music database. The song information acquisition unit 100 acquires song information of a song whose similarity with the audio data to be analyzed exceeds a predetermined threshold value (first threshold value) from the server. The recording data generation unit 101, which will be described later, adds song information of a song exceeding the first threshold value to the video data. When a plurality of song information is acquired as song information of a song exceeding the first threshold value, that is, a plurality of candidates are acquired as song information of one song sung during shooting by the camera 11. In this case, the recording data generation unit 101 adds song information of the song having the highest similarity to the video data. However, the recording data generation unit 101 may not select any of them, and may add song information of these a plurality of songs to the video data as a candidate list, as will be described later.

As described above, the song information acquisition unit 100 may acquire position information that identifies where in the song the song is being recorded. This can be done, for example, by specifying on the server which part of the song determined to be similar to the audio data corresponds to the music feature amount of the unit time of the audio data to be analyzed. it can.

Further, the song information acquisition unit 100 acquires the start timing and end timing of the song based on the analysis result of the audio data for each continuous unit time. That is, the song information acquisition unit 100 sets the timing at which the audio data determined to be similar to a certain song by analysis is obtained as the start timing of this song, and the audio data determined to be similar to this song cannot be obtained. The timing is the end timing of this song.

When a song is sung in a plurality of periods during one recording, the song information acquisition unit 100 acquires song information, a start timing, and an end timing for each of the periods. Further, the song information acquisition unit 100 may acquire song information, a start timing, and an end timing for each song that is being sung when different songs are sung in succession.

As described above, in the present embodiment, the analysis process is performed by the server, but it may be performed as the process of the song information acquisition unit 100. That is, all the processes described as the server processes or some of the processes may be performed as the processes in the recording device 10.

The recording data generation unit 101 generates recording data in which the song information acquired by the song information acquisition unit 100, the start timing, and the end timing are added to the video data of the moving image captured by the camera 11. Specifically, the recording data generation unit 101 includes time information indicating the start timing and end timing of the song in the moving image, and the song of the song sung during the period, for each period in which the song is sung. Information is added to the video data as tag information. When the above-mentioned position information for specifying the singing position is acquired by the song information acquisition unit 100, the recording data generation unit 101 further adds this position information as tag information for each song. The recorded data may be generated. As a result, the song can be output from the position indicated by the position information during reproduction by the reproduction device 20 described later.

Further, when the song information acquisition unit 100 acquires a plurality of candidates as song information of one song sung during shooting by the camera 11, the recording data generation unit 101 uses the acquired song information as the song of the song. Recorded data added as a candidate list of information may be generated. That is, the candidate list may be set in the recorded data as the tag information. As a result, as will be described later, the user can select appropriate song information from the song information given as candidates.

The recording data generation unit 101 may generate the recording data including the video data of the moving image taken by the camera 11, but the recording data including the video data and the audio data of the sound acquired by the microphone 12 may be generated. It may be generated.

The storage control unit 102 controls the storage of the recorded data generated by the recording data generation unit 101 in the storage device 14 (internal storage). For example, when the storage device 14 runs out of free space, the storage control unit 102 controls to store new recorded data by overwriting the oldest recorded data in order. The storage control unit 102 may prohibit overwriting of the recorded data from the start timing to the end timing. By doing so, it is possible to prevent the recorded data of the section in which the song is sung from being deleted. That is, it is possible to prevent the image of an impressive scene from being deleted. When the storage capacity of the new recorded data in the storage device 14 falls below a predetermined reference amount, the storage control unit 102 may cancel the overwrite prohibition on the recorded data from the start timing to the end timing. .. Here, the storable capacity of the new recorded data is the total amount of the capacity for which the overwrite process is permitted and the free capacity. The recording device 10 may need to store a more important image, such as an image when a vehicle accident occurs. Therefore, it is not preferable that such an important image cannot be stored when the storage capacity of the new recorded data becomes small. By canceling the overwrite prohibition, the occurrence of such a situation can be suppressed.

Next, the functional configuration of the reproduction device 20 will be described. As shown in FIG. 5, the reproduction device 20 includes a reproduction control unit 200, an instruction reception unit 201, and a song data acquisition unit 202.

The reproduction control unit 200 reproduces the recorded data acquired from the recording device 10 and displays the video of the recorded data on the display 21. Further, when the recorded data includes the audio data of the sound acquired by the microphone 12, the sound of the audio data may be output from the speaker 22.

Further, when the recorded data is reproduced, the reproduction control unit 200 controls to output the song from the speaker 22 based on the song information associated with the section during the reproduction of the section from the start timing to the end timing. Specifically, the reproduction control unit 200 controls to output this song from the speaker 22 by using the audio data of the song acquired based on the song information. The reproduction control unit 200 outputs a song from the speaker 22 using the audio data acquired by the song data acquisition unit 202 during the reproduction of the section. Note that this output may be superimposed on the audio output of the audio data of the sound acquired by the microphone 12. The ratio of the two volumes to be superimposed can be set arbitrarily. Further, the sound acquired by the microphone 12 may be set to be muted.

Further, when the above-mentioned position information is added to the recorded data, the reproduction control unit 200 controls to reproduce and output the song from the position specified based on the position information when outputting the song. You may. That is, the reproduction control unit 200 may output the song so that the singing when the song is actually sung and the singing when the song is reproduced are aligned. By doing so, the situation at the time of recording can be reproduced more faithfully.

Further, when the recorded data includes a plurality of sections from the start timing to the end timing and the interval between adjacent sections is shorter than a predetermined time, the playback control unit 200 outputs the song to be output in one section and the other section. You may control to crossfade the song to be output. That is, the reproduction control unit 200 may output the song so that the preceding song is faded out and the succeeding song is faded in. By doing this, you can enjoy the song more naturally.

Further, when the length of the section from the start timing to the end timing is longer than the length of the song associated with the section, the playback control unit 200 may control the song to be looped and output. .. That is, when the length of this section is longer than the length of the song acquired by the song data acquisition unit 202 described later, the playback control unit 200 is the song acquired by the song data acquisition unit 202 during the reproduction of the section. May be controlled to repeatedly output. By doing so, it is possible to output the song as much as the actual period during which the song was sung, so that the situation at the time of recording can be reproduced more faithfully.

The reproduction control unit 200 may reproduce the entire recorded data, or may perform digest reproduction of the recorded data. When performing digest reproduction, the reproduction control unit 200 may extract a section from the start timing to the end timing of the recorded data and perform digest reproduction. By doing so, it is possible to provide a digest reproduction that extracts an impressive scene.

The instruction receiving unit 201 receives an instruction regarding the output of the song in the section from the start timing to the end timing. The instruction receiving unit 201 receives an instruction input by the user via the input device 23. The reproduction control unit 200 may control to output a song in a section from the start timing to the end timing according to the received instruction. For example, in the recorded data, when a candidate list including a plurality of song information is added to one song in a certain section, the instruction receiving unit 201 selects one of the song information listed in the candidate list. Accept instructions to select. In this case, the reproduction control unit 200 controls to output a song based on the selected song information. That is, the reproduction control unit 200 controls to output this song from the speaker 22 by using the audio data of the song acquired based on the selected song information. According to such a configuration, the user can select appropriate song information from the song information given as candidates, and thus can listen to an appropriate song.

Further, for example, the instruction receiving unit 201 may receive an instruction for designating a playback start position in the song. In this case, the reproduction control unit 200 controls to reproduce and output the song from the designated reproduction start position in the song when the song is output in the section from the start timing to the end timing. For example, when instructed to play from the second song, the playback control unit 200 outputs from the second song. By doing so, the user can listen to the song from the desired playback start position.

The song data acquisition unit 202 acquires the audio data of the song output during the reproduction of the section from the start timing to the end timing of the recorded data. Specifically, the song data acquisition unit 202 acquires the audio data of the original song of the song sung during the recording of the recording device 10. That is, the song data acquisition unit 202 acquires the audio data of the song sung or played by the actual artist. In the present embodiment, the song data acquisition unit 202 acquires audio data having song information that matches the song information given to the recorded data. However, when the song information is selected from the candidate list by the instruction receiving unit 201, the song data acquisition unit 202 acquires audio data having song information that matches the selected song information.

The song data acquisition unit 202 may acquire audio data to be output during reproduction of the section from the audio data group stored in the storage device 25 of the reproduction device 20, or may acquire audio data to be output during reproduction of the section, via the network interface 24. Audio data may be acquired from other devices that are communicably connected by wire or wirelessly. For example, audio data distributed by network streaming may be acquired. As described above, the reproduction control unit 200 uses the audio data acquired by the song data acquisition unit 202 to output the song during the reproduction of the recorded data from the start timing to the end timing.

Next, an example of the operation of the moving image system 1 will be described with reference to a flowchart. FIG. 6 is a flowchart showing an example of the tag information setting operation by the recording device 10. Hereinafter, an example of the tag information setting operation will be described with reference to FIG.

In step S100, the song information acquisition unit 100 continuously monitors the audio data of the sound acquired by the microphone 12 and determines whether or not the user's voice is detected. If no voice is detected (No in step S100), this step continues. When the voice is detected (Yes in step S100), the process proceeds to step S101.

In step S101, the song information acquisition unit 100 starts accumulating the detected audio audio data in the buffer.

Next, in step S102, the song information acquisition unit 100 determines whether or not the voice is no longer detected. If the voice is detected (No in step S102), the process proceeds to step S103, and if the voice is no longer detected (Yes in step S102), the process proceeds to step S109.

In step S103, the song information acquisition unit 100 determines whether or not audio data for a predetermined unit time has been accumulated. If the audio data for a unit time is accumulated (Yes in step S103), the process proceeds to step S104, and if not (No in step S103), the process returns to step S102.

In step S104, the song information acquisition unit 100 acquires song information using the accumulated audio data. Specifically, as described above, the song information acquisition unit 100 acquires the song information obtained by the song search using the audio data. The song information acquisition unit 100 may acquire the above-mentioned position information, which is information about which position of the song the audio data corresponds to.

Next, in step S105, the song information acquisition unit 100 determines whether or not the tag information of the start timing has been set. Considering that a plurality of sections specified by the start timing and the end timing can be set for one recorded data, more specifically, the determination of this step is not yet paired with the tag information of the end timing. It is a determination as to whether or not the tag information of the start timing that has not been set is set. Therefore, even if one or more pairs of start timing and end timing tag information are already set, such start timing tag information is ignored and the start timing tag information has already been set. Whether or not it is determined. If the start timing tag information is not set (No in step S105), the process proceeds to step S106. On the other hand, when the tag information of the start timing is set (Yes in step S105), the process proceeds to step S107.

In step S106, the song information acquisition unit 100 sets the acquisition timing of the audio data used in the processing of step S104 as the start timing. In this way, the song information acquisition unit 100 acquires the start timing. Then, the recording data generation unit 101 sets the tag information of the start timing in the recording data as the tag information at the recording time corresponding to the acquisition timing of the audio data. Further, the recording data generation unit 101 also sets the tag information of the song information obtained in step S104 in the recording data as the tag information at the recording time corresponding to the acquisition timing of the audio data. When the position information is also acquired in step S104, the recording data generation unit 101 also uses the tag information of the position information as the tag information at the recording time corresponding to the acquisition timing of the audio data. Set.

After step S106, the process returns to step S102, and the above-described process is repeated. That is, when the voice continues, the music is searched for the audio data for the next unit time. As described above, when the voice is no longer detected (Yes in step S102), the process proceeds to step S109.

In step S109, the song information acquisition unit 100 sets the timing at which the sound is no longer detected as the end timing. In this way, the song information acquisition unit 100 acquires the end timing. That is, the recording data generation unit 101 sets the tag information of the end timing in the recording data as the tag information at the recording time corresponding to the timing when the voice is no longer detected. Then, the accumulation of audio data is completed. The end timing tag information is set when only the start timing tag information is set.

On the other hand, in step S107, it is determined whether or not the song information searched for the previous audio data is different from the song information searched for the current audio data. That is, it is determined whether or not the searched song has changed compared to the previous time. As a result, even if the song to be sung changes, tag information can be appropriately added. When the searched song has changed from the previous time (Yes in step S107), the process proceeds to step S108. If the searched song has not changed from the previous time (No in step S107), the process returns to step S102, and the above-described process is repeated. That is, when the voice continues, the music is searched for the audio data for the next unit time.

In step S108, the song information acquisition unit 100 sets the acquisition timing of the audio data used in the processing of step S104 as the end timing. In this way, the song information acquisition unit 100 acquires the end timing. That is, the recording data generation unit 101 sets the tag information of the end timing in the recording data as the tag information at the recording time corresponding to the acquisition timing of the audio data. After step S108, the process proceeds to step S106.

By the above processing, tag information as shown in FIG. 7, for example, is set for the recorded data.

Next, an example of the operation of the reproduction device 20 will be described. FIG. 8 is a flowchart showing an example of the reproduction operation by the reproduction device 20. Hereinafter, an example of the reproduction operation will be described with reference to FIG.

In step S200, the reproduction control unit 200 determines whether or not the reproduction of the video is completed. When the reproduction is finished (Yes in step S200), the process ends. When the reproduction is continued (No in step S200), the process proceeds to step S201.

In step S201, the reproduction control unit 200 determines whether or not the tag information of the start timing has been detected. That is, the reproduction control unit 200 determines whether or not the start timing has arrived for the recorded data currently being reproduced. If the start timing has not arrived (No in step S201), the process returns to step S200. When the start timing arrives (Yes in step S201), the process proceeds to step S202.

In step S202, the song data acquisition unit 202 acquires the audio data of the song based on the song information added as a tag to the recorded data.

Next, in step S203, the reproduction control unit 200 outputs a song using the audio data acquired in step S202. That is, the reproduction control unit 200 reproduces the recorded data by superimposing the song on the video.

Next, in step S204, the reproduction control unit 200 determines whether or not the tag information of the end timing is detected. That is, the reproduction control unit 200 determines whether or not the end timing has arrived for the recorded data currently being reproduced. If the end timing has not arrived (No in step S204), the playback of the song is continued.

On the other hand, when the end timing has arrived (Yes in step S204), the process proceeds to step S205, and in step S205, the reproduction control unit 200 ends the reproduction of the song. After step S205, the process returns to step S200.

The first embodiment has been described above. According to the recording device 10 described above, when a song is sung during recording, information for specifying in which section of the recorded data the song was sung (that is, start timing and end timing) and which song is sung. Information about Taka (song information) can be recorded. Therefore, the user can listen to the song at the timing when the song is actually sung when the image of the camera of the vehicle is reproduced. Therefore, the user can enjoy the image of the camera of the vehicle. Therefore, it is possible to improve the convenience for viewing the video.

In the above-described embodiment, the song information is the song identification information, but the song information can also be used as the audio data of the song. In this case, the recording data generation unit 101 may generate recording data in which the video data and the audio data of the song are superimposed for the section from the start timing to the end timing. By doing so, even in a general playback device, when the recorded data is played back, the song can be output to the section in which the song is sung.

When the recording device 10 generates the recording data in which the video data and the audio data of the song are superimposed, the recording data generation unit 101 may generate two types of recording data. That is, the recording data generation unit 101 is the first recording data which is the recording data in which the video data and the audio data of the song are superimposed, and the second recording data which is the recording data of the video data in which the audio data of the song is not superimposed. Recorded data may be generated. Since superimposing the audio data of the song means that the drive image is processed, the value as the drive image may be lost. Therefore, such a situation can be avoided by creating two types of recorded data.

Further, in the above-described embodiment, the song analysis process is performed at the same time as the recording, but it does not necessarily have to be performed during the recording.

<Embodiment 2>
In the above-described embodiment, when a song whose similarity with the audio data to be analyzed exceeds a predetermined threshold value (first threshold value) is not found, a song corresponding to the interval from the start timing to the end timing can be determined. Can not. Therefore, in such a case, the song cannot be output when the section is reproduced. Therefore, in the present embodiment, a technique capable of outputting a song even in such a case is provided. Hereinafter, points different from the above-described embodiment will be described.

In the present embodiment, the song information acquisition unit 100 sets a predetermined second threshold value lower than the first threshold value when a song whose similarity with the audio data to be analyzed exceeds the first threshold value is not found. Get song information for songs that exceed. That is, when there is a song whose similarity with the song sung during the shooting of the camera 11 exceeds a predetermined first threshold value, the song information acquisition unit 100 uses the song information of the song with the camera 11. Acquire as song information of the song sung during shooting. Then, the song information acquisition unit 100 determines that if there is no song whose similarity with the song sung during the shooting by the camera 11 exceeds a predetermined first threshold value, the song information acquisition unit 100 has a song whose similarity exceeds the second threshold value. The song information of the song is acquired as a candidate for the song information of the song sung during the shooting of the camera 11.

Further, in the present embodiment, when the song information of the song exceeding the second threshold value is acquired, the recording data generation unit 101 sings the song information of the song exceeding the second threshold value during the shooting of the camera 11. Generates recorded data added as a candidate list of song information of the song. That is, the recording data generation unit 101 sets the candidate list including the song information of the song exceeding the second threshold value as the tag information in the recording data.

Then, in the playback device 20 of the present embodiment, when there is no song exceeding the first threshold value, that is, when song information of the song exceeding the first threshold value cannot be obtained, the instruction receiving unit 201 is set to the first. The user receives an instruction to select one of the song information listed in the candidate list including the song information of the song exceeding the threshold value of 2. Then, when there is no song exceeding the first threshold value, that is, when it is desired to obtain song information of the song exceeding the first threshold value, the playback control unit 200 starts based on the song information selected by the instruction. Controls to output songs in the section from the timing to the end timing. Therefore, the song data acquisition unit 202 acquires the audio data of the song having the song information selected by the instruction. When the song information of the song exceeding the first threshold value is obtained, the playback control unit 200 from the start timing to the end timing based on the song information as described in the first embodiment. Controls to output the song of the section.

The instruction receiving unit 201 receives instructions in advance, for example, before playing back the recorded data. In the present embodiment, the instruction receiving unit 201 sets the song information specified by the received instruction as tag information in the recorded data.

In order to have the user instruct the song information, a configuration for calling attention may be provided. That is, as shown in FIG. 9, the reproduction device 20 may further have a warning unit 203. The processing of the alert unit 203 is realized, for example, by executing a program by the processor 26. The attention alerting unit 203 alerts the user when there is recorded data in which the song to be output is not specified during the reproduction of the section from the start timing to the end timing. For example, the alerting unit 203 may display a display prompting the user to specify the song information when the user tries to play back the recorded data in which the song is not specified. Further, the caution unit 203 may display a mark indicating that the recorded data for which the song is not specified is the recorded data for which the song is not specified, on the reproducible recorded data list screen. By doing so, it is possible to prompt the user to specify a song.

When the reproduction of the recorded data for which the song to be output is not specified during the reproduction of the section from the start timing to the end timing is started, the reproduction control unit 200 outputs a predetermined song during the reproduction of the section. It may be controlled as follows. That is, when the recorded data is played back without the user specifying the song, a predetermined song such as a recommended song may be played back. In particular, as this predetermined song, a song specified by any of the song information listed in the candidate list may be used. By doing so, even if the song is not specified, the song can be output in the section.

Further, the instruction receiving unit 201 may accept an instruction to specify a song when there is no song exceeding the second threshold value. When there is no song exceeding the second threshold value, the song to be output during playback of the section from the start timing to the end timing cannot be specified because the above-mentioned candidate list is not set in the recorded data. Therefore, in such a case, by receiving an instruction to specify a song from the user, it is possible to specify the song to be output during the reproduction of the section. In this case, the reproduction control unit 200 controls to output the song specified by the instruction received by the instruction reception unit 201 during the reproduction of the section.

Next, an example of the operation in the present embodiment will be described with reference to a flowchart. FIG. 10 is a flowchart showing an example of a tag information setting operation related to song information by the recording device 10. Hereinafter, an example of the tag information setting operation related to the song information in the present embodiment will be described with reference to FIG.

In step S300, the recording data generation unit 101 determines whether or not a song whose similarity exceeds the first threshold value has been searched. That is, the recording data generation unit 101 determines whether or not the song information acquisition unit 100 has acquired the song information of the songs whose similarity exceeds the first threshold value. If a song whose similarity exceeds the first threshold value is searched (Yes in step S300), the process proceeds to step S301, and if not (No in step S300), the process proceeds to step S303.

In step S301, the recording data generation unit 101 sets the song information of the song having the highest similarity as tag information in the recording data. That is, when a plurality of songs having a similarity exceeding the first threshold value are found, the recording data generation unit 101 sets the song information of the song having the highest similarity as tag information in the recording data. After step S301, the process proceeds to step S302.

In step S302, when there are a plurality of songs exceeding the first threshold value, the recording data generation unit 101 sets the song information in the recording data as tag information of the candidate list. As a result, as described in the first embodiment, the user can select appropriate song information from the song information given as candidates. When step S302 ends, the setting of tag information related to song information ends.

On the other hand, in step S303, the recording data generation unit 101 sets the tag information indicating that there is no song corresponding to the search condition, that is, the tag information indicating that there is no song exceeding the first threshold value in the recording data. .. After step S303, the process proceeds to step S304.

In step S304, when there are songs exceeding the second threshold value, the recording data generation unit 101 sets the song information in the recording data as tag information of the candidate list. That is, when the song information acquisition unit 100 acquires the song information of the songs whose similarity exceeds the second threshold value, the recording data generation unit 101 records the song information of such songs as the tag information of the candidate list. Set in the data. As a result, the user can select the desired song information even when the song corresponding to the search condition is not found.

FIG. 11 is a flowchart showing an example of music information setting processing by the user. Hereinafter, an example of the song information setting process by the user in the present embodiment will be described with reference to FIG. For example, the user performs an operation of designating a song for the recorded data to which tag information indicating that there is no song exceeding the first threshold value is added.

In step S400, the instruction receiving unit 201 determines whether or not the candidate list is set in the recorded data. If the candidate list is set (Yes in step S400), the process proceeds to step S401. On the other hand, when the candidate list is not set (No in step S400), the process proceeds to step S403.

In step S401, the instruction receiving unit 201 presents the user with a list of song information listed in the candidate list. After step S401, the process proceeds to step S402.

In step S402, the instruction receiving unit 201 receives an instruction from the user to select any one of the song information listed in the candidate list. Then, the instruction receiving unit 201 newly sets the song information specified by this instruction as the tag information of the song information in the recorded data. This completes the song information setting process.

On the other hand, in step S403, the instruction receiving unit 201 receives the input of the song information from the user. Then, the instruction receiving unit 201 newly sets the input song information as the tag information of the song information in the recorded data. This completes the song information setting process.

The second embodiment has been described above. According to the present embodiment, when a song corresponding to the search condition is not found, the user can select desired song information. Therefore, even if a song corresponding to the search condition is not found, the song can be output when the recorded data is played back.

<Embodiment 3>
In the above-described embodiment, when the audio data of the song specified by the song information set in the recorded data by the recording device 10 cannot be acquired, the playback device 20 reproduces the section from the start timing to the end timing of the song in the recorded data. At that time, the song cannot be output. Therefore, in the present embodiment, there is provided a technique that enables playback of a song even when the audio data of the song specified by the song information set in the recording data by the recording device 10 cannot be acquired. Hereinafter, points different from the above-described embodiment will be described.

When the song data acquisition unit 202 according to the present embodiment cannot acquire the audio data of the song specified by the song information set in the recording data by the recording device 10, the song data acquisition unit 202 is associated with the song specified by the song information. The songs for which data can be acquired are candidates for songs to be output during playback in the section from the start timing to the end timing. That is, the song data acquisition unit 202 creates a related song list which is a list of songs related to the song specified by the song information and for which audio data can be acquired. The songs listed in this related song list are candidates for songs to be output when the recorded data is played back. The related song list may be referred to as a candidate list. Then, when the playback control unit 200 cannot acquire the audio data of the song specified by the song information set in the recording data by the recording device 10, one of the songs included in the above-mentioned candidates is selected during the playback of the section. Control to output. That is, the reproduction control unit 200 outputs any of the songs listed in the related song list.

Here, the related song means, for example, a song having the same predetermined attributes regarding the song. Specifically, for example, the related songs may be songs recorded in the same album, songs of the same artist, songs of the same genre, or artists. The songs may belong to the same category, or may be songs of the same release year. It should be noted that these are merely examples, and related songs may be defined according to other predetermined related criteria. The song data acquisition unit 202 identifies related songs by, for example, referring to a database about songs.

The playback control unit 200 may output a song automatically selected from the songs listed in the related song list, or may output the selected song according to an instruction from the user. When automatically selected, for example, the playback control unit 200 is among the songs included in the related song list (that is, among the songs included in the candidates) during the reproduction of the section from the start timing to the end timing of the recorded data. ), Controls to output the song selected according to a predetermined selection rule. At this time, the song data acquisition unit 202 acquires the audio data of the selected song, and the playback control unit 200 outputs the song using this audio data. Any rule can be used as the predetermined selection rule. The predetermined selection rule may be, for example, a rule of randomly selecting a song, or a rule of selecting the song with the latest release time. By selecting automatically, it is possible to save the trouble of specifying by the user.

Further, when selecting according to an instruction from the user, for example, the instruction receiving unit 201 gives an instruction from the user to specify one of the songs included in the related song list (that is, among the songs included in the candidates). Accept. Then, the reproduction control unit 200 controls to output the song designated by the instruction during the reproduction of the section from the start timing to the end timing of the recorded data. At this time, the song data acquisition unit 202 acquires the audio data of the designated song, and the playback control unit 200 outputs the song using the audio data. By selecting a song according to the user's instruction, the user can listen to the desired song.

Further, also in the present embodiment, a configuration for calling attention may be provided in order to encourage the user to select a song. That is, the reproduction device 20 may have a warning unit 203. In this case, the alert unit 203 alerts when the audio data of the song specified by the song information set in the recording data by the recording device 10 cannot be acquired. For example, the alert unit 203 may display a display prompting the user to select a song from the related song list when the user tries to play back the recorded data in which the song is not selected. In addition, the alert unit 203 may display a mark indicating that on the reproducible recording data list screen with respect to the recording data in which a song for which audio data cannot be acquired is set. By doing so, it is possible to prompt the user to select a song.

When the instruction receiving unit 201 receives an instruction to specify one of the songs included in the related song list, the song data acquisition unit 202 provides information indicating the location where the audio data of the specified song exists. You may want to save it. Further, when the song data acquisition unit 202 can acquire the audio data of the song specified by the song information set in the recording data by the recording device 10, the song data acquisition unit 202 stores information indicating the location where the audio data of the song exists. It may be stored in 25. Further, the song data acquisition unit 202 may set information indicating the location where the audio data exists in the recorded data as tag information. Here, the information indicating the location where the audio data of the song exists may be, for example, a file path, a URL (Uniform Resource Locator) of a network streaming service that distributes the song, or a download. It may be a URL for doing so. In this way, by temporarily storing the information indicating the location where the audio data of the song exists, it is possible to easily acquire the audio data when the recorded data is repeatedly reproduced.

Next, in the present embodiment, an example of the processing flow for determining the song to be output in the section from the start timing to the end timing when the recorded data is reproduced will be described. FIG. 12 is a flowchart showing an example of a processing flow for determining a song to be output. Hereinafter, an example of the processing flow for determining the song to be output will be described with reference to FIG.

The song data acquisition unit 202 searches for audio data of a song specified by the song information set in the recording data by the recording device 10. Specifically, the following steps S500 to S502 are performed.

In step S500, the song data acquisition unit 202 searches the playback device 20. Specifically, the song data acquisition unit 202 searches the music library stored in the storage device 25. If the audio data to be searched does not exist in the playback device 20 (No in step S500), the process proceeds to step S501. On the other hand, when the audio data to be searched exists in the playback device 20 (Yes in step S500), the process proceeds to step S507.

In step S501, the song data acquisition unit 202 searches for the audio data to be searched via the network. In this step, the song data acquisition unit 202 searches for data available to the user on the network. The song data acquisition unit 202 may, for example, search the online storage on the cloud, or may search the audio data provided by the music distribution service available to the user. At this time, the song data acquisition unit 202 may access these connection destinations by using the user account of the playback device 20 or the like. If the audio data to be searched is not available via the network (No in step S501), the process proceeds to step S502. On the other hand, when the audio data to be searched is available via the network (Yes in step S501), the process proceeds to step S507.

Step S502 searches for whether or not the audio data to be searched exists in the audio data provided by the music distribution service that is not currently available to the user. Here, the music distribution service that is not currently available to the user specifically means, for example, a service for which the user has not made a contract or purchase procedure for providing the service with the music distribution service. If the audio data to be searched can be provided by a music distribution service that is not currently available (Yes in step S502), the process proceeds to step S503, and if not (No in step S502), the process is step S505. Move to.

In step S503, the song data acquisition unit 202 presents on the display 21 information about a music distribution service that is a music distribution service that can provide audio data to be searched and is not currently available to the user. That is, by using such a music distribution service, the user is notified that the audio data can be acquired. This encourages streaming distribution contracts or download purchase procedures with such music distribution services. As described above, when the song data acquisition unit 202 needs to provide audio data from a music distribution service that is not currently available to the user in order to acquire the audio data of the song specified by the song information, the song data acquisition unit 202 is concerned. Information about music distribution services may be presented. By doing so, it is possible to output a song specified by the song information. After step S503, the process proceeds to step S504.

In step S504, the song data acquisition unit 202 determines whether or not a music distribution service capable of providing the audio data to be searched has become available. If the music distribution service is not available (No in step S504), the process proceeds to step S505. On the other hand, when the music distribution service becomes available (Yes in step S504), the process proceeds to step S507.

In step S505, the song data acquisition unit 202 creates a list of songs (that is, a list of related songs) that are related to the songs specified by the song information and for which audio data can be acquired. After step S505, the process proceeds to step S506.

In step S506, the instruction receiving unit 201 receives an instruction from the user to select a song from the related song list. After step S506, the process proceeds to step S507.

In step S507, the reproduction control unit 200 determines a song to be output during reproduction of the section from the start timing to the end timing. When the audio data to be searched exists in the playback device 20 (Yes in step S500) and available via the network (Yes in step S501 or Yes in step S504), the playback control unit 200 records. The song specified by the song information set in the data is the song to be output. On the other hand, when designated from the related song list (step S506), the playback control unit 200 sets the designated song as the output target song. In addition, with the determination of the song to be output, the reproduction control unit 200 may update the song information set as the tag information in the recorded data to the song information of the determined song. That is, the reproduction control unit 200 may update the tag information.

The third embodiment has been described above. According to this embodiment, a related song list is created. Therefore, even when the audio data of the song specified by the song information set in the recording data by the recording device 10 cannot be acquired, the song can be reproduced.

<Embodiment 4>
In the above-described embodiment, when the free space of the storage device 14 is exhausted, new recorded data is controlled to be stored by overwriting the oldest recorded data in order. Therefore, the overwriting process may delete the recorded data of the section in which the song is sung. In order to prevent this, as described above, it is possible to prohibit overwriting the recorded data of the section in which the song is sung. However, in that case, as the amount of recorded data for which overwriting is prohibited increases in the storage device 14, the storable capacity of the new recorded data decreases, and the image becomes more like an image at the time of a vehicle accident. There is a risk that important images cannot be memorized. In order to prevent this, as described above, it is possible to cancel the overwrite prohibition when the storage capacity of the new recorded data becomes low. However, in that case, there is a risk that the recorded data of the section in which the song is sung will be deleted again. As described above, in the above-described embodiment, it may be difficult to save the recorded data of the section in which the song is sung and the more important recorded data. Therefore, in the present embodiment, a technique capable of appropriately storing recorded data is provided. Hereinafter, points different from the above-described embodiment will be described.

FIG. 13 is a block diagram showing an example of the hardware configuration of the recording device 10 according to the present embodiment. The configuration shown in FIG. 13 is different from the configuration shown in FIG. 2 in that it further includes an external storage interface 16 to which removable media 19 such as a memory card and a USB (Universal Serial Bus) memory are connected. The external storage interface 16 is an interface circuit that communicably connects the removable media 19 that functions as the external storage of the recording device 10 and the recording device 10.

The storage control unit 102 in the present embodiment controls to move the data of the section from the start timing to the end timing of the recorded data, that is, the data of the section in which the song is sung, from the storage device 14 to another storage device. .. More specifically, when the free space of the other storage device described above is equal to or larger than the size of the data in the section, the storage control unit 102 moves the data in the section from the storage device 14 to the other storage device. Here, the other storage device refers to any external storage other than the storage device 14, specifically, the removable media 19 or the online storage connected via the network interface 13. You may. When online storage is used as another storage device, the external storage interface 16 does not necessarily have to be provided.

Next, in the present embodiment, an example of the flow of the operation of moving the recorded data of the section in which the song is sung to another storage device will be described. FIG. 14 is a flowchart showing an example of the flow of processing for moving the recorded data of the section in which the song is sung to another storage device. Hereinafter, an example of the flow of movement of recorded data will be described with reference to FIG.

In step S600, the storage control unit 102 determines whether or not the recorded data including the section in which the song is sung is stored in the storage device 14. That is, the storage control unit 102 determines whether or not the recorded data in which the tag information of the start timing and the tag information of the end timing are set is stored in the storage device 14. If there is no such recorded data, the storage control unit 102 repeats this step after a predetermined time (No in step S600). On the other hand, when such recorded data is stored in the storage device 14 (Yes in step S600), the process proceeds to step S601.

In step S601, the storage control unit 102 sets the section portion of the recorded data including the section in which the song is sung, which is found in step S600, as the movement target. That is, the storage control unit 102 targets the partially recorded data corresponding to the section portion of the recorded data including the section in which the song is sung to another storage device.

In step S602, the storage control unit 102 determines whether or not there is another storage device that is accessible and has sufficient free space. Here, sufficient free space means that there is free space larger than the size of the data to be moved. When a plurality of other storage devices may exist, the storage control unit 102 checks in order whether or not all of them are accessible and have sufficient free space. If the other storage device is accessible and has sufficient free space (Yes in step S602), the process proceeds to step S603. On the other hand, when there is no other storage device that is accessible and has sufficient free space (No in step S602), the processing of step S600 and subsequent steps is performed again after a predetermined time.

In step S603, the storage control unit 102 moves the data to be moved in step S601 from the storage device 14 to another storage device that is accessible and has sufficient free space. As a result, the storage capacity of the storage device 14 is restored by the size of the data to be moved. After step S603, after a predetermined time, the processes after step S600 are performed again.

The fourth embodiment has been described above. As described above, in the present embodiment, the storage control unit 102 moves the data of the section in which the song is sung from the storage device 14 in which the recorded data is stored to another storage device. As a result, it is possible to prevent the data of the section in which the song is sung from being overwritten and deleted in the storage device 14, and at the same time, a more important image such as an image at the time of a vehicle accident is stored. The storage capacity for this can be secured.

<Embodiment 5>
In the present embodiment, a technique for improving the convenience of designating the playback start position of the recorded data is provided. Specifically, when the user sings a song, the playback device 20 moves the playback position of the recorded data to a position corresponding to the song. Hereinafter, points different from the above-described embodiment will be described.

FIG. 15 is a block diagram showing an example of the hardware configuration of the playback device 20 according to the present embodiment. The reproduction device 20 shown in FIG. 15 is different from the configuration shown in FIG. 3 in that a microphone 27 is added. The microphone 27 is a device that collects sounds around the playback device 20, and acquires the voice of the user of the playback device 20.

FIG. 16 is a block diagram showing an example of the functional configuration of the reproduction device 20 according to the present embodiment. The reproduction device 20 shown in FIG. 16 is different from the configuration shown in FIG. 5 in that the music information acquisition unit 204 is added. The processing of the song information acquisition unit 204 is realized, for example, by executing a program by the processor 26. The reproduction device 20 of the present embodiment may further have a warning unit 203 as in FIG. 9.

The song information acquisition unit 204 acquires song information (for example, identification information) of a song sung by a user who uses the playback device 20 from the audio data of the sound acquired by the microphone 27. In addition, the song information acquisition unit 204 may further acquire position information that specifies the position of a part of the song corresponding to the user's singing in the entire song. That is, when a song is sung, the song information acquisition unit 204 starts singing from where in the song (for example, from the beginning of the song, from the chorus, or from No. 2). You may acquire the position information to specify. The song information acquisition unit 204 can acquire the song information and the position information by the same processing as the song information acquisition unit 100 in the recording device 10. Therefore, the song information acquisition unit 204 acquires the song information and the position information of the song whose similarity with the audio data to be analyzed exceeds a predetermined threshold value from the server. In the acquisition of the song information and the position information in the playback device 20, the analysis process may be performed as the processing of the song information acquisition unit 204 (that is, the playback device 20) instead of the server.

In the playback control unit 200 according to the present embodiment, when the song information acquisition unit 204 acquires the song information, among the song information added to the recorded data, the song information acquisition unit 204 of the playback device 20 has acquired the song. The section associated with the song information corresponding to the information is set as the playback start position. Then, the reproduction control unit 200 reproduces the recorded data from the reproduction start position with the output of the song. FIG. 17 is a schematic diagram showing a reproduction start position of the recorded data. As shown in FIG. 17, for example, in a certain recorded data, it is assumed that a section to which the song A is associated and a section to which the song B is associated are set. In this case, when the song information acquisition unit 204 of the playback device 20 acquires the song information of the song A, the playback control unit 200 starts playback which is the start timing of the section of the song A (specifically, for example, the section of the song A). Playback of the recorded data is started from the position P). As a result, the user can easily start viewing the portion where the desired song is sung.

In the above-mentioned control of the playback start position, the skip to the beginning of the section in which the song is sung is performed, but by using the position information, it is possible to skip to the middle of the section in which the song is sung. That is, the recorded data includes the singing position information (hereinafter referred to as the first position information), and the song information acquisition unit 204 of the playback device 20 starts singing position information (hereinafter referred to as the second position information). When the above is acquired, the reproduction control unit 200 may adjust the reproduction start position of the recorded data according to the difference between the first position information and the second position information. This will be specifically described with reference to FIG. FIG. 18 is a schematic view showing a reproduction start position of the recorded data. As shown in FIG. 18, for example, in a certain recorded data, it is assumed that a section to which the song A is associated and a section to which the song B is associated are set. In particular, it is assumed that the start timing of the section to which the song A is associated is associated with the position information indicating the position where 1 minute and 15 seconds have passed from the beginning of the song as the first position information. That is, in this case, when the playback device 20 reproduces the section in which the song is sung from the start timing, the song is played from the middle of the song (more specifically, from the position where 1 minute and 15 seconds have passed from the beginning of the song). Is output. Here, the song information acquisition unit 204 of the playback device 20 acquires the song information of the song A, and further acquires the position information indicating the position where 2 minutes and 15 seconds have passed from the beginning of the song A as the second position information. Suppose you did. In this case, the playback control unit 200 reads the recorded data from the playback start position P, which is the time when 1 minute (= [2 minutes 15 seconds]-[1 minute 15 seconds]) has elapsed from the beginning of the section in which the song A is sung. Start playback. Thereby, the reproduction start position can be determined in more detail.

Further, in the present embodiment, the user may instruct whether or not the reproduction from the reproduction start position based on the acquired song information may be performed. That is, when the instruction receiving unit 201 specifies a section of recorded data associated with the song information corresponding to the song information acquired by the song information acquisition unit 204 of the playback device 20, the section is set as the playback start position. You may accept instructions that allow you to do so. In this case, the reproduction control unit 200 sets the section as the reproduction start position when the permission instruction is obtained. According to such a configuration, the user can decide whether or not to control the playback start position based on the song information.

Further, when a plurality of sections of recorded data associated with the song information corresponding to the song information acquired by the song information acquisition unit 204 of the playback device 20 are specified, the instruction receiving unit 201 starts reproducing any section. You may accept an instruction to select the position. In this case, the reproduction control unit 200 sets the section selected by the instruction as the reproduction start position. According to such a configuration, a more appropriate section can be reproduced.

The instruction receiving unit 201 may display a list of time information or thumbnail images at the playback start position based on the song information on the display 21 in order to clearly indicate to the user the playback start position based on the song information.

Next, an example of the flow of the operation of determining the reproduction start position in the present embodiment will be described. FIG. 19 is a flowchart showing an example of the flow of the operation of determining the reproduction start position of the recorded data. Hereinafter, an example of the flow of the operation of determining the reproduction start position will be described with reference to FIG.

In step S700, the reproduction control unit 200 determines whether or not the song information acquisition unit 204 has acquired the song information. If the song information has not been acquired (No in step S700), this process is repeated. When the song information is acquired (Yes in step S700), the process proceeds to step S701.

In step S701, the reproduction control unit 200 determines whether or not the recorded data to be reproduced includes a section in which the song information acquired by the song information acquisition unit 204 is set. If the recorded data to be reproduced does not have a section in which the song information is set (No in step S701), the process returns to step S700. On the other hand, when the recorded data to be reproduced has a section in which the song information is set (Yes in step S701), the process proceeds to step S702.

In step S702, the reproduction control unit 200 determines whether or not the instruction receiving unit 201 has received a predetermined instruction. Specifically, the reproduction control unit 200 determines whether or not an instruction for permitting reproduction from the reproduction start position based on the acquired song information has been accepted. Further, when the recorded data to be played back has a plurality of sections in which the acquired song information is set, has the playback control unit 200 received an instruction to select which section is to be the playback start position? Judge whether or not. If such an instruction is not accepted (No in step S702), the process returns to step S700. On the other hand, when such an instruction is accepted (Yes in step S702), the process proceeds to step S703.

In step S703, the reproduction control unit 200 sets the position based on the song information as the reproduction start position. As a result, the reproduction of the recorded data is started from the section in which the song sung by the user of the reproduction device 20 is sung.

The fifth embodiment has been described above. As described above, in the present embodiment, the recorded data is reproduced from the position corresponding to the song sung by the user of the reproduction device 20. Therefore, according to the present embodiment, it is possible to improve the convenience of designating the playback start position of the recorded data.

<Embodiment 6>
The present embodiment provides a technique capable of appropriately setting song information in the recorded data even when a plurality of people sing a song at the same time in the vehicle. Hereinafter, points different from the above-described embodiment will be described.

The recording data generation unit 101 according to the present embodiment uses the camera 11 to obtain song information of a song sung by a person determined according to a predetermined priority rule, the start timing of the song, and the end timing of the song. Generates recorded data added to the video data of the recorded video. According to such a configuration, even when a plurality of people sing a song at the same time in the car, the song information can be appropriately set in the recorded data.

The predetermined priority rule described above determines, for example, a person to acquire song information according to a priority set for a seat in the vehicle. Specifically, for example, in order to prioritize the acquisition of song information of a song sung by a person in a specific seat, a rule is used in which the specific seat is prioritized over other seats. That is, this particular seat has a higher priority than the other seats. The specific seat may be, for example, a rear seat on the right side, a rear seat on the left side, or a driver's seat. It should be noted that these seat types are merely examples, and seats at arbitrary positions provided in the vehicle may be used as specific seats. Further, as the priority, in order to give priority to any one seat over the other seats, a higher priority may be set only for the one seat than the other seats, or a priority of a plurality of seats may be set. In order to determine, a priority corresponding to the priority may be set for each seat.

When a predetermined priority rule that determines a person to be acquired for song information is used according to a priority set for a seat in the vehicle, the recording data generation unit 101 generates the following recording data. To do. That is, in this case, the recording data generation unit 101 captures the song information of the song sung by the person in the seat selected based on the priority, the start timing of the song, and the end timing of the song. The recorded data added to the video data of the moving image taken by 11 is generated.

The song information of the song sung by the person in the seat selected based on the priority can be obtained as follows. For example, by using a directional microphone as the microphone 12, the voice of a person in a specific seat can be extracted. Further, for example, the voice of a person in a specific seat may be extracted by known blind signal separation using a plurality of microphones 12 such as a microphone array. The song information acquisition unit 100 acquires song information using the voice extracted in this way. That is, the song information acquisition unit 100 acquires the song information of the song sung by the person in the seat by using the audio data of the voice from the seat selected based on the priority.

Further, the predetermined priority rule described above may determine the person to be acquired with the song information according to the priority set for the voice of the person. Specifically, for example, in order to prioritize the acquisition of song information of a song sung by a specific person, a rule is used in which the voice of the specific person is prioritized over the voice of another person. That is, the voice of this particular person has a higher priority than the voice of another person. Further, as the priority, in order to give priority to any one person over the other person, a higher priority may be set only for the voice of the one person than the voice of the other person, or the voice of a plurality of persons may be set. In order to determine the priority, the priority corresponding to the priority may be set for the voices of a plurality of persons.

When a predetermined priority rule that determines the person to be acquired for the song information is used according to the priority set for the voice of the person, the recording data generation unit 101 generates the following recording data. To do. That is, in this case, the recording data generation unit 101 captures the song information of the song sung by the person whose voice is selected based on the priority, the start timing of the song, and the end timing of the song with the camera 11. Generates recorded data added to the video data of the video.

The song information of the song sung by the person whose voice is selected based on the priority can be obtained as follows. The song information acquisition unit 100 extracts the voice of a specific person from the audio data acquired by the microphone 12 by using a known sound source separation technique. For example, the song information acquisition unit 100 inputs audio data acquired by the microphone 12 to a model previously learned by machine learning such as deep learning using training data including the voice of a specific person, thereby producing a sound source. Performs separation processing and extracts the voice of a specific person. The song information acquisition unit 100 acquires song information using the voice extracted in this way. That is, the song information acquisition unit 100 acquires the song information of the song sung by the person having the voice by using the audio data of the voice selected based on the priority.

The priority may be used for purposes other than determining the acquisition target of song information. For example, as shown in FIG. 20, the recording device 10 may further include a camera control unit 103. The processing of the camera control unit 103 is realized, for example, by executing a program by the processor 15. The camera control unit 103 controls the camera 11 that shoots the inside of the vehicle so as to zoom in on the person determined according to the priority rule and shoot. According to such a configuration, it is possible to generate recorded data focusing on a person having a high priority. Specifically, when the priority set for the seats in the vehicle is used, the camera control unit 103 uses, for example, information in which the relative positional relationship between each seat in the vehicle and the camera 11 is defined in advance. The shooting range and zoom magnification of the camera 11 are controlled. Further, when the priority set for the voice of the person is used, the camera control unit 103 performs, for example, a face recognition process for collating the face image of the person registered in advance with the image taken by the camera 11. , The position of the person is specified, and the shooting range and zoom magnification of the camera 11 are controlled.

It should be noted that, instead of zooming in on the person determined according to the priority rule at the time of shooting, the video display zoomed in on the person may be performed at the time of reproducing the recorded data. That is, the reproduction control unit 200 of the reproduction device 20 may display the image zoomed in on the person when reproducing the section in which the song is sung. When the priority set for the seat in the vehicle is used, specifically, for example, the image display in which the person determined according to the priority rule is zoomed in is realized as follows. The recording data generation unit 101 adds information indicating which seat person's song information is added to the recorded data for each section in which the song is sung. The reproduction control unit 200 of the reproduction device 20 uses this information and the predefined information indicating in which area in the image captured by the camera 11 each seat is imaged, in the captured image of the camera 11. By enlarging the area, a zoomed-in image is displayed on the person. Further, when the priority set for the voice of the person is used, specifically, for example, a video display in which the person determined according to the priority rule is zoomed in is realized as follows. The recording data generation unit 101 identifies a position in the image of the person by face recognition processing for collating the face image of the person registered in advance with the image taken by the camera 11, and sets information indicating this position. Is added to the recorded data for each section in which is sung. Using this information, the reproduction control unit 200 of the reproduction device 20 enlarges the area in which the person is located in the captured image of the camera 11 to display the image zoomed in on the person.

By the way, the song information acquisition unit 100 acquires song information for one person selected based on priority, for example, but may acquire song information for each of a plurality of persons. That is, the song information acquisition unit 100 may acquire song information of each song sung at the same time during shooting by the camera 11 from the audio data of the sound acquired by the microphone 12. In this case, the song information acquisition unit 100 extracts each voice of each person from the audio data of the sound acquired by the microphone 12, and acquires each song information using the audio data of each voice. As a result, the song information acquisition unit 100 can identify the songs sung by each person, and thus determines whether or not one person selected based on the priority sings the same song as the other person. can do. Therefore, when song information is acquired for each of a plurality of persons, this determination result can be used for controlling other processes. Specifically, for example, the camera control unit 103 may perform the following control. When the song information of the songs sung at the same time is the same, the camera control unit 103 controls the camera 11 to shoot the entire interior of the vehicle without zooming in on the person determined according to the priority rule, and sings at the same time. When the song information of the songs is different, the camera 11 may be controlled so as to zoom in on a person determined according to the priority rule and perform shooting. According to such a configuration, it is possible to generate recorded data of video contents according to the situation. It should be noted that the video display in which such control is performed may be performed when the recorded data is reproduced. In this case, the recording data generation unit 101 adds information indicating whether or not the song information of the songs sung at the same time is the same for each section in which the song is sung, and the playback control unit 200 adds information indicating whether or not the song information is the same. Based on the information, the above-mentioned control is performed.

Next, the tag information setting operation by the recording device 10 according to the present embodiment will be described. FIG. 21 is a flowchart showing an example of the tag information setting operation by the recording device 10. Hereinafter, an example of the tag information setting operation will be described with reference to FIG. 21, but the description of the same processing as the flowchart shown in FIG. 6 will be omitted as appropriate.

In the flowchart shown in FIG. 21, when the user's voice is detected in step S100 (Yes in step S100), the process proceeds to step S800.

In step S800, the song information acquisition unit 100 extracts the audio data of the person whose song information is to be acquired from the audio data from the microphone 12. When acquiring song information for only one person selected based on the priority, the song information acquisition unit 100 extracts the audio data of the voice of that person from the audio data from the microphone 12. Further, when acquiring song information for each of a plurality of persons, the song information acquisition unit 100 extracts audio data of the voices of these persons from the audio data from the microphone 12. After step S800, the process proceeds to step S801.

In step S801, the song information acquisition unit 100 starts accumulating the audio data extracted in step S800 in the buffer.

Next, in step S802, the song information acquisition unit 100 determines whether or not the voice of the person whose song information is to be acquired is no longer detected. When the voice of the person whose song information is to be acquired is no longer detected (Yes in step S802), the process proceeds to S109 described with reference to FIG. That is, in step S109, when only the tag information of the start timing is set, the tag information of the end timing is set. Then, the accumulation of audio data is completed. When the voice of the person whose song information is to be acquired is detected (No in step S802), the process proceeds to step S103.

The processing after step S103 is the same as the processing of each step described with reference to FIG. However, in the present embodiment, as described above, the song information, the start timing, and the tag information of the end timing set in the recording data are the song information, the start timing, and the song information for one person determined according to a predetermined priority rule. And the end timing.

The sixth embodiment has been described above. As described above, in the present embodiment, it is controlled which person the song information is added to the recorded data according to the priority rule. Therefore, even when a plurality of people sing a song at the same time in the car, the song information can be appropriately set in the recorded data.

The present invention is not limited to the above embodiment, and can be appropriately modified without departing from the spirit. As long as there is no technical conflict, some or all of the configurations or functions described in each embodiment may be combined with the configurations or functions described in other embodiments. In addition, some functions or configurations of each embodiment may be excluded as long as there is no technical contradiction.

1 Video system 10 Recording device 11 Camera 12 Microphone 13 Network interface 14 Storage device 15 Processor 16 External storage interface 19 Removable media 20 Playback device 21 Display 22 Speaker 23 Input device 24 Network interface 25 Storage device 26 Processor 27 Microphone 100 Song information acquisition unit 101 Recording data generation unit 102 Storage control unit 103 Camera control unit 200 Playback control unit 201 Instruction reception unit 202 Music data acquisition unit 203 Attention calling unit 204 Music information acquisition unit

Claims (17)

With a camera that shoots videos inside or outside the car,
A microphone that acquires the sound inside the vehicle during shooting by the camera, and
A song information acquisition unit that acquires song information of a song sung during shooting by the camera and start timing and end timing of the song in the moving image from the audio data of the sound acquired by the microphone.
A recording device having a recording data generation unit that generates recording data in which the song information, the start timing, and the end timing are added to the video data of a moving image taken by the camera. It also has a storage control unit that controls the storage of recorded data in the storage device.
The recording device according to claim 1, wherein the storage control unit prohibits overwriting of recorded data from the start timing to the end timing. 2. The storage control unit cancels the overwrite prohibition on the recorded data from the start timing to the end timing when the storable capacity of the new recorded data in the storage device falls below a predetermined reference amount. The recording device described in. The song information acquisition unit further acquires position information for specifying the position of a part of the song acquired by the microphone at the start timing in the entire song.
The recording device according to any one of claims 1 to 3, wherein the recording data generation unit generates the recording data to which the position information is further added. The song information is song identification information, and is
When the song information acquisition unit acquires a plurality of candidates as song information of one song sung during shooting by the camera, the recording data generation unit uses the acquired plurality of song information as the song information of the song. The recording device according to any one of claims 1 to 4, which generates the recorded data added as a candidate list. The song information is audio data of the song, and is
The recording device according to any one of claims 1 to 4, wherein the recording data generation unit generates the recording data in which the video data and the audio data of the song are superimposed. The recording data generation unit is a first recording data which is the recording data in which the video data and the audio data of the song are superimposed, and the recording data of the video data in which the audio data of the song is not superimposed. The recording device according to claim 6, which generates a second recorded data. A recording device that generates recording data and
A playback device that acquires the recorded data from the recording device and reproduces the recorded data is provided.
The recording device is
With a camera that shoots videos inside or outside the car,
A microphone that acquires the sound inside the vehicle during shooting by the camera, and
A song information acquisition unit that acquires song information of a song sung during shooting by the camera and start timing and end timing of the song in the moving image from the audio data of the sound acquired by the microphone.
It has a recording data generation unit that generates recording data in which the song information, the start timing, and the end timing are added to the video data of the moving image taken by the camera.
The playback device is
A moving image system having a playback control unit that controls to output a song based on the song information during playback of a section from the start timing to the end timing when the recorded data is reproduced. When the recorded data includes a plurality of sections from the start timing to the end timing and the interval between adjacent sections is shorter than a predetermined time, the playback control unit may output a song in one of the sections and the other. The moving image system according to claim 8, wherein the song to be output in the section is controlled to be cross-faded and output. When the length of the section from the start timing to the end timing is longer than the length of the song associated with the section, the reproduction control unit controls to output the song in a loop. Or the moving image system according to 9. The song information acquisition unit further acquires position information for specifying the position of a part of the song acquired by the microphone at the start timing in the entire song.
The recorded data generation unit generates the recorded data to which the position information is further added.
The moving image system according to any one of claims 8 to 10, wherein the reproduction control unit controls to reproduce and output the song from a position specified based on the position information when outputting the song. .. The playback device further includes an instruction receiving unit that receives an instruction regarding output of a song in a section from the start timing to the end timing.
The moving image system according to any one of claims 8 to 11, wherein the reproduction control unit controls to output a song in a section from the start timing to the end timing in accordance with the instruction. When the song information acquisition unit acquires a plurality of candidates as song information of one song sung during shooting by the camera, the recording data generation unit uses the acquired plurality of song information as the song information of the song. Generate the recorded data added as a candidate list,
The instruction receiving unit receives an instruction to select one of the song information listed in the candidate list, and receives an instruction to select one of them.
The moving image system according to claim 12, wherein the playback control unit controls to output a song based on the selected song information. The instruction receiving unit receives an instruction to specify a playback start position in the song, and receives the instruction.
The moving image system according to claim 12 or 13, wherein when the song is output, the playback control unit controls to play and output the song from the playback start position in the song. The moving image system according to any one of claims 8 to 14, wherein the reproduction control unit extracts a section from the start timing to the end timing of the recorded data and reproduces the digest. Acquires the sound inside the vehicle while the camera is shooting the video inside or outside the vehicle. From the audio data of the sound acquired by the microphone, the song information of the song sung during the shooting by the camera and the video of the song. Get the start timing and end timing in
A recording method for generating recording data in which the song information, the start timing, and the end timing are added to the video data of a moving image taken by the camera. Acquires the sound inside the vehicle while the camera is shooting the video inside or outside the vehicle. From the audio data of the sound acquired by the microphone, the song information of the song sung during the shooting by the camera and the video of the song. The song information acquisition step to acquire the start timing and end timing in
A program that causes a computer to execute a recording data generation step of generating recording data in which the song information, the start timing, and the end timing are added to the video data of a moving image taken by the camera.

Images