JP7451999B2 - Recording device, video system, recording method, and program - Google Patents

Description

The present invention relates to a recording device, a moving image system, a recording method, and a program.

Various research and developments are being conducted on functions related to drive recorders. For example, Patent Document 1 discloses a technique for adding BGM (background music) with a tempo that corresponds to the traveling speed of a vehicle to a digest video summarizing a drive video.

Furthermore, in recent years, there has been a need to use drive recorders not only for recording accidents but also for enjoying driving images. Note that, as a technique for enjoying images taken by a camera, there is a technique described in Patent Document 2, for example. In this technology, an image recording device uses a microphone to detect music playing from a car stereo, acquires music data of the music, and associates this music data with image data. By doing this, the music that was being played at the time of photography can be associated with the photographed image. Therefore, according to this technique, it is possible to provide an electronic album that allows the user to play back photographed images while remembering the atmosphere at the time of photography.

Japanese Patent Application Publication No. 2016-167771 Japanese Patent Application Publication No. 2006-109151

As mentioned above, in recent years, there has been a need to enjoy recorded drive videos. Therefore, there is a need to provide various technologies that meet such needs. In particular, there is a need to provide technology that improves the convenience of viewing and enjoying images from a vehicle's camera.

An object of the present disclosure is to provide a recording device, a video system, a recording method, and a program that can improve the convenience of viewing and enjoying videos from a vehicle camera.

The recording device according to the first aspect includes:
A camera that takes videos inside or outside the vehicle,
a microphone that captures sounds inside the car while the camera is shooting;
a song information acquisition unit that acquires song information of a song sung during shooting with the camera, and a start timing and an end timing of the song in a video from audio data of a sound acquired by the microphone;
and a recorded data generation unit that generates recorded data in which the song information, the start timing, and the end timing are added to video data of a moving image shot by the camera.

The video system according to the second aspect includes:
a recording device that generates recorded data;
a playback device that acquires the recorded data from the recording device and plays the recorded data;
The recording device is
A camera that takes videos inside or outside the vehicle,
a microphone that captures sounds inside the car while the camera is shooting;
a song information acquisition unit that acquires song information of a song sung during shooting with the camera, and a start timing and an end timing of the song in a video from audio data of a sound acquired by the microphone;
a recorded data generation unit that generates recorded data in which the song information, the start timing, and the end timing are added to video data of a moving image shot by the camera;
The playback device includes:
When reproducing the recorded data, the apparatus includes a reproduction control section that controls to output a song based on the song information during reproduction of the section from the start timing to the end timing.

In the recording method according to the third aspect,
From the audio data of the sound acquired by the microphone that acquires the sound inside the car while the camera that takes video inside or outside the car is recording, the song information of the song sung while the camera is recording, and the video of the song. Obtain the start timing and end timing within
Recording data is generated by adding the song information, the start timing, and the end timing to video data of a moving image shot by the camera.

The program according to the fourth aspect is
From the audio data of the sound acquired by the microphone that acquires the sound inside the car while the camera that takes video inside or outside the car is recording, the song information of the song sung while the camera is recording, and the video of the song. a song information acquisition step of acquiring a start timing and an end timing within the song;
A computer is caused to execute a recorded data generation step of generating recorded data in which the song information, the start timing, and the end timing are added to video data of a moving image shot by the camera.

According to the present invention, it is possible to provide a recording device, a video system, a recording method, and a program that can improve the convenience of viewing and enjoying videos from a vehicle camera.

1 is a schematic diagram showing an example of the configuration of a video system according to an embodiment. 1 is a block diagram showing an example of a hardware configuration of a recording device according to an embodiment. FIG. 1 is a block diagram showing an example of a hardware configuration of a playback device according to an embodiment. FIG. 1 is a block diagram showing an example of a functional configuration of a recording device according to an embodiment. FIG. 1 is a block diagram showing an example of a functional configuration of a playback device according to an embodiment. FIG. 7 is a flowchart illustrating an example of a tag information setting operation performed by the recording device according to the embodiment. FIG. 3 is a schematic diagram showing tag information set in recorded data. 3 is a flowchart illustrating an example of a playback operation by the playback device according to the embodiment. 1 is a block diagram showing an example of a functional configuration of a playback device according to an embodiment. FIG. 7 is a flowchart illustrating an example of an operation of setting tag information regarding song information by the recording device according to the embodiment. 3 is a flowchart illustrating an example of a process for setting music information by a user. 12 is a flowchart illustrating an example of a process flow for determining a song to output. 1 is a block diagram showing an example of a hardware configuration of a recording device according to an embodiment. FIG. It is a flowchart which shows an example of the flow of the process of moving the recorded data of the section in which a song was sung to another storage device. 1 is a block diagram showing an example of a hardware configuration of a playback device according to an embodiment. FIG. 1 is a block diagram showing an example of a functional configuration of a playback device according to an embodiment. FIG. FIG. 3 is a schematic diagram showing a playback start position of recorded data. FIG. 3 is a schematic diagram showing a playback start position of recorded data. 12 is a flowchart illustrating an example of the flow of an operation for determining a playback start position of recorded data. 1 is a block diagram showing an example of a functional configuration of a recording device according to an embodiment. FIG. 7 is a flowchart illustrating an example of a tag information setting operation performed by the recording device according to the embodiment.

Embodiments will be described below with reference to the drawings. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate.

<Embodiment 1>
FIG. 1 is a schematic diagram showing an example of the configuration of a video system 1 according to the first embodiment. The video system 1 includes a recording device 10 that generates recorded data, and a playback device 20 that acquires recorded data from the recording device 10 and plays back the recorded data. Note that the playback device 20 may acquire recorded data directly from the recording device 10 through wired or wireless communication, or may acquire recorded data from the recording device 10 via another device or the like. .

The recording device 10 and the playback device 20 will be specifically explained below. FIG. 2 is a block diagram showing an example of the hardware configuration of the recording device 10. As shown in FIG. Further, FIG. 3 is a block diagram showing an example of the hardware configuration of the playback device 20. As shown in FIG. Further, FIG. 4 is a block diagram showing an example of the functional configuration of the recording device 10. As shown in FIG. FIG. 5 is a block diagram showing an example of the functional configuration of the playback device 20.

The recording device 10 is a device installed in a vehicle such as a four-wheeled vehicle, and is a so-called drive recorder. As shown in FIG. 2, the recording device 10 includes a camera 11, a microphone 12, a network interface 13, a storage device 14, and a processor 15.

The camera 11 is a digital camera, and takes a video of the inside or outside of the vehicle in which the recording device 10 is mounted. The microphone 12 is a device that collects sounds inside the car, and acquires the sounds inside the car while the camera 11 is photographing.

Network interface 13 is used to communicate with other devices. Note that the other device may be, for example, a server that performs part of the processing of the recording device 10, or may be the playback device 20 that receives recorded data. Network interface 13 may include, for example, a network interface card (NIC).

The storage device 14 includes a memory group including one or both of volatile memory and nonvolatile memory, and internal storage such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Note that in this embodiment, the memory and internal storage are collectively referred to as the storage device 14, but the memory and internal storage do not need to be built into the recording device 10 as a single device, but may be installed as separate devices. It may be built into the recording device 10.

The internal storage is used to store, for example, recorded data and various data necessary for processing of the recording device 10. The memory is used to store, for example, software (computer program) that is executed by the processor 15 and includes one or more instructions.

The program can be stored and provided to a computer using various types of non-transitory computer readable media. Non-transitory computer-readable media includes various types of tangible storage media. Examples of non-transitory computer-readable media are magnetic recording media (e.g., flexible disks, magnetic tape, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), Compact Disc Read Only Memory (CD-ROM), CD- R, CD-R/W, semiconductor memory (e.g., mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, random access memory (RAM)). The program may also be supplied to the computer on various types of transitory computer readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can provide the program to the computer via wired communication channels, such as electrical wires and optical fibers, or wireless communication channels.

The processor 15 performs processing of the components shown in FIG. 4, for example, by reading software (computer program) from the memory and executing it. That is, each process of the music information acquisition section 100, the recorded data generation section 101, and the storage control section 102 shown in FIG. 4 may be realized by executing a program. In this way, the recording device 10 has the function of a computer. The processor 15 may be, for example, a microprocessor, an MPU (Micro Processor Unit), or a CPU (Central Processing Unit). Processor 15 may include multiple processors.

The playback device 20 is a device that plays back recorded data recorded by the recording device 10, and specifically, is, for example, a smartphone, a tablet terminal, a PC (Personal Computer), or the like. As shown in FIG. 3, the playback device 20 includes a display 21 that displays video of recorded data, and a speaker 22 that outputs audio of recorded data. The recording device 10 further includes an input device 23, a network interface 24, a storage device 25, and a processor 26.

The input device 23 is a device that receives operation input from a user who uses the playback device 20, and is, for example, a pointing device.
Network interface 24 is used to communicate with other devices such as recording device 10 that transmits recorded data. Network interface 24 may include, for example, a network interface card (NIC).

The storage device 25, like the storage device 14 in the recording device 10, includes a memory group and internal storage. The internal storage is used, for example, to store recorded data acquired from the recording device 10 and various data necessary for processing by the playback device 20. The memory is used, for example, to store software (computer programs) including one or more instructions that are executed by the processor 26. The program may also be provided to the computer using various types of non-transitory computer readable media, and may be provided to the computer by various types of temporary computer readable media.

The processor 26 reads software (computer program) from the memory and executes it, thereby processing the components shown in FIG. 5, for example. That is, each process of the reproduction control section 200, the instruction reception section 201, and the music data acquisition section 202 shown in FIG. 5 may be realized by executing a program. In this way, the playback device 20 also has the function of a computer. Processor 26 may be, for example, a microprocessor, MPU, or CPU. Processor 26 may include multiple processors.

Next, the functional configuration of the recording device 10 will be explained. As shown in FIG. 4, the recording device 10 includes a song information acquisition section 100, a recorded data generation section 101, and a storage control section 102. When the user sings a song while driving or the like, the recording device 10 acquires the song information of the song and associates this song information with the video data of the camera 11. At this time, it is specified from which point in the video data to which point in time the user was actually singing, and the video data is associated with the song information. Note that song information may be obtained when the melody of a song is sung without lyrics, such as humming, or song information may be obtained when the lyrics are sung.

The song information acquisition unit 100 acquires, from the audio data of the sound acquired by the microphone 12, the song information of the song sung during the shooting with the camera 11, and the start timing and end timing of the song in the video. Here, the song information is song identification information. In other words, it is information for uniquely identifying a song. Further, the song information acquisition unit 100 may further acquire position information for specifying the position in the entire song for a portion of the song acquired by the microphone 12 at the start timing. That is, when a certain song is sung, the song information acquisition unit 100 determines where in the song the song begins (for example, from the beginning of the song, from the chorus, from the second verse, etc.) ) to obtain location information that identifies the location. For example, this position information is time information indicating the elapsed time from the start of the song.

Specifically, the song information acquisition unit 100 acquires these in the following manner, for example. The song information acquisition unit 100 continuously monitors the audio data of the sound acquired by the microphone 12, and determines whether the user's voice is detected. When a voice is detected, the song information acquisition unit 100 starts accumulating audio data of the detected voice in a buffer. When audio data for a predetermined unit time is accumulated, song information is analyzed for the audio data for a predetermined unit time. In this embodiment, this analysis process is performed by, for example, a server communicably connected to the recording device 10. Note that this server performs analysis processing by, for example, having a processor of the server execute a program stored in a memory of the server.

The song information acquisition unit 100 transmits audio data for a unit time to the server. This server uses known song search technology such as humming search, which searches for songs from audio, to search for songs that have similar music features to the music features of the received audio data, and stores the music features for each song. Search by referring to the song database. The song information acquisition unit 100 acquires, from the server, song information of songs whose similarity to the audio data to be analyzed exceeds a predetermined threshold (first threshold). The recorded data generation unit 101, which will be described later, adds song information of songs that exceed this first threshold to the video data. Note that if the song information of multiple songs is acquired as the song information of the song exceeding the first threshold, that is, if multiple candidates are acquired as the song information of one song sung while the camera 11 is shooting. In this case, the recorded data generation unit 101 adds the song information of the song with the highest degree of similarity to the video data. However, the recorded data generation unit 101 may add song information of these plural songs as a candidate list to the video data, as described later, without selecting one of them.

As described above, the song information acquisition unit 100 may acquire position information that specifies where in the song the song that is being recorded begins. This can be done, for example, by specifying on the server which part of the music feature of a song that is determined to be similar to the audio data for a unit time that is the subject of analysis corresponds to this audio data. can.

The song information acquisition unit 100 also acquires the start timing and end timing of a song based on the analysis results of audio data for each continuous unit time. That is, the song information acquisition unit 100 sets the timing at which audio data determined to be similar to a certain song is obtained through analysis as the start timing of this song, and the timing when audio data determined to be similar to this song is no longer obtained. Set the timing as the end timing of this song.

When a song is sung in multiple periods during one recording, the song information acquisition unit 100 acquires the song information, start timing, and end timing for each period. Furthermore, when different songs are sung one after another, the song information acquisition unit 100 may acquire the song information, start timing, and end timing for each song being sung.

Note that, as described above, in this embodiment, the analysis process is performed by the server, but it may also be performed as a process by the music information acquisition unit 100. That is, all or part of the processing described as server processing may be performed as processing in recording device 10.

The recorded data generation unit 101 generates recorded data by adding the song information, start timing, and end timing acquired by the song information acquisition unit 100 to the video data of the moving image captured by the camera 11. Specifically, the recorded data generation unit 101 generates, for each period in which the song was sung, time information indicating the start timing and end timing of the song in the video, and the songs sung during the period. information is added to the video data as tag information. Note that if the song information acquisition unit 100 has acquired the above-mentioned position information that specifies the position at which the song begins, the recorded data generation unit 101 further adds this position information to each song as tag information. You may also generate recorded data. Thereby, when the music is played back by the playback device 20, which will be described later, it is possible to output the music from the position indicated by this position information.

In addition, when the song information acquisition section 100 acquires a plurality of candidates as song information for one song sung while the camera 11 is shooting, the recording data generation section 101 uses the acquired plurality of song information as the song information for the song. Recorded data added as an information candidate list may be generated. That is, a candidate list may be set in the recorded data as tag information. Thereby, as will be described later, the user can select appropriate song information from among the song information listed as candidates.

Note that the recorded data generation unit 101 may generate recorded data that includes video data of a moving image captured by the camera 11; May be generated.

The storage control unit 102 controls storage of the recorded data generated by the recorded data generation unit 101 in the storage device 14 (internal storage). For example, when the storage device 14 runs out of free space, the storage control unit 102 controls to store new recorded data by sequentially overwriting the oldest recorded data. The storage control unit 102 may prohibit overwriting of recorded data from the start timing to the end timing. By doing this, it is possible to prevent the recorded data of the section in which the song was sung from being deleted. In other words, it is possible to prevent images of impressive scenes from being deleted. Note that the storage control unit 102 may cancel the overwriting prohibition on the recorded data from the start timing to the end timing when the storage capacity of new recorded data in the storage device 14 is less than a predetermined reference amount. . Here, the storable capacity of new recorded data is the total amount of capacity for which overwriting processing is permitted and free capacity. The recording device 10 may need to store more important images, such as images taken when a vehicle accident occurs. For this reason, it is undesirable that such important video cannot be stored when the storage capacity of new recorded data is reduced. By canceling the overwrite prohibition, it is possible to prevent such a situation from occurring.

Next, the functional configuration of the playback device 20 will be explained. As shown in FIG. 5, the playback device 20 includes a playback control section 200, an instruction reception section 201, and a music data acquisition section 202.

The playback control unit 200 plays back the recorded data acquired from the recording device 10 and displays an image of the recorded data on the display 21. Further, when the recorded data includes audio data of the sound acquired by the microphone 12, the sound of this audio data may be output from the speaker 22.

Furthermore, when playing back recorded data, the playback control unit 200 controls the speaker 22 to output a song based on song information associated with the section during playback of the section from the start timing to the end timing. Specifically, the playback control unit 200 controls the speaker 22 to output this song using the audio data of the song acquired based on this song information. The playback control unit 200 outputs the song from the speaker 22 using the audio data acquired by the song data acquisition unit 202 during playback of the section. Note that this output may be superimposed on the output of the sound of the audio data of the sound acquired by the microphone 12. The ratio of the two volumes to be superimposed can be set arbitrarily. Further, the sound acquired by the microphone 12 may be set to be muted.

Furthermore, when the above-mentioned location information is added to the recorded data, when outputting the song, the playback control unit 200 controls the playback and output of the song from the position specified based on this location information. You may. That is, the playback control unit 200 may output the song so that the beginning of the song when it is actually sung and the beginning of the song during playback are the same. By doing so, the situation at the time of recording can be more faithfully reproduced.

In addition, if the recorded data has multiple sections from the start timing to the end timing, and the interval between adjacent sections is shorter than a predetermined time, the playback control unit 200 outputs the song in one section and the song output in the other section. It may also be possible to control the output so as to cross-fade the music. That is, the playback control unit 200 may output the music so that the preceding music is faded out and the following music is faded in. By doing this, you can enjoy the music more naturally.

Furthermore, if the length of the section from the start timing to the end timing is longer than the length of the song associated with the section, the playback control unit 200 may control the song to be looped and output. . That is, if the length of this section is longer than the length of the song acquired by the song data acquisition section 202, which will be described later, the playback control section 200 controls the length of the song acquired by the song data acquisition section 202 while playing the section. may be controlled to be output repeatedly. By doing this, the song can be output for the same length of time as the actual period in which it was sung, so the situation at the time of recording can be more faithfully reproduced.

Note that the playback control unit 200 may play back the entire recorded data, or may perform digest playback of the recorded data. When performing digest playback, the playback control unit 200 may extract a section from the start timing to the end timing from the recorded data and perform digest playback. By doing so, it is possible to provide a digest playback in which impressive scenes are extracted.

The instruction receiving unit 201 receives instructions regarding the output of the music in the section from the start timing to the end timing. The instruction receiving unit 201 receives instructions input by the user via the input device 23. The playback control unit 200 may control the output of the song in the section from the start timing to the end timing according to the received instruction. For example, in the recorded data, if a candidate list including multiple pieces of song information is added to one song in a certain section, the instruction receiving unit 201 selects one of the pieces of song information listed in the candidate list. Accept instructions to select. In this case, the playback control unit 200 controls to output the song based on the selected song information. That is, the playback control unit 200 controls the speaker 22 to output this song using the audio data of the song acquired based on the selected song information. According to such a configuration, the user can select appropriate song information from among the song information listed as candidates, and therefore can listen to an appropriate song.

Further, for example, the instruction receiving unit 201 may receive an instruction specifying a playback start position in a song. In this case, when outputting a song in the section from the start timing to the end timing, the playback control unit 200 controls the song to be played and output from a specified playback start position in the song. For example, if an instruction is given to play the song starting from the second song, the playback control unit 200 outputs the song starting from the second song. By doing so, the user can listen to the song from the desired playback start position.

The song data acquisition unit 202 acquires audio data of a song that is output during playback of the section from the start timing to the end timing in the recorded data. Specifically, the song data acquisition unit 202 acquires the audio data of the original song of the song sung during recording by the recording device 10 . That is, the song data acquisition unit 202 acquires audio data of a song sung or performed by an actual artist. In this embodiment, the song data acquisition unit 202 acquires audio data having song information that matches the song information added to the recorded data. However, if the instruction receiving unit 201 selects song information from the candidate list, the song data acquisition unit 202 acquires audio data having song information that matches the selected song information.

The song data acquisition unit 202 may acquire audio data to be output during playback of the section from the audio data group stored in the storage device 25 of the playback device 20, or may acquire the audio data to be output during playback of the section, via the network interface 24, Audio data may be acquired from another device communicatively connected by wire or wirelessly. For example, audio data distributed by network streaming may be acquired. As described above, the playback control unit 200 uses the audio data acquired by the song data acquisition unit 202 to output a song while playing the recorded data from the start timing to the end timing.

Next, an example of the operation of the video system 1 will be described using a flowchart. FIG. 6 is a flowchart showing an example of the tag information setting operation by the recording device 10. An example of the tag information setting operation will be described below with reference to FIG.

In step S100, the song information acquisition unit 100 continuously monitors the audio data of the sound acquired by the microphone 12, and determines whether the user's voice is detected. If no audio is detected (No in step S100), this step continues. If audio is detected (Yes in step S100), the process moves to step S101.

In step S101, the song information acquisition unit 100 starts accumulating audio data of the detected voice in the buffer.

Next, in step S102, the song information acquisition unit 100 determines whether or not audio is no longer detected. If audio is detected (No in step S102), the process moves to step S103, and if audio is no longer detected (Yes in step S102), the process moves to step S109.

In step S103, the song information acquisition unit 100 determines whether audio data for a predetermined unit time has been accumulated. If audio data for a unit time has been accumulated (Yes in step S103), the process moves to step S104; otherwise (No in step S103), the process returns to step S102.

In step S104, the song information acquisition unit 100 uses the accumulated audio data to acquire song information. Specifically, as described above, the song information acquisition unit 100 acquires song information obtained through a song search using this audio data. Note that the song information acquisition unit 100 may acquire the above-mentioned position information, which is information about which position in the song this audio data corresponds to.

Next, in step S105, the song information acquisition unit 100 determines whether the tag information of the start timing has been set. In addition, considering that multiple sections specified by the start timing and end timing can be set for one recorded data, more specifically, the determination in this step is not yet paired with the tag information of the end timing. This is a determination as to whether tag information with a start timing that is not specified is set. Therefore, even if one or more sets of tag information for a pair of start timing and end timing have already been set, such tag information for the start timing is ignored and the tag information for the start timing has already been set. A determination is made as to whether or not. If the tag information of the start timing is not set (No in step S105), the process moves to step S106. On the other hand, if the tag information of the start timing is set (Yes in step S105), the process moves to step S107.

In step S106, the song information acquisition unit 100 sets the acquisition timing of the audio data used in the process of step S104 as the start timing. In this way, the song information acquisition unit 100 acquires the start timing. Then, the recorded data generation unit 101 sets the tag information of this start timing in the recorded data as the tag information of the recording time corresponding to the acquisition timing of this audio data. The recorded data generation unit 101 also sets the tag information of the song information obtained in step S104 in the recorded data as tag information at the recording time corresponding to the acquisition timing of this audio data. Note that in step S104, if the location information is also acquired, the recorded data generation unit 101 also adds the tag information of this location information to the recorded data as tag information at the recording time corresponding to the acquisition timing of this audio data. Set.

After step S106, the process returns to step S102, and the above-described process is repeated. That is, if the audio continues, the next unit time worth of audio data will be searched for music. As described above, if audio is no longer detected (Yes in step S102), the process moves to step S109.

In step S109, the song information acquisition unit 100 sets the timing at which no audio is detected as the end timing. In this way, the song information acquisition unit 100 acquires the end timing. That is, the recorded data generation unit 101 sets the tag information of this end timing in the recorded data as the tag information of the recording time corresponding to the timing when audio is no longer detected. Then, the accumulation of audio data is completed. Note that the end timing tag information is set when only the start timing tag information is set.

On the other hand, in step S107, it is determined whether the song information searched for the previous audio data and the song information searched for the current audio data are different. That is, it is determined whether the searched song has changed compared to the previous time. Thereby, even if the song being sung changes, tag information can be appropriately added. If the searched song has changed compared to the previous time (Yes in step S107), the process moves to step S108. If the searched song has not changed compared to the previous time (No in step S107), the process returns to step S102, and the above-described process is repeated. That is, if the audio continues, the next unit time worth of audio data will be searched for music.

In step S108, the song information acquisition unit 100 sets the acquisition timing of the audio data used in the process of step S104 as the end timing. In this way, the song information acquisition unit 100 acquires the end timing. That is, the recorded data generation unit 101 sets the tag information of this end timing in the recorded data as the tag information of the recording time corresponding to the acquisition timing of this audio data. After step S108, the process moves to step S106.

Through the processing described above, tag information as shown in FIG. 7, for example, is set for the recorded data.

Next, an example of the operation of the playback device 20 will be described. FIG. 8 is a flowchart showing an example of a playback operation by the playback device 20. An example of the reproduction operation will be described below with reference to FIG.

In step S200, the playback control unit 200 determines whether the playback of the video has ended. If the playback is finished (Yes in step S200), the process ends. If playback continues (No in step S200), the process moves to step S201.

In step S201, the playback control unit 200 determines whether tag information of the start timing is detected. That is, the playback control unit 200 determines whether the start timing has arrived for the recorded data currently being played back. If the start timing has not arrived (No in step S201), the process returns to step S200. When the start timing has arrived (Yes in step S201), the process moves to step S202.

In step S202, the song data acquisition unit 202 acquires the audio data of the song based on the song information added as a tag to the recorded data.

Next, in step S203, the playback control unit 200 outputs a song using the audio data acquired in step S202. That is, the playback control unit 200 plays back the recorded data by superimposing the song on the video.

Next, in step S204, the reproduction control unit 200 determines whether tag information of the end timing has been detected. That is, the playback control unit 200 determines whether the end timing has arrived for the recorded data currently being played back. If the end timing has not arrived (No in step S204), reproduction of the song continues.

On the other hand, when the end timing has arrived (Yes in step S204), the process moves to step S205, and in step S205, the playback control unit 200 ends the playback of the song. After step S205, the process returns to step S200.

The first embodiment has been described above. According to the recording device 10 described above, when a song is sung during recording, information specifying in which section of the recorded data the song was sung (i.e., start timing and end timing) and which song is sung are provided. Information about music (song information) can be recorded. Therefore, the user can listen to the song at the timing when the song is actually sung when playing back the video from the vehicle's camera. Therefore, the user can enjoy the video captured by the vehicle's camera. Therefore, the convenience of viewing videos can be improved.

Note that in the embodiment described above, the song information is the identification information of the song, but the song information can also be the audio data of the song. In this case, the recorded data generation unit 101 may generate recorded data in which the video data and the audio data of the song are superimposed for the section from the start timing to the end timing. By doing so, even in a general playback device, when playing back recorded data, a song can be output in the section where the song is sung.

Note that when the recording device 10 generates recorded data in which video data and audio data of a song are superimposed, the recorded data generation unit 101 may generate two types of recorded data. That is, the recorded data generation unit 101 generates first recorded data, which is recorded data in which video data and audio data of a song are superimposed, and second recorded data, which is recorded data of video data on which audio data of a song is not superimposed. Recorded data may also be generated. Superimposing the audio data of a song means that the driving video is processed, which may cause it to lose its value as a driving video. Therefore, such a situation can be avoided by creating two types of recorded data.

Further, in the embodiment described above, song analysis processing is performed at the same time as recording, but it does not necessarily have to be performed during recording.

<Embodiment 2>
In the embodiment described above, if no song is found whose similarity to the audio data to be analyzed exceeds a predetermined threshold (first threshold), it is possible to determine a song to be associated with the section from the start timing to the end timing. Can not. Therefore, in such a case, the song cannot be output when playing that section. Therefore, in this embodiment, a technique is provided that allows music to be output even in such a case. Hereinafter, points different from the embodiments described above will be explained.

In the present embodiment, if no song is found whose similarity to the audio data to be analyzed exceeds the first threshold, the song information acquisition unit 100 sets a predetermined second threshold that is lower than the first threshold. Get song information for songs that exceed. In other words, if there is a song whose similarity to the song sung while the camera 11 was shooting exceeds a predetermined first threshold, the song information acquisition unit 100 acquires the song information of the song from the camera 11. Obtained as song information for songs sung during shooting. Then, if there is no song whose similarity with the song sung while the camera 11 was shooting exceeds a predetermined first threshold, the song information acquisition unit 100 selects a song whose similarity exceeds a second threshold. The song information of the song sung while the camera 11 was shooting is acquired as a song information candidate.

Further, in the present embodiment, when the song information of the song exceeding the second threshold is acquired, the recording data generation unit 101 generates the song information of the song exceeding the second threshold that is sung while the camera 11 is shooting. The recorded data added as a candidate list of song information of the selected songs is generated. That is, the recorded data generation unit 101 sets, as tag information, a candidate list including song information of songs exceeding the second threshold value in the recorded data.

In the playback device 20 of the present embodiment, if there is no song exceeding the first threshold, that is, if song information of a song exceeding the first threshold is not obtained, the instruction receiving unit 201 An instruction is received from the user to select one of the song information listed in the candidate list that includes the song information of the song that exceeds the threshold value of 2. Then, if there is no song that exceeds the first threshold, that is, if it is desired to obtain song information of a song that exceeds the first threshold, the playback control unit 200 starts playback based on the song information selected by the instruction. Control is performed to output the music in the section from the timing to the end timing. Therefore, the song data acquisition unit 202 acquires the audio data of the song having the song information selected by the instruction. Note that if song information of a song that exceeds the first threshold is obtained, the playback control unit 200 performs playback from the start timing to the end timing based on this song information, as described in the first embodiment. Controls the output of the section of music.

Note that the instruction receiving unit 201 receives an instruction in advance, for example, before playing the recorded data. In this embodiment, the instruction receiving unit 201 sets the music information specified by the received instruction in the recording data as tag information.

In order to have the user specify the song information, a configuration may be provided to alert the user. That is, as shown in FIG. 9, the playback device 20 may further include an attention calling section 203. The processing of the attention calling unit 203 is realized by, for example, executing a program by the processor 26. The attention calling unit 203 calls attention when there is recorded data for which a song to be output is not specified during playback of the section from the start timing to the end timing. For example, when the user attempts to play recorded data for which no song has been specified, the attention calling unit 203 may display a message prompting the user to specify song information. Further, the attention calling unit 203 may display a mark indicating that the recorded data for which no song is specified is the recorded data for which no song is specified on the list screen of playable recorded data. By doing so, it is possible to prompt the user to specify a song.

Note that, if playback of recorded data for which a song to be output is not specified is started during playback of the section from the start timing to the end timing, the playback control unit 200 outputs a predetermined song during playback of the section. It may be controlled as follows. That is, when recorded data is played back without the user specifying a song, a predetermined song such as a recommended song may be played back. In particular, a song specified by any of the song information listed in the candidate list may be used as the predetermined song. By doing this, even if a song is not specified, it is possible to output the song in the section.

Further, the instruction receiving unit 201 may receive an instruction to specify a song when there is no song that exceeds the second threshold value. If there is no song that exceeds the second threshold, the above-mentioned candidate list is not set in the recorded data, so it is not possible to specify the song to be output during playback of the section from the start timing to the end timing. Therefore, in such a case, by receiving an instruction from the user to specify a song, it becomes possible to specify the song to be output during playback of the section. In this case, the reproduction control unit 200 controls to output the music specified by the instruction received by the instruction receiving unit 201 while the section is being reproduced.

Next, an example of the operation in this embodiment will be described using a flowchart. FIG. 10 is a flowchart illustrating an example of the operation of setting tag information regarding song information by the recording device 10. An example of the operation of setting tag information regarding song information in this embodiment will be described below with reference to FIG.

In step S300, the recorded data generation unit 101 determines whether a song with a degree of similarity exceeding a first threshold has been searched. That is, the recorded data generation unit 101 determines whether the song information acquisition unit 100 has acquired the song information of a song whose similarity exceeds the first threshold. If a song whose similarity exceeds the first threshold is retrieved (Yes in step S300), the process moves to step S301; otherwise (No in step S300), the process moves to step S303.

In step S301, the recorded data generation unit 101 sets the song information of the song with the highest degree of similarity as tag information in the recorded data. That is, when a plurality of songs whose similarity exceeds the first threshold is found, the recorded data generation unit 101 sets the song information of the song with the highest similarity among them as tag information in the recorded data. After step S301, the process moves to step S302.

In step S302, if there are a plurality of songs that exceed the first threshold, the recorded data generation unit 101 sets information about those songs in the recorded data as tag information of the candidate list. Thereby, as described in the first embodiment, the user can select appropriate song information from among the song information listed as candidates. When step S302 is completed, the setting of tag information regarding the song information is completed.

On the other hand, in step S303, the recorded data generation unit 101 sets tag information indicating that there is no song matching the search condition, that is, tag information indicating that there is no song exceeding the first threshold, in the recorded data. . After step S303, the process moves to step S304.

In step S304, if there are songs that exceed the second threshold, the recorded data generation unit 101 sets information about those songs in the recorded data as tag information of the candidate list. That is, if the song information acquisition unit 100 has acquired song information of a song whose similarity exceeds the second threshold, the recording data generation unit 101 records the song information of such a song as tag information of the candidate list. Set to data. Thereby, even if a song matching the search conditions is not found, the user can select desired song information.

FIG. 11 is a flowchart illustrating an example of a process for setting music information by a user. An example of a process for setting music information by a user in this embodiment will be described below with reference to FIG. 11. For example, the user performs an operation to specify a song for recorded data to which tag information indicating that there is no song exceeding a first threshold value is added.

In step S400, the instruction receiving unit 201 determines whether a candidate list is set in the recorded data. If a candidate list has been set (Yes in step S400), the process moves to step S401. On the other hand, if the candidate list has not been set (No in step S400), the process moves to step S403.

In step S401, the instruction receiving unit 201 presents the user with a list of song information listed in the candidate list. After step S401, the process moves to step S402.

In step S402, the instruction receiving unit 201 receives an instruction from the user to select one of the pieces of music information listed in the candidate list. Then, the instruction receiving unit 201 newly sets the song information specified by this instruction to the recorded data as tag information of the song information. With the above steps, the song information setting process is completed.

On the other hand, in step S403, the instruction receiving unit 201 receives input of song information from the user. Then, the instruction receiving unit 201 newly sets the input song information to the recorded data as tag information of the song information. With the above steps, the song information setting process is completed.

The second embodiment has been described above. According to this embodiment, if a song matching the search conditions is not found, the user can select desired song information. Therefore, even if a song matching the search conditions is not found, the song can be output when playing recorded data.

<Embodiment 3>
In the embodiment described above, when the recording device 10 cannot acquire the audio data of the song specified by the song information set in the recorded data, the playback device 20 reproduces the section from the start timing to the end timing of the song in the recorded data. In this case, the song cannot be output. Therefore, in the present embodiment, a technique is provided that allows the recording device 10 to reproduce a song even when the audio data of the song specified by the song information set in the recording data cannot be acquired. Hereinafter, points different from the embodiments described above will be explained.

When the recording device 10 cannot acquire the audio data of the song specified by the song information set in the recording data, the song data acquisition unit 202 according to the present embodiment is configured to perform an audio recording system that performs audio data related to the song specified by this song information. Songs for which data can be obtained are selected as song candidates to be output during playback in the section from the start timing to the end timing. That is, the song data acquisition unit 202 creates a related song list that is a list of songs that are related to the song specified by the song information and for which audio data can be obtained. The songs listed in this related song list become song candidates to be output when playing the recorded data. Note that the related song list may also be referred to as a candidate list. If the audio data of the song specified by the song information set in the recording data by the recording device 10 cannot be acquired, the playback control unit 200 selects one of the songs included in the above-mentioned candidates during playback of the section. Control output. That is, the playback control unit 200 outputs any of the songs listed in the related song list.

Here, related songs refer to, for example, songs that have the same predetermined attributes. Specifically, for example, related songs may be songs included in the same album, songs by the same artist, songs of the same genre, or songs by the same artist. The songs may belong to the same category or may be released in the same year. Note that these are just examples, and related songs may be defined according to other predetermined related criteria. The song data acquisition unit 202 identifies related songs, for example, by referring to a database about songs.

The playback control unit 200 may output a song that is automatically selected from among the songs listed in the related song list, or may output a song that is selected according to an instruction from the user. In the case of automatic selection, for example, the playback control unit 200 selects one of the songs included in the related song list (i.e., one of the songs included in the candidates) during playback of the section from the start timing to the end timing of the recorded data. ), the selected songs are controlled to be output according to a predetermined selection rule. At this time, the song data acquisition section 202 acquires the audio data of the selected song, and the playback control section 200 outputs the song using this audio data. Note that any arbitrary rule can be used as the predetermined selection rule. The predetermined selection rule may be, for example, a rule of selecting at random, or a rule of selecting a song released most recently. Automatic selection saves the user the trouble of specifying.

Further, when selecting according to an instruction from the user, for example, the instruction receiving unit 201 receives an instruction from the user to specify one of the songs included in the related song list (that is, songs included in the candidates). accept. Then, the playback control unit 200 controls to output the music specified by the instruction during playback of the section from the start timing to the end timing of the recorded data. At this time, the song data acquisition section 202 acquires the audio data of the specified song, and the playback control section 200 outputs the song using this audio data. By selecting a song according to the user's instructions, the user can listen to the desired song.

Further, in this embodiment as well, a configuration may be provided to alert the user in order to encourage the user to select a song. That is, the playback device 20 may include the attention calling section 203. In this case, the alert unit 203 issues an alert when audio data of a song specified by the song information set in the recording data by the recording device 10 cannot be obtained. For example, when the user attempts to play recorded data for which no song has been selected, the attention calling unit 203 may display a message prompting the user to select a song from the related song list. Further, the attention calling unit 203 may display a mark indicating this on the list screen of reproducible recorded data for recorded data in which a song for which audio data cannot be obtained is set. By doing so, it is possible to prompt the user to select a song.

Note that when the instruction receiving unit 201 receives an instruction to specify one of the songs included in the related song list, the song data acquisition unit 202 receives information indicating the location where the audio data of the specified song exists. You may also save it. In addition, when audio data of a song specified by the song information set in the recording data by the recording device 10 can be obtained, the song data acquisition unit 202 transmits information indicating the location where the audio data of this song exists to the storage device. 25 may be stored. Further, the song data acquisition unit 202 may set information indicating a location where audio data exists in the recorded data as tag information. Here, the information indicating the location where the audio data of the song exists may be, for example, a file path, a URL (Uniform Resource Locator) of a network streaming service that distributes the song, or a download It may be a URL for doing so. In this way, by temporarily storing information indicating the location where the audio data of a song exists, it becomes possible to easily obtain the audio data when repeatedly playing recorded data.

Next, in the present embodiment, an example of a process flow for determining a song to be output in the interval from the start timing to the end timing when playing recorded data will be described. FIG. 12 is a flowchart illustrating an example of the process flow for determining the music to be output. An example of the flow of processing for determining the music to be output will be described below with reference to FIG. 12.

The song data acquisition unit 202 searches for audio data of a song specified by the song information set in the recording data by the recording device 10. Specifically, the following steps S500 to S502 are performed.

In step S500, the song data acquisition unit 202 searches the playback device 20. Specifically, the song data acquisition unit 202 searches a music library stored in the storage device 25. If the audio data to be searched does not exist in the playback device 20 (No in step S500), the process moves to step S501. On the other hand, if the audio data to be searched exists in the playback device 20 (Yes in step S500), the process moves to step S507.

In step S501, the song data acquisition unit 202 searches for search target audio data via the network. In this step, the song data acquisition unit 202 searches for data that can be used by users on the network. For example, the song data acquisition unit 202 may search online storage on the cloud, or search for audio data provided by a music distribution service available to the user. Note that at this time, the song data acquisition unit 202 may access these connection destinations by using the account of the user of the playback device 20, for example. If the audio data to be searched is not available via the network (No in step S501), the process moves to step S502. On the other hand, if the audio data to be searched is available via the network (Yes in step S501), the process moves to step S507.

In step S502, a search is performed to determine whether or not audio data to be searched exists among audio data provided by a music distribution service that is not currently available to the user. Here, the music distribution service that is not currently available to the user specifically refers to, for example, a service for which the user has not entered into a contract or purchase procedure for service provision with the music distribution service. If the audio data to be searched can be provided by a music distribution service that is not currently available (Yes in step S502), the process moves to step S503; otherwise (No in step S502), the process moves to step S505. Move to.

In step S503, the song data acquisition unit 202 presents on the display 21 information regarding a music distribution service that can provide the audio data to be searched and that is not currently available to the user. That is, the user is notified that audio data can be obtained by using such a music distribution service. This prompts the user to enter into a streaming distribution contract or download purchase procedure with such a music distribution service. In this way, if it is necessary to provide audio data from a music distribution service that is not currently available to the user in order to acquire audio data of a song specified by the song information, the song data acquisition unit 202 Information regarding music distribution services may also be presented. By doing so, it becomes possible to output a song specified by the song information. After step S503, the process moves to step S504.

In step S504, the song data acquisition unit 202 determines whether a music distribution service that can provide the audio data to be searched is available. If the music distribution service is not available (No in step S504), the process moves to step S505. On the other hand, if the music distribution service becomes available (Yes in step S504), the process moves to step S507.

In step S505, the song data acquisition unit 202 creates a list of songs that are related to the song specified by the song information and for which audio data can be obtained (ie, a related song list). After step S505, the process moves to step S506.

In step S506, the instruction receiving unit 201 receives an instruction from the user to select a song from the related song list. After step S506, the process moves to step S507.

In step S507, the playback control unit 200 determines a song to be output during playback in the section from the start timing to the end timing. If the audio data to be searched exists in the playback device 20 (Yes in step S500) or is available via the network (Yes in step S501 or Yes in step S504), the playback control unit 200 controls the recording The song specified by the song information set in the data is the song to be output. On the other hand, if the song is specified from the related song list (step S506), the playback control unit 200 sets the specified song as the song to be output. Note that, upon determining the song to be output, the playback control unit 200 may update the song information set as tag information in the recorded data to the song information of the determined song. That is, the playback control unit 200 may update the tag information.

The third embodiment has been described above. According to this embodiment, a related song list is created. Therefore, even if the audio data of the song specified by the song information set in the recording data by the recording device 10 cannot be obtained, the song can be played back.

<Embodiment 4>
In the embodiment described above, when the storage device 14 runs out of free space, control is performed to store new recorded data by sequentially overwriting the oldest recorded data. Therefore, there is a risk that the recorded data of the section in which the song was sung may be deleted due to the overwriting process. In order to prevent this, as described above, it is also possible to prohibit overwriting of the recorded data of the section in which the song is sung. However, in that case, the more recorded data that is prohibited to be overwritten increases in the storage device 14, the more the storage capacity of new recorded data decreases, and the more recorded data that is prohibited to be overwritten increases, the more the storage capacity of new recorded data decreases. There is a risk that you will not be able to remember important images. To prevent this, as described above, it is also possible to cancel the overwrite prohibition when the storage capacity of new recorded data becomes small. However, in that case, there is a risk that the recorded data of the section in which the song was sung may be deleted. As described above, in the embodiment described above, it may be difficult to save the recorded data of the section in which the song was sung and the more important recorded data. Therefore, the present embodiment provides a technique that can appropriately store recorded data. Hereinafter, points different from the embodiments described above will be explained.

FIG. 13 is a block diagram showing an example of the hardware configuration of the recording device 10 in this embodiment. The configuration shown in FIG. 13 differs from the configuration shown in FIG. 2 in that it further includes an external storage interface 16 to which a removable medium 19 such as a memory card or a USB (Universal Serial Bus) memory is connected. The external storage interface 16 is an interface circuit that communicably connects the removable medium 19 that functions as an external storage for the recording device 10 and the recording device 10 .

The storage control unit 102 in the present embodiment controls to move the data of the section from the start timing to the end timing of the recorded data, that is, the data of the section in which the song was sung, from the storage device 14 to another storage device. . More specifically, if the free space of the other storage device mentioned above is greater than or equal to the size of the data in the section, the storage control unit 102 moves the data in the section from the storage device 14 to the other storage device. Here, the other storage device refers to any external storage other than the storage device 14, and specifically, it may be the removable media 19 or an online storage connected via the network interface 13. It's okay. Note that when online storage is used as another storage device, the external storage interface 16 does not necessarily need to be provided.

Next, in this embodiment, an example of the flow of operations for moving recorded data of a section in which a song was sung to another storage device will be described. FIG. 14 is a flowchart showing an example of the flow of processing for moving recorded data of a section in which a song was sung to another storage device. An example of the flow of moving recorded data will be described below with reference to FIG.

In step S600, the storage control unit 102 determines whether recorded data including the section in which the song was sung is stored in the storage device 14. That is, the storage control unit 102 determines whether the recording data in which the tag information of the start timing and the tag information of the end timing are set is stored in the storage device 14. If there is no such recorded data, the storage control unit 102 repeats this step after a predetermined period of time (No in step S600). On the other hand, if such recorded data is stored in the storage device 14 (Yes in step S600), the process moves to step S601.

In step S601, the storage control unit 102 sets the recorded data including the section in which the song was sung, found in step S600, as a moving target. That is, the storage control unit 102 selects partial recorded data corresponding to the section of the recorded data including the section in which the song is sung to be moved to another storage device.

In step S602, the storage control unit 102 determines whether there is another storage device that is accessible and has sufficient free space. Here, "sufficient free space" means that there is free space that is larger than the size of the data to be moved. Note that if there may be a plurality of other storage devices, the storage control unit 102 sequentially checks whether all of these storage devices are accessible and have sufficient free space. If any other storage device is accessible and has sufficient free space (Yes in step S602), the process moves to step S603. On the other hand, if there is no other storage device that is accessible and has sufficient free space (No in step S602), the processes from step S600 onward are performed again after a predetermined period of time.

In step S603, the storage control unit 102 moves the data targeted for movement in step S601 from the storage device 14 to another storage device that is accessible and has sufficient free space. As a result, the storage capacity of the storage device 14 is recovered by the size of the data targeted for movement. After a predetermined period of time after step S603, the processes from step S600 onwards are performed again.

The fourth embodiment has been described above. As described above, in this embodiment, the storage control unit 102 moves the data of the section in which the song was sung from the storage device 14 where the recorded data is stored to another storage device. This prevents the data of the section in which the song was sung from being overwritten and deleted in the storage device 14, and also allows more important images, such as images from a vehicle accident, to be stored. It is possible to secure storage capacity for

<Embodiment 5>
This embodiment provides a technique that improves the convenience of specifying the playback start position of recorded data. Specifically, when the user sings a song, the playback device 20 moves the playback position of the recorded data to a position corresponding to the song. Hereinafter, points different from the embodiments described above will be explained.

FIG. 15 is a block diagram showing an example of the hardware configuration of the playback device 20 according to this embodiment. The playback device 20 shown in FIG. 15 differs from the configuration shown in FIG. 3 in that a microphone 27 is added. The microphone 27 is a device that collects sounds around the playback device 20, and acquires the voice of the user of the playback device.

FIG. 16 is a block diagram showing an example of the functional configuration of the playback device 20 according to the present embodiment. The playback device 20 shown in FIG. 16 differs from the configuration shown in FIG. 5 in that a song information acquisition section 204 is added. The processing of the music information acquisition unit 204 is realized by, for example, executing a program by the processor 26. Note that the playback device 20 of this embodiment may also further include an attention calling unit 203 as in FIG. 9 .

The song information acquisition unit 204 acquires song information (for example, identification information) of a song sung by the user using the playback device 20 from the audio data of the sound acquired by the microphone 27 . Further, the song information acquisition unit 204 may further acquire position information for specifying the position in the entire song for a portion of the song that corresponds to the beginning of the song by the user. In other words, when a certain song is sung, the song information acquisition unit 204 determines where in the song the song begins (for example, from the beginning of the song, from the chorus, from the second verse). You may also obtain location information that specifies the location. Note that the song information acquisition unit 204 can acquire the song information and position information using the same process as the song information acquisition unit 100 in the recording device 10. Therefore, the song information acquisition unit 204 acquires the song information and position information of the song whose similarity to the audio data to be analyzed exceeds a predetermined threshold value from the server. Note that when acquiring the song information and position information in the playback device 20, the analysis process may be performed by the song information acquisition unit 204 (that is, the playback device 20) instead of the server.

When the song information acquisition unit 204 acquires song information, the playback control unit 200 according to the present embodiment controls the playback control unit 200 to select a song that has been acquired by the song information acquisition unit 204 of the playback device 20 from among the song information added to the recorded data. The section associated with the song information corresponding to the information is set as the playback start position. Then, the playback control unit 200 plays back the recorded data from the playback start position while outputting the song. FIG. 17 is a schematic diagram showing the playback start position of recorded data. As shown in FIG. 17, for example, assume that in some recorded data, a section to which song A is associated and a section to which song B is associated are set. In this case, when the song information acquisition unit 204 of the playback device 20 acquires the song information of the song A, the playback control unit 200 starts the playback at the start timing of the section of the song A (specifically, for example, the start timing of the section of the song A). Playback of recorded data is started from position P). Thereby, the user can easily start viewing the part where the desired song is sung.

In controlling the playback start position described above, skipping is performed to the beginning of the section in which the song is sung, but by using position information, it is also possible to skip to the middle of the section in which the song is sung. That is, the recorded data includes the position information of the beginning of the song (hereinafter referred to as first position information), and the song information acquisition unit 204 of the playback device 20 includes the position information of the beginning of the song (hereinafter referred to as second position information). , the playback control unit 200 may adjust the playback start position of the recorded data according to the difference between the first position information and the second position information. This will be specifically explained using FIG. 18. FIG. 18 is a schematic diagram showing the playback start position of recorded data. As shown in FIG. 18, for example, assume that in some recorded data, a section to which song A is associated and a section to which song B is associated are set. In particular, it is assumed that the start timing of the section to which song A is associated is associated with position information representing a position 1 minute and 15 seconds after the beginning of the song as first position information. That is, in this case, when reproducing the section in which the song is sung from the start timing, the playback device 20 starts from the middle of the song (more specifically, from the position 1 minute and 15 seconds after the beginning of the song). Output. Here, the song information acquisition unit 204 of the playback device 20 acquires the song information of song A, and further acquires position information representing a position 2 minutes and 15 seconds after the beginning of song A as second position information. Suppose we did. In this case, the playback control unit 200 starts the recording data from the playback start position P, which is the time when one minute (=[2 minutes 15 seconds] - [1 minute 15 seconds]) has elapsed from the beginning of the section in which song A was sung. Start playing. Thereby, the playback start position can be determined in more detail.

Furthermore, in the present embodiment, the user may instruct whether or not to perform playback from the playback start position based on the acquired song information. That is, when a section of recorded data associated with song information corresponding to the song information acquired by the song information acquisition section 204 of the playback device 20 is specified, the instruction receiving section 201 sets the section as the playback start position. You may also accept instructions that allow you to do so. In this case, when an instruction to permit is obtained, the reproduction control unit 200 sets the relevant section as the reproduction start position. According to such a configuration, the user can decide whether or not to control the playback start position based on the song information.

In addition, when a plurality of sections of recorded data associated with the song information corresponding to the song information acquired by the song information acquisition section 204 of the playback device 20 are identified, the instruction receiving section 201 starts the reproduction of which section. An instruction to select the location may be accepted. In this case, the playback control unit 200 sets the section selected by the instruction as the playback start position. According to such a configuration, a more appropriate section can be reproduced.

Note that the instruction receiving unit 201 may display time information or a list of thumbnail images at the playback start position based on the song information on the display 21 in order to clearly indicate to the user the playback start position based on the song information.

Next, an example of the flow of the operation for determining the playback start position in this embodiment will be described. FIG. 19 is a flowchart showing an example of the flow of the operation for determining the playback start position of recorded data. An example of the flow of the operation for determining the playback start position will be described below with reference to FIG.

In step S700, the playback control unit 200 determines whether the song information acquisition unit 204 has acquired song information. If the song information has not been acquired (No in step S700), this process is repeated. When the song information is acquired (Yes in step S700), the process moves to step S701.

In step S701, the playback control unit 200 determines whether the recorded data to be played includes a section in which the song information acquired by the song information acquisition unit 204 is set. If there is no section in which the song information is set in the recorded data to be played back (No in step S701), the process returns to step S700. On the other hand, if there is a section in which the song information is set in the recorded data to be played back (Yes in step S701), the process moves to step S702.

In step S702, the reproduction control unit 200 determines whether the instruction receiving unit 201 has received a predetermined instruction. Specifically, the playback control unit 200 determines whether an instruction to permit playback from the playback start position based on the acquired song information has been received. Furthermore, if the recorded data to be played includes multiple sections in which the acquired song information is set, the playback control unit 200 determines whether an instruction to select which section is to be the playback start position is received. Determine whether or not. If such an instruction is not accepted (No in step S702), the process returns to step S700. On the other hand, if such an instruction is accepted (Yes in step S702), the process moves to step S703.

In step S703, the playback control unit 200 sets the position based on the song information as the playback start position. As a result, playback of the recorded data is started from the section in which the song sung by the user of the playback device 20 is sung.

The fifth embodiment has been described above. As described above, in this embodiment, the recorded data is played back from the position corresponding to the song sung by the user of the playback device 20. Therefore, according to the present embodiment, it is possible to improve the convenience of specifying the playback start position of recorded data.

<Embodiment 6>
The present embodiment provides a technology that allows song information to be appropriately set in recorded data even when a plurality of people sing a song at the same time in a car. Hereinafter, points different from the embodiments described above will be explained.

The recorded data generation unit 101 according to the present embodiment uses the camera 11 to collect song information of a song sung by a person determined according to a predetermined priority rule, the start timing of the song, and the end timing of the song. Recording data added to the video data of the shot moving image is generated. According to such a configuration, even if a plurality of people sing a song at the same time in the car, the song information can be appropriately set in the recorded data.

The above-mentioned predetermined priority rule determines the person from whom music information is to be acquired, for example, based on the priority set for the seats in the car. Specifically, for example, in order to prioritize the acquisition of song information of a song sung by a person in a specific seat, a rule is used that prioritizes the specific seat over other seats. In other words, this particular seat is given a higher priority than other seats. Note that the specific seat may be, for example, the right rear seat, the left rear seat, or the driver's seat. Note that these seat types are merely examples, and a seat at any position provided in the vehicle may be used as the specific seat. In addition, in order to give priority to any one seat over other seats, a higher priority may be set for only that one seat than other seats, or a priority for multiple seats may be set. In order to determine this, a priority level corresponding to the priority level may be set for each seat.

When a predetermined priority rule is used that determines the person from whom song information is to be acquired based on the priority set for the seats in the car, the recorded data generation unit 101 generates the following recorded data. do. That is, in this case, the recorded data generation unit 101 uses the camera to collect the song information of the song sung by the person in the seat selected based on the priority, the start timing of the song, and the end timing of the song. 11 to generate recorded data added to the video data of the moving image photographed.

Note that the song information of the song sung by the person in the seat selected based on the priority can be acquired as follows. For example, by using a directional microphone as the microphone 12, it is possible to extract the voice of a person in a specific seat. Alternatively, the voice of a person at a particular seat may be extracted by known blind signal separation using a plurality of microphones 12 such as a microphone array. The song information acquisition unit 100 uses the voice extracted in this way to acquire song information. That is, the song information acquisition unit 100 uses the audio data of the voice from the seat selected based on the priority to acquire the song information of the song sung by the person in the seat.

Further, the above-mentioned predetermined priority rule may determine the person from whom the song information is to be acquired based on the priority set for the person's voice. Specifically, for example, in order to prioritize the acquisition of song information for a song sung by a specific person, a rule is used in which the voice of the specific person is given priority over the voices of other people. In other words, this specific person's voice is given a higher priority than other people's voices. In addition, in order to give priority to one person over the others, the voice of one person may be given a higher priority than the voices of other people, or the voice of multiple people may be given higher priority than the voices of other people. In order to determine the priority order, priorities corresponding to the priority order may be set for the voices of a plurality of people.

When a predetermined priority rule is used that determines the person from whom song information is to be acquired based on the priority set for the person's voice, the recorded data generation unit 101 generates the following recorded data. do. That is, in this case, the recorded data generation unit 101 uses the camera 11 to photograph the song information of the song sung by the person whose voice is selected based on the priority, the start timing of the song, and the end timing of the song. This process generates recorded data added to the video data of the video.

Note that the song information of the song sung by the person whose voice is selected based on the priority can be acquired as follows. The song information acquisition unit 100 extracts the voice of a specific person from the audio data acquired by the microphone 12 using a known sound source separation technique. For example, the song information acquisition unit 100 inputs audio data acquired by the microphone 12 to a model that has been trained in advance by machine learning such as deep learning using training data that includes the voice of a specific person. Separation processing is performed to extract the voice of a specific person. The song information acquisition unit 100 uses the voice extracted in this way to acquire song information. That is, the song information acquisition unit 100 uses the audio data of the voice selected based on the priority to acquire the song information of the song sung by the person with the voice.

The priority may be used for purposes other than determining the acquisition target of song information. For example, as shown in FIG. 20, the recording device 10 may further include a camera control section 103. The processing of the camera control unit 103 is realized, for example, by execution of a program by the processor 15. The camera control unit 103 controls the camera 11 that photographs the interior of the vehicle so as to zoom in and photograph the person determined according to the priority rule. According to such a configuration, it is possible to generate recorded data focusing on a person with a high priority. Specifically, when the priority set for the seats in the car is used, the camera control unit 103 uses, for example, information that predefines the relative positional relationship between each seat in the car and the camera 11. to control the photographing range and zoom magnification of the camera 11. In addition, when the priority set for a person's voice is used, the camera control unit 103 performs, for example, a face recognition process that compares a pre-registered face image of the person with an image captured by the camera 11. , the position of the person is specified, and the photographing range and zoom magnification of the camera 11 are controlled.

Note that, instead of zooming in on the person determined according to the priority rule at the time of photographing, an image zoomed in on the person may be displayed when the recorded data is played back. That is, the playback control unit 200 of the playback device 20 may display a zoomed-in image of the person when playing back the section in which the song is sung. When priorities set for seats in a car are used, specifically, for example, a zoomed-in video display of a person determined according to the priority rules is realized in the following manner. The recorded data generation unit 101 adds information indicating to which seat the music information of the person is added to the recorded data for each section in which the song is sung. The playback control unit 200 of the playback device 20 uses this information and predefined information indicating in which area in the image shot by the camera 11 each seat is to be photographed. By enlarging the area, a zoomed-in image of the person is displayed. Further, when the priority set for the voice of a person is used, specifically, for example, a video display zoomed in on the person determined according to the priority rule is realized in the following manner. The recorded data generation unit 101 identifies the position of the person in the image through face recognition processing that compares the face image of the person registered in advance with the image captured by the camera 11, and converts information indicating this position. is added to the recorded data for each section in which the song is sung. Using this information, the playback control unit 200 of the playback device 20 displays an image zoomed in on the person by enlarging the area where the person is located in the image taken by the camera 11.

By the way, the song information acquisition unit 100 acquires the song information for one person selected based on the priority, but may acquire the song information for each of a plurality of people. That is, the song information acquisition unit 100 may acquire the song information of each song sung simultaneously while the camera 11 is shooting from the audio data of the sound acquired by the microphone 12. In this case, the song information acquisition unit 100 extracts each person's voice from the audio data of the sound acquired by the microphone 12, and uses the audio data of each voice to acquire the respective song information. As a result, the song information acquisition unit 100 can identify the songs sung by each person, so it can be determined whether one person selected based on the priority is singing the same song as another person. can do. Therefore, when music information is acquired for each of a plurality of people, this determination result can be used to control other processes. Specifically, for example, the camera control unit 103 may perform the following control. If the song information of the songs sung at the same time is the same, the camera control unit 103 controls the camera 11 to photograph the entire inside of the car without zooming in on the person determined according to the priority rule, If the song information of the selected song is different, the camera 11 may be controlled to zoom in on and photograph the person determined according to the priority rule. According to such a configuration, it is possible to generate recorded data of video content depending on the situation. Note that video display under such control may be performed when the recorded data is played back. In this case, the recorded data generation section 101 adds information indicating whether or not the song information of the songs sung at the same time is the same for each section in which the songs are sung, and the playback control section 200 adds this information to the recorded data. The above-mentioned control is performed based on the information.

Next, the tag information setting operation by the recording device 10 according to the present embodiment will be explained. FIG. 21 is a flowchart showing an example of the tag information setting operation by the recording device 10. An example of the tag information setting operation will be described below with reference to FIG. 21, but descriptions of processes similar to the flowchart shown in FIG. 6 will be omitted as appropriate.

In the flowchart shown in FIG. 21, if the user's voice is detected in step S100 (Yes in step S100), the process moves to step S800.

In step S800, the song information acquisition unit 100 extracts the audio data of the person from whom the song information is to be obtained from the audio data from the microphone 12. When acquiring song information for only one person selected based on the priority, the song information acquisition unit 100 extracts the audio data of the voice of the one person from the audio data from the microphone 12. Further, when acquiring song information for each of a plurality of people, the song information acquisition unit 100 extracts audio data of each of these people's voices from the audio data from the microphone 12. After step S800, the process moves to step S801.

In step S801, the song information acquisition unit 100 starts accumulating the audio data extracted in step S800 in the buffer.

Next, in step S802, the song information acquisition unit 100 determines whether the voice of the person from whom song information is to be acquired is no longer detected. If the voice of the person whose song information is to be acquired is no longer detected (Yes in step S802), the process moves to S109 described with reference to FIG. 6. That is, in step S109, if only the tag information of the start timing is set, the tag information of the end timing is set. Then, the accumulation of audio data is completed. If the voice of the person whose song information is to be acquired is detected (No in step S802), the process moves to step S103.

The processing after step S103 is the same as the processing of each step described with reference to FIG. However, in this embodiment, as described above, the tag information of the song information, start timing, and end timing set in the recording data is the song information, start timing, and tag information for one person determined according to a predetermined priority rule. and end timing.

The sixth embodiment has been described above. As described above, in this embodiment, which person's song information is added to the recorded data is controlled according to the priority rule. Therefore, even if a plurality of people sing a song simultaneously in the car, the song information can be appropriately set in the recorded data.

Note that the present invention is not limited to the above embodiments, and can be modified as appropriate without departing from the spirit. Unless technically inconsistent, some or all of the configurations or functions described in each embodiment may be combined with the configurations or functions described in other embodiments. Furthermore, some functions or configurations of each embodiment may be omitted unless technically inconsistent.

1 Video system 10 Recording device 11 Camera 12 Microphone 13 Network interface 14 Storage device 15 Processor 16 External storage interface 19 Removable media 20 Playback device 21 Display 22 Speaker 23 Input device 24 Network interface 25 Storage device 26 Processor 27 Microphone 100 Song information acquisition unit 101 Recorded data generation section 102 Storage control section 103 Camera control section 200 Playback control section 201 Instruction reception section 202 Song data acquisition section 203 Attention calling section 204 Song information acquisition section

Claims (9)

A camera that takes videos inside or outside the vehicle,
a microphone that captures sounds inside the car while the camera is shooting;
Send the audio data of the voice acquired by the microphone during shooting with the camera to a server, acquire song information that is identification information that uniquely identifies a song corresponding to the audio data of the voice from the server, a song information acquisition unit that acquires the start timing and end timing in the video;
A recording device comprising: a recording data generation unit that generates recording data in which the song information, the start timing, and the end timing are added to video data of a moving image shot by the camera. The song information acquisition unit further acquires position information for specifying a position in the entire song for a portion of the song acquired by the microphone at the start timing,
The recording device according to claim 1, wherein the recorded data generation section generates the recorded data to which the position information is further added. When the song information acquisition section acquires a plurality of candidates as song information of one song sung during the shooting with the camera, the recording data generation section converts the acquired plurality of song information into song information of the song. The recording device according to claim 1 or 2, wherein the recording data added as a candidate list is generated. The acquired song information further includes audio data of the song,
The recording device according to claim 1 or 2, wherein the recorded data generation unit generates the recorded data in which the video data and audio data of the song are superimposed. a recording device that generates recorded data;
a playback device that acquires the recorded data from the recording device and plays the recorded data;
The recording device is
A camera that takes videos inside or outside the vehicle,
a microphone that captures sounds inside the car while the camera is shooting;
Send the audio data of the voice acquired by the microphone during shooting with the camera to a server, acquire song information that is identification information that uniquely identifies the song corresponding to the audio data of the voice from the server, a song information acquisition unit that acquires the start timing and end timing in the video;
a recorded data generation unit that generates recorded data in which the song information, the start timing, and the end timing are added to video data of a moving image shot by the camera;
The playback device includes:
When playing back the recorded data, playback control controls to output the song from a speaker using the audio data of the song specified by the song information during playback of the section from the start timing to the end timing. A video system with a division. The song information acquisition unit further acquires position information for specifying a position in the entire song for a portion of the song acquired by the microphone at the start timing,
The recorded data generation unit generates the recorded data to which the position information is further added,
The video system according to claim 5, wherein, when the song is output from the speaker , the playback control unit controls the song to be played from a position specified based on the position information and output from the speaker. The playback device further includes an instruction receiving unit that receives an instruction regarding the output of the song from the speaker in the section from the start timing to the end timing,
The video system according to claim 5 , wherein the playback control unit controls the speaker to output a song in a section from the start timing to the end timing in accordance with the instruction. The audio data of the voice inside the car acquired by a microphone during the shooting of a camera that takes a video inside or outside the car is transmitted to a server, and the identification information that uniquely identifies the song corresponding to the audio data of the voice from the server. Acquire certain song information, acquire the start timing and end timing of the song in the video,
A recording method that generates recorded data by adding the song information, the start timing, and the end timing to video data of a moving image shot by the camera. The audio data of the voice inside the car acquired by a microphone during the shooting of a camera that takes a video inside or outside the car is transmitted to a server, and the identification information that uniquely identifies the song corresponding to the audio data of the voice from the server. a song information obtaining step of obtaining certain song information and obtaining the start timing and end timing of the song in the video;
A program that causes a computer to execute a recording data generation step of generating recording data in which the song information, the start timing, and the end timing are added to video data of a moving image shot by the camera.

Images