JP7275382B2 - Recording control device and recording control method - Google Patents

Description

The present disclosure relates to a recording control device and a recording control method.

In recent years, technologies have been proposed for vehicle recording devices such as drive recorders and EDRs (Event Data Recorders) that record images of the vehicle surroundings before and after an accident and voices inside the vehicle for ex-post analysis of accident factors. ing. For example, Patent Literature 1 proposes a recording device that records driver's consciousness.

JP 2014-21767 A

Sounds such as running sounds of vehicles and music from audio equipment act as noise on the driver's voice, so the important driver's voice may not be recorded with high accuracy. In other words, the prior art cannot record the driver's voice for accurate reconstruction after an event such as an accident. Therefore, there was a problem that various signals for performing appropriate analysis after the event could not be recorded.

Therefore, the present disclosure has been made in view of the above problems, and aims to provide a technique capable of recording signals for performing appropriate analysis after an event.

A recording control device according to the present disclosure is a recording control device that controls recording by a recording device, and includes an outside image acquisition unit that acquires an image around the vehicle as an outside image signal, and a driver that is the state of the driver of the vehicle. a driver state acquisition unit that acquires the state as a driver state signal; a first voice acquisition unit that acquires at least one of voice inside and outside the vehicle as a first voice signal; a second audio acquisition unit that acquires, as a second audio signal, audio with a higher precision of the driver's voice than the first audio signal, and detects occurrence of a specific event in the vehicle, using the audio acquisition device having an event detection unit, when the occurrence of a specific event is detected by the event detection unit, the outside video signal, the driver status signal, and the first signal within a predetermined period including the occurrence of the specific event; A recording control section for controlling recording of the audio signal and the second audio signal in the recording device as event recording information is provided. The driver status signal is included in the external video signal by electronic watermarking and recorded in the recording device .

According to the present disclosure, when the occurrence of a specific event is detected, the external video signal, the driver status signal, and the first audio signal are generated within a predetermined period including the occurrence of the specific event. , and the second audio signal as event recording information in the recording device. With such a configuration, it is possible to record a signal for appropriate analysis after the event.

Objects, features, aspects and advantages of the present disclosure will become more apparent from the following detailed description and accompanying drawings.

1 is a block diagram showing the configuration of a recording control device according to Embodiment 1; FIG. 2 is a block diagram showing the configuration of a recording control device according to Embodiment 2; FIG. FIG. 9 is a block diagram showing the configuration of a microphone array according to Embodiment 2; FIG. FIG. 10 is a diagram showing a recording structure of a recording apparatus according to Embodiment 2; FIG. FIG. 10 is a diagram showing the recording structure of a unit recording block according to Embodiment 2; 9 is a flowchart showing the operation of the recording control device according to Embodiment 2; FIG. 13 is a block diagram showing the configuration of a microphone array according to Modification 5 of Embodiment 2; FIG. 11 is a block diagram showing the configuration of a recording control device according to Embodiment 3; FIG. 12 is a block diagram showing the configuration of a second voice acquisition unit and its surroundings according to Embodiment 3; FIG. 13 is a block diagram showing the configuration of a recording control device according to Embodiment 4; FIG. 13 is a diagram showing a recording structure of a unit recording block according to Embodiment 5; FIG. 11 is a block diagram showing a hardware configuration of a recording control device according to another modified example; FIG. FIG. 11 is a block diagram showing a hardware configuration of a recording control device according to another modified example; FIG. FIG. 11 is a block diagram showing the configuration of a server according to another modified example; FIG. 11 is a block diagram showing the configuration of a communication terminal according to another modified example;

<Embodiment 1>
FIG. 1 is a block diagram showing the configuration of a recording control device 1 according to the first embodiment. Although the recording control device 1 is mainly described below as being mounted on a vehicle, it does not have to be mounted on the vehicle as will be described later.

The recording control device 1 of FIG. 1 is connected to the recording device 51 so as to be able to communicate wirelessly or by wire, and controls the recording of the recording device 51 . The recording device 51 may be provided in the vehicle such as a drive recorder including a recording memory and an EDR, or may be provided outside the vehicle.

The recording control device 1 of FIG. and

The vehicle exterior image acquisition unit 11 acquires an image around the vehicle as an exterior image signal. The vehicle exterior image acquisition unit 11 includes, for example, at least one of an exterior image capturing device that captures images around the vehicle and an exterior image input unit that is an interface thereof. The image around the vehicle is, for example, an image in front of the vehicle, but is not limited to this.

The driver state acquisition unit 12 acquires the driver state, which is the state of the driver of the vehicle, as a driver state signal. The driver state acquisition unit 12 includes, for example, at least one of a driver sensing device that detects the driver state and a driver state input unit that is an interface thereof. An example of the driver's state will be described in Embodiment 2 and the like.

The first sound acquisition unit 13 acquires at least one of the sound inside the vehicle and outside the vehicle as the first sound signal. The second voice acquisition unit 14 uses a voice acquisition device having directivity to acquire voice, and acquires the voice of the driver's side with higher precision than the first voice signal as the second voice signal.

The first voice acquisition unit 13 includes, for example, at least one of an omnidirectional microphone provided at a specific position of the vehicle and a voice input device as an interface therefor. However, if the first audio acquisition unit 13 can acquire the first audio signal with higher accuracy than the second audio signal for audio other than the driver's voice, instead of the omnidirectional microphone, for example, the front of the vehicle It may also include a microphone or the like having directivity to.

The second voice acquisition unit 14 includes, for example, at least one of a voice acquisition device having directivity for voice acquisition and a voice input device as an interface thereof. The sound acquisition device is, for example, a microphone array that has directivity for sound acquisition by performing beamforming processing on a plurality of microphones, or one or more directional microphones that have directivity for sound acquisition by an internal mechanism. be. The voice acquisition device may be provided at a specific position of the vehicle, preferably in front of the driver.

The event detector 15 detects occurrence of a specific event in the vehicle. The event detection section 15 detects occurrence of a specific event based on sensor information detected by a sensor of the event detection section 15 or a sensor provided outside the recording control apparatus 1 . The sensor information is, for example, acceleration due to an impact received by the vehicle, and the specific event is, for example, an event such as a vehicle accident that is subject to ex-post analysis.

When the event detection unit 15 detects the occurrence of a specific event, the recording control unit 16 outputs the outside video signal, the driver status signal, and the Control is performed to record the first audio signal and the second audio signal in the recording device 51 as event recording information.

<Summary of Embodiment 1>
According to the recording control apparatus 1 according to the first embodiment as described above, when the occurrence of a specific event is detected, the event record information within a predetermined period including the occurrence of the specific event is It is recorded in the recording device 51 . Since the event recording information includes the first audio signal with high precision of the voice other than the driver's voice and the second audio signal with high precision of the driver's voice, according to the above configuration, it is possible to properly perform the event after the event. The signal can be recorded for further analysis. In addition, since the event recording information includes the video signal outside the vehicle and the driver's condition signal, according to the above configuration, it is possible to record the signal for performing appropriate analysis after the event. For example, if a driver's utterance such as "dangerous" is recorded, the record can be used to support when and under what circumstances the driver recognized a dangerous situation, contributing to accident analysis. material. In addition, when the driver's utterance such as "from the right" is recorded, the record can be used as evidence that some dangerous element is approaching from the right, and can contribute to accident analysis.

<Embodiment 2>
FIG. 2 is a block diagram showing the configuration of the recording control device 1 according to the second embodiment. Hereinafter, among the constituent elements according to the second embodiment, constituent elements that are the same as or similar to the above-described constituent elements are denoted by the same or similar reference numerals, and different constituent elements will be mainly described.

The recording control device 1 of FIG. 2 is connected to a recording device 51, a driver sensing device 52, and a microphone array 53 so as to communicate wirelessly or by wire. The recording device 51 is the same as the recording device 51 described in the first embodiment.

The driver sensing device 52 detects, for example, the driver's physiological state such as pulse and respiration, and the driver's psychological state such as arousal and tension as the driver state described in the first embodiment. In this case, the driver sensing device 52 includes, for example, a pulse detection device that detects the pulse using infrared rays, a respiration detection device that detects respiration using a millimeter wave radar, and image processing of the driver's face image. It includes a recognition device that recognizes the degree of arousal and the degree of tension from facial expressions. The driver state and the driver sensing device 52 are not limited to those described above.

The microphone array 53 is used for the second sound acquisition section 14 . FIG. 3 is a block diagram showing the configuration of the microphone array 53 according to the second embodiment. This microphone array 53 is included in the concept of the sound acquisition device described in the first embodiment, and may be provided in the recording control device 1 or may not be provided in the recording control device 1 .

The microphone array 53 of FIG. 3 includes a plurality of microphones 53a1 to 53a5 and a processing filter 53b that performs beamforming processing on the plurality of microphones 53a1 to 53a5. As the beam forming process, for example, the processes described in Japanese Patent Application Laid-Open No. 2007-523514 and Japanese Patent Application Laid-Open No. 2016-127300 may be used. An overview of the beamforming process according to the second embodiment will be described below.

The processing filter 53b considers the phase difference that occurs according to the difference in distance between the driver's mouth and each of the plurality of microphones 53a1 to 53a5, and processes the audio signals MS1 to MS5 output from the plurality of microphones 53a1 to 53a5. Perform filter processing to add This amplifies the audio signal of the voice from the driver's mouth and/or attenuates the audio signal of other voices to improve the accuracy of the driver's voice, i.e. clarity and signal-to-noise ratio. becomes higher. As a result, even if each of the plurality of microphones 53a1 to 53a5 is an omnidirectional microphone, the microphone array 53 as a whole can acquire directivity for sound.

Here, in FIG. 3, a case will be described where S0(t) is the speech voice from the driver 61 and N(t) is the noise voice that is the other voice. In this case, each of the multiple microphones 53a1 to 53a5 acquires the sound fn{S0(t)}+gn{N(t)}. Note that fn is the transfer characteristic of speech voice, and gn is the transfer characteristic of noise voice.

The processing filter 53b performs the beam forming process on the audio signals MS1 to MS5 of the sounds acquired by the plurality of microphones 53a1 to 53a5. As a result, the processing filter 53b removes the audio signal of the noise audio N(t) from the audio signals MS1 to MS5 to generate the audio audio signal SS(t) as the second audio signal. Although this audio signal SS(t) is slightly affected by the noise audio N(t), it becomes an audio signal close to the audio signal of only the utterance audio S0(t).

As in Embodiment 1, the recording control device 1 of FIG. A recording control unit 16 is provided.

The vehicle exterior image acquisition unit 11 acquires an image around the vehicle as an exterior image signal. The driver state acquisition unit 12 acquires the driver state including the physiological state and psychological state of the driver detected by the driver sensing device 52 as a driver state signal.

The first sound acquisition unit 13 acquires at least one of sounds inside and outside the vehicle as a first sound signal from a microphone (not shown) different from the microphone array 53 . The second voice acquisition unit 14 acquires a voice having a higher precision of the driver's voice than the first voice signal, such as the utterance voice S0(t), in which the influence of the noise voice N(t) is reduced from the microphone array 53. , as the second audio signal SS(t).

The event detector 15 detects occurrence of a specific event in the vehicle. When the event detection unit 15 detects the occurrence of a specific event, the recording control unit 16 controls recording of event record information in the recording device 51 within a predetermined period including the occurrence of the specific event. I do. The event recording information recorded in the recording device 51 includes not only the external video signal, the driver status signal, the first audio signal, and the second audio signal, but also the event occurrence time and the event occurrence position. It may include information such as

FIG. 4 is a diagram showing the recording structure of the recording device 51 according to the second embodiment. The recording device 51 includes a constant recording area 51a and a protection recording area 51b. The recording control unit 16 records six minutes of event recording information corresponding to one file as one unit recording block 51c in the constant recording area 51a and the protection recording area 51b.

FIG. 5 is a diagram showing the recording structure of the unit recording block 51c. The unit recording block 51c corresponds to 6 minutes of event recording information, that is, 6 minutes of the outside video signal, the driver status signal, the first audio signal, and the second audio signal. In the second embodiment, the first audio signal is recorded as stereo L-channel audio, and the second audio signal is recorded as stereo R-channel audio. Conversely, the first audio signal may be recorded as stereo R-channel audio, and the second audio signal may be recorded as stereo L-channel audio.

Returning to FIG. 4, the recording control unit 16 records the unit recording block 51c in the constant recording area 51a at regular intervals (6 minutes in this example) after the recording control device 1 and the recording device 51 are activated. In the constant recording area 51a, for example, 24-hour unit recording blocks 51c, that is, 240 (=24 hours/6 minutes) unit recording blocks 51c can be recorded.

The recording control unit 16 records the unit recording blocks 51c in the order of the partial areas LM1, LM2, . . . , LM240 shown in FIG. When the power of the recording control device 1 and the recording device 51 is turned off, the recording control unit 16 stores information for identifying the partial area in which the unit recording block 51c is recorded, and when the power is turned on after that, the next part of the partial area is stored. A unit recording block 51c is recorded from the partial area. After recording the unit recording block 51c in the partial area LM240, the recording control unit 16 returns to the partial area LM1 and overwrites and records the unit recording block 51c in the partial areas LM1, LM2, . That is, the recording control section 16 uses the constant recording area 51a as a ring buffer.

When the event detection unit 15 detects the occurrence of a specific event, the recording control unit 16 protects and records the unit recording block 51c at the time of occurrence of the specific event among the unit recording blocks 51c recorded in the constant recording area 51a. Record in area 51b. At this time, in addition to the unit recording block 51c at the time of occurrence of the specific event, the recording control unit 16 selects at least one of the unit recording block 51c of the partial area before the specific event and the unit recording block 51c of the partial area after it. One may be recorded in the protection recording area 51b.

For example, ten unit recording blocks 51c can be recorded in the protection recording area 51b. The recording control unit 16 overwrites the unit recording block 51c recorded in the constant recording area 51a with a new unit recording block 51c. On the other hand, the recording control unit 16 prohibits erasing by overwriting the unit recording block 51c recorded in the protection recording area 51b until a predetermined condition is satisfied. Predetermined conditions include, for example, that a button (not shown) is operated, and that a certain period of time, such as one month, has elapsed since recording. As a result, the recording period of the event recording information recorded in the protection recording area 51b when the occurrence of the specific event is detected is recorded in the constant recording area 51a when the occurrence of the specific event is not detected. It is longer than the recording period of the event recording information.

The file structure, the recordable time of the constant recording area 51a and the protected recording area 51b, the capacity of the unit recording block 51c, and the role of the folder are not limited to those described above. good or big.

The recording device 51 may be provided in the vehicle, or may be provided outside the vehicle, for example. Specifically, both the constant recording area 51a and the protection recording area 51b may be provided in the vehicle, or may be provided outside the vehicle, or one of the constant recording area 51a and the protection recording area 51b may be provided in the vehicle. and the other may be provided outside the vehicle.

Also, in the above example, the constant recording area 51a and the protection recording area 51b are separated, but they do not have to be separated. For example, a protection recording area 51b may be defined within the constant recording area 51a. Specifically, when the event detection unit 15 detects the occurrence of a specific event, the recording control unit 16 selects unit recording at the time of occurrence of the specific event among the unit recording blocks 51c recorded in the constant recording area 51a. A protected recording area flag is given to the block 51c. Then, the recording control unit 16 excludes the unit recording block 51c to which the protected recording area flag is added from the constant cycle recording. In such a configuration, since the unit recording block 51c to which the protection recording area flag is assigned is substantially the same as the unit recording block 51c recorded in the protection recording area 51b, the protection recording area 51c is always included in the recording area 51a. 51b is substantially defined.

<Action>
FIG. 6 is a flow chart showing the operation of the recording control device 1 according to the second embodiment. The operation of FIG. 6 is started, for example, when the recording control device 1 and the recording device 51 are powered on by starting a drive source such as a vehicle engine.

In step S1, the exterior image acquisition unit 11 acquires an exterior image signal. In step S2, the driver's state acquisition unit 12 acquires a driver's state signal. At step S3, the first audio acquisition unit 13 acquires the first audio signal. In step S4, the second audio acquisition unit 14 acquires the second audio signal. Note that the order of operations in steps S1 to S4 may be changed as appropriate.

In step S5, the recording control unit 16 records the event recording information obtained by the operations of steps S1 to S4 in the constant recording area 51a.

In step S6, the recording control unit 16 determines whether or not the event detection unit 15 has detected the occurrence of a specific event. If it is determined that the occurrence of the specific event has been detected, the process proceeds to step S7, and if it is determined that the occurrence of the specific event has not been detected, the process returns to step S1. Note that an emergency power supply and a capacitor capable of supplying power in such a case are provided in the recording control apparatus 1 so that the operations after step S6 can be performed even when a specific event such as a vehicle accident occurs. may be provided in

In step S7, the recording control unit 16 records the event recording information within a predetermined period including the occurrence of the specific event in the protection recording area 51b.

In step S8, the recording control unit 16 determines whether or not the vehicle has finished driving. When it is determined that the operation has ended, the operation of FIG. 6 ends, and when it is not determined that the operation has ended, the process returns to step S1.

<Summary of Embodiment 2>
According to the recording control apparatus 1 according to the second embodiment as described above, when the occurrence of a specific event is detected, the event record information within a predetermined period including the occurrence of the specific event is It is recorded in the recording device 51 . Therefore, as in the first embodiment, it is possible to record a signal for appropriate analysis after the event.

In the second embodiment, the recording period of the event record information recorded in the protection recording area 51b when the occurrence of a specific event is detected is set to the constant recording area when the occurrence of the specific event is not detected. It is longer than the recording period of the event recording information recorded in 51a. Therefore, it is possible to suppress as much as possible erasure of the signal for performing appropriate analysis after the event.

<Modification 1 of Embodiment 2>
In Embodiment 2, the voice acquisition device is the microphone array 53, but it is not limited to this. As described in Embodiment 1, the sound acquisition device may be a single directional microphone that has directivity for sound acquisition by an internal mechanism and the directivity is designed for the driver. With such a configuration, the accuracy of the driver's voice is lower than that of the configuration in which the microphone array 53 is used as the voice acquisition device, but the cost can be reduced.

<Modification 2 of Embodiment 2>
In the second embodiment, the driver state acquired by the driver state acquiring unit 12 includes the physiological state and psychological state of the driver detected by the driver sensing device 52, but is not limited to this.

For example, the driver state acquired by the driver state acquisition unit 12 may include the appearance state (physical state) of the driver's body. The appearance state of the driver's body includes, for example, the opened/closed state of the driver's mouth, the driver's looking aside state, the driver's line of sight, the opened/closed state of the driver's eyes, and the inclination state of the driver's body. . In this case, the driver sensing device 52 includes, for example, an image processing device, an ultrasonic sensor, a pressure sensor, and the like. The driver sensing device 52 may determine the presence or absence of a voice from the driver based on the opening and closing of the driver's mouth, or may detect the presence or absence of the driver's voice by lip-reading the opening and closing of the driver's mouth. Words may be detected.

Further, for example, the driver state acquired by the driver state acquisition unit 12 may include an image of the driver. In this case, the driver sensing device 52 includes, for example, an image capturing device.

Further, for example, the driver state acquired by the driver state acquisition unit 12 is at least one of the physiological state of the driver, the psychological state of the driver, the physical appearance of the driver, and an image of the driver. It may include the above.

<Modification 3 of Embodiment 2>
In the second embodiment, as shown in FIG. 5, in the unit recording block 51c, the driver state signal is recorded in a recording area separate from the outside video signal, the first audio signal, and the second audio signal. is not limited to this. For example, the driver status signal may be cryptographically included in the external video signal like an electronic watermark and recorded in the recording device 51, thereby being recorded in the same recording area as the external video signal. In this case, the external video signal, the first audio signal, and the second audio signal can be easily confirmed with a normal video playback application or a normal video playback device, and the driver status signal is a dedicated video playback. It can be analyzed with an application or a dedicated video playback device.

It should be noted that the video recording method such as the video signal outside the vehicle is preferably a general method such as AVCHD (registered trademark), MPEG2, MPEG4, MP4, JPEG2000, and AVI, but is not limited to these.

Further, in the configuration of Modification 3 in which the driver's state signal is included in the outside video signal and recorded in the recording device 51, the driver's state signal may be recorded in the recording device 51 as a video signal. For example, the numerical value of the driver's condition represented by the driver's condition signal may be recorded as a video signal so that it can be displayed as a graph with time as a parameter, that is, so that it can be displayed like the voltage waveform display of an oscilloscope. good. At this time, the display area of the outside video signal is larger than the display area of the driver status signal. may be occupied.

<Modification 4 of Embodiment 2>
The event detection unit 15 may detect occurrence of a specific event based on the second audio signal acquired by the second audio acquisition unit 14 . For example, the event detection unit 15 recognizes the driver's voice by analyzing the second audio signal, and the driver's voice includes a voice indicating fear, a voice commanding the start of recording, and a live commentary of the situation outside the vehicle. It may be determined whether it is at least one of the voices to be played. Then, when it is determined that the driver's voice is at least one of the above, the event detection unit 15 may determine that the occurrence of a specific event has been detected.

Further, when the driver state represented by the driver state signal includes an image of the driver, the event detector 15 may detect occurrence of a specific event based on the image. For example, the event detection unit 15 may determine that the occurrence of a specific event has been detected when the driver's image analysis determines that the expression of the driver indicates fear.

<Modification 5 of Embodiment 2>
In Embodiment 2, the first audio acquisition unit 13 acquires the first audio signal from a microphone different from the microphone array 53, but the invention is not limited to this. As shown in FIG. 7, for example, the first sound acquisition unit 13 acquires at least one of the inside and outside of the vehicle acquired by any one of the plurality of microphones of the microphone array 53 (microphone 53a3 in FIG. 7). may be obtained as the first audio signal. With such a configuration, the number of microphones can be reduced.

<Embodiment 3>
FIG. 8 is a block diagram showing the configuration of the recording control device 1 according to the third embodiment. Hereinafter, among the constituent elements according to the third embodiment, constituent elements that are the same as or similar to the above-described constituent elements are denoted by the same or similar reference numerals, and different constituent elements will be mainly described.

The recording control device 1 of FIG. 8 is connected not only to the recording device 51, the driver sensing device 52 and the microphone array 53 so as to be able to communicate wirelessly or by wire, but also to the audio device 54 so as to be able to communicate wirelessly or by wire. ing.

The audio device 54 generates a musical sound based on an audio signal for generating a musical sound, and outputs the musical sound from a speaker 54a, which is an audio output unit of the vehicle. The audio device 54 includes, for example, a reproducing device that reproduces the sound of music from an audio signal such as music data for reproduction, an amplifier that amplifies the audio signal, and the like.

The second sound acquisition unit 14 reduces, preferably eliminates, the music sound signal corresponding to the sound of the music from the second sound signal based on the audio signal for generating the sound of the music output from the speaker 54a. .

FIG. 9 is a block diagram showing the configuration of the second voice acquisition unit 14 and its surroundings according to the third embodiment. M(t) is the sound of music emitted from the speaker 54a into the vehicle. When beamforming processing is performed by the processing filter 53b, the audio signal SS(t) of the audio close to the speech S0(t) and the music audio signal MM(t) corresponding to the music audio M(t) are mixed. mixed audio is generated.

The second audio acquisition unit 14 provisionally acquires mixed audio from the processing filter 53b as a second audio signal. Also, an audio signal Mo(t) for generating a musical sound M(t) is input to the second sound acquisition unit 14 .

The second speech acquisition unit 14 according to the third embodiment includes an adaptive filter 14a. The adaptive filter 14a performs music canceling processing to reduce the music audio signal MM(t) from the temporarily acquired second audio signal based on the audio signal Mo(t), thereby reducing the second audio signal SSS ( t). Since the correlation between the audio signal Mo(t) and the music audio signal MM(t) included in the provisionally obtained second audio signal is high, the second audio signal SSS ( t) is an audio signal close to the audio signal of only the uttered voice S0(t).

<Summary of Embodiment 3>
According to the recording control device 1 according to the third embodiment as described above, since the music audio signal is reduced from the second audio signal, even if the music is loud, the accuracy of the driver's voice is high. A second audio signal can be recorded.

<Embodiment 4>
FIG. 10 is a block diagram showing the configuration of the recording control device 1 according to the fourth embodiment. Hereinafter, among the constituent elements according to the fourth embodiment, constituent elements that are the same as or similar to the above-described constituent elements are denoted by the same or similar reference numerals, and different constituent elements will be mainly described.

The driver sensing device 52 captures an image of the driver and detects the position of the driver's mouth based on the image. In this case, the driver sensing device 52 may detect the position of the mouth using image processing such as image recognition, or may detect the position of the mouth using an ultrasonic sensor or the like.

In the second embodiment, the microphone array 53 performs filtering (beamforming) on the assumption that the position of the driver's mouth is constant. In contrast, in the fourth embodiment, the microphone array 53 adjusts the parameters of the filtering process (beam forming process) based on the position of the mouth of the driver, so that the directivity of voice acquisition is adjusted to the position of the mouth of the driver. Control the directivity of voice acquisition to correspond to According to such a configuration, the prediction accuracy of the time difference (phase difference) at which the driver's voice such as the uttered voice S0(t) reaches each of the microphones 53a1 to 53a5, and thus the prediction accuracy of the transfer characteristics can be improved. can be enhanced. Therefore, the precision of the driver's voice can be further improved.

<Summary of Embodiment 4>
According to the recording control apparatus 1 according to the fourth embodiment as described above, the directivity of the microphone array 53 is controlled based on the position of the driver's mouth. According to such a configuration, it is possible to record the second audio signal in which the precision of the driver's voice is further improved by relatively easy processing.

<Embodiment 5>
The block configuration of the recording control device 1 according to the fifth embodiment is the same as the block configuration of the recording control device 1 according to the second embodiment (FIG. 2). Hereinafter, among the constituent elements according to the fifth embodiment, constituent elements that are the same as or similar to the above-described constituent elements are denoted by the same or similar reference numerals, and different constituent elements will be mainly described.

In the second embodiment, as shown in FIG. 5, the first audio signal is recorded as stereo L-channel audio, and the second audio signal is recorded as stereo R-channel audio. When these sounds are reproduced by a reproduction device (not shown), at least one of the sounds inside and outside the vehicle from the L channel and the driver's side sound from the R channel are reproduced at the same time. However, since it is assumed that these sounds include sounds from different sound sources, sounds that are difficult to hear will be reproduced.

Therefore, in the fifth embodiment, as shown in FIG. 11, both the first audio signal and the second audio signal are stereo audio signals, and one of the first audio signal and the second audio signal is recorded as main audio. , the other of the first audio signal and the second audio signal is recorded as sub-audio. In FIG. 11, the first audio signal is recorded as the main audio of the L and R channels, and the second audio signal is recorded as the secondary audio of the L and R channels, but the present invention is not limited to this. For example, the second audio signal may be recorded as the L channel and R channel main audio, and the first audio signal may be recorded as the L channel and R channel sub audio.

In order to record the stereo format first audio signal, the first audio acquisition unit 13 may acquire the first audio signal using a stereo microphone. Similarly, in order to record the second audio signal in stereo format, the second audio acquisition unit 14 may acquire the second audio signal using a stereo microphone.

<Summary of Embodiment 5>
According to the recording control apparatus 1 according to the fifth embodiment as described above, one of the first audio signal and the second audio signal is recorded as the main audio signal, and the other of the first audio signal and the second audio signal is recorded as the secondary audio signal. Recorded as audio. According to such a configuration, by switching the main audio and the sub-audio at the time of audio reproduction, the audio of the first audio signal and the audio of the second audio signal can be switched and reproduced. Also, by simultaneously reproducing the main audio and the sub-audio when reproducing the audio, it is possible to easily compare the audio of the first audio signal and the audio of the second audio signal.

<Modification 1 of Embodiment 5>
The recording control unit 16 may record the event record information in the recording device 51 in a format that can be directly reproduced by a video reproduction application or a video reproduction device such as a television. According to such a configuration, it is possible to switch between the main audio and the sub-audio and reproduce the video simply by connecting or inserting the recording device 51 into a video reproducing device such as a television. In the configuration in which the driver status signal is included in the video signal outside the vehicle and recorded in the recording device 51, the event recording information can be composed only of the video signal and the audio signal. be able to.

<Modification 2 of Embodiment 5>
In the above description, the recording control device 1 and the recording device 51 did not have the reproduction function for reproducing the event record information, but they may have the reproduction function. According to such a configuration, the event record information can be reproduced without connecting a dedicated reproduction device to the recording control device 1 or the like.

<Other Modifications>
The vehicle exterior video acquisition unit 11, the driver state acquisition unit 12, the first audio acquisition unit 13, the second audio acquisition unit 14, the event detection unit 15, and the recording control unit 16 shown in FIG. Part 11”. The outside image acquisition unit 11 and the like are realized by a processing circuit 81 shown in FIG. 12 . That is, the processing circuit 81 includes an exterior image acquisition unit 11 that acquires an image around the vehicle as an exterior image signal, and a driver state acquisition unit 12 that acquires a driver state, which is the state of the driver of the vehicle, as a driver state signal. Then, using a first sound acquisition unit 13 that acquires at least one of the sound inside the vehicle and outside the vehicle as the first sound signal, and a sound acquisition device that has directivity for acquiring sound, from the first sound signal A second audio acquisition unit 14 that acquires a highly accurate audio signal from the driver's side as a second audio signal, an event detection unit 15 that detects the occurrence of a specific event in the vehicle, and an event detection unit 15. When the occurrence of the event is detected, the outside video signal, the driver status signal, the first audio signal, and the second audio signal within a predetermined period including the occurrence of the specific event , and a recording control unit 16 that controls recording in the recording device 51 as event record information. Dedicated hardware may be applied to the processing circuit 81, or a processor that executes a program stored in a memory may be applied. Processors include, for example, central processing units, processing units, arithmetic units, microprocessors, microcomputers, and DSPs (Digital Signal Processors).

When the processing circuit 81 is dedicated hardware, the processing circuit 81 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each function of each unit such as the outside image acquisition unit 11 may be realized by a circuit in which processing circuits are distributed, or the functions of each unit may be collectively realized by one processing circuit.

When the processing circuit 81 is a processor, the functions of the vehicle exterior video acquisition unit 11 and the like are realized by combining software and the like. Software and the like correspond to, for example, software, firmware, or software and firmware. Software or the like is written as a program and stored in memory. As shown in FIG. 13, a processor 82 applied to a processing circuit 81 reads out and executes a program stored in a memory 83 to implement the functions of each unit. That is, when the recording control device 1 is executed by the processing circuit 81, a step of acquiring an image around the vehicle as an outside image signal and a step of acquiring a driver state, which is the state of the driver of the vehicle, as a driver state signal. a step of acquiring at least one of voice inside the vehicle and outside the vehicle as a first voice signal; a step of acquiring a voice with high precision of the voice on the side as a second audio signal; a step of detecting occurrence of a specific event in the vehicle; a step of performing control to record the external video signal, the driver status signal, the first audio signal, and the second audio signal in the recording device 51 as event recording information within a predetermined period including time; , has a memory 83 for storing the program that will result in execution. In other words, it can be said that this program causes a computer to execute the procedures and methods of the outside video acquisition unit 11 and the like. Here, the memory 83 is, for example, a non-volatile or Volatile semiconductor memories, HDDs (Hard Disk Drives), magnetic disks, flexible disks, optical disks, compact disks, mini disks, DVDs (Digital Versatile Disks), their drive devices, etc., or any storage media that will be used in the future. may

The configuration in which each function of the vehicle exterior video acquisition unit 11 and the like is realized by either hardware or software has been described above. However, the present invention is not limited to this, and a configuration may be adopted in which a part of the vehicle exterior video acquisition unit 11 and the like is realized by dedicated hardware, and another part is realized by software or the like. For example, the outside image acquisition unit 11, the driver state acquisition unit 12, the first audio acquisition unit 13, the second audio acquisition unit 14, and the event detection unit 15 are provided with a processing circuit 81, an interface and a receiver as dedicated hardware. etc., and other functions can be realized by the processing circuit 81 as the processor 82 reading and executing the program stored in the memory 83 .

As described above, the processing circuit 81 can implement each of the functions described above using hardware, software, etc., or a combination thereof.

The recording control device 1 described above includes at least one of a vehicle device such as a DMS (Driver Monitoring System), a communication terminal including a mobile terminal such as a mobile phone, a smart phone, and a tablet, and a vehicle device and a communication terminal. It can also be applied to a recording control system constructed as a system by appropriately combining functions of installed applications and a server. In this case, each function or each component of the recording control apparatus 1 described above may be arranged in a distributed manner in each device constituting the system, or may be centrally arranged in one of the devices. good.

FIG. 14 is a block diagram showing the configuration of the server 91 according to this modification. A server 91 in FIG. 14 includes a communication unit 91a and a control unit 91b, and is capable of wirelessly communicating with a vehicle device 93 such as a DMS device of a vehicle 92 .

A vehicle device 93 corresponding to the outside video acquisition unit 11, the driver state acquisition unit 12, the first voice acquisition unit 13, and the second voice acquisition unit 14 receives the outside video signal, the driver state signal, the first voice signal, and obtaining a second audio signal and obtaining event-related information about a particular event.

The communication unit 91 a receives event-related information acquired by the vehicle device 93 by performing wireless communication with the vehicle device 93 .

The control unit 91b has functions similar to those of the event detection unit 15 and the recording control unit 16 shown in FIG. ing. That is, the control unit 91b detects occurrence of a specific event in the vehicle based on the event-related information. Then, when the occurrence of a specific event is detected, the control unit 91b controls the external video signal, the driver status signal, and the first audio signal within a predetermined period including the occurrence of the specific event. and the second audio signal in the recording device 51 as event recording information. The communication unit 91 a then transmits the control signal generated by the control unit 91 b to the vehicle device 93 . According to the server 91 configured in this way, the same effects as those of the recording control device 1 described in the first embodiment can be obtained.

FIG. 15 is a block diagram showing the configuration of a communication terminal 96 according to this modification. A communication terminal 96 of FIG. 15 includes a communication unit 96a similar to the communication unit 91a and a control unit 96b similar to the control unit 91b, and can perform wireless communication with a vehicle device 98 of a vehicle 97. ing. As the communication terminal 96, for example, a portable terminal such as a mobile phone, a smart phone, or a tablet carried by the driver of the vehicle 97 is applied. According to the communication terminal 96 configured in this way, it is possible to obtain the same effect as the recording control device 1 described in the first embodiment.

It should be noted that it is possible to freely combine each embodiment and each modification, and to modify or omit each embodiment and each modification as appropriate.

The above description is, in all aspects, illustrative and not restrictive. It is understood that innumerable variations not illustrated can be envisaged.

1 recording control device, 11 outside video acquisition unit, 12 driver state acquisition unit, 13 first audio acquisition unit, 14 second audio acquisition unit, 15 event detection unit, 16 recording control unit, 51 recording device, 53 microphone array, 53a Microphone.

Claims (10)

A recording control device for controlling recording of a recording device,
an outside image acquisition unit that acquires an image around the vehicle as an outside image signal;
a driver state acquisition unit that acquires a driver state, which is the state of the driver of the vehicle, as a driver state signal;
a first sound acquisition unit that acquires at least one of sounds inside the vehicle and outside the vehicle as a first sound signal;
a second sound acquisition unit that acquires, as a second sound signal, a sound of the driver's side whose accuracy is higher than that of the first sound signal, using a sound acquisition device having directivity for sound acquisition;
an event detection unit that detects occurrence of a specific event in the vehicle;
When the occurrence of the specific event is detected by the event detection unit, the external video signal, the driver status signal, and the second signal are detected within a predetermined period including the occurrence of the specific event. a recording control unit that controls recording of the first audio signal and the second audio signal in the recording device as event recording information;
The recording control device, wherein the driver status signal is included in the vehicle exterior video signal by electronic watermarking and recorded in the recording device. The recording control device according to claim 1 ,
The recording control device, wherein the driver state includes at least one of a physiological state and a psychological state of the driver. The recording control device according to claim 1 ,
The recording control device, wherein the driver condition includes the appearance condition of the driver's body. The recording control device according to claim 1 ,
The recording control device, wherein the driver state includes an image of the driver. The recording control device according to claim 1 ,
A recording control device, wherein the sound capture device includes a microphone array that controls the directivity based on the position of the driver's mouth. The recording control device according to claim 1 ,
The recording control device, wherein the driver state signal is a video signal, and the numerical value of the driver state is recorded in the recording device in a manner displayed in a graph with time as a parameter. The recording control device according to claim 1 ,
The event detection unit
A recording control device that detects occurrence of the specific event based on the second audio signal acquired by the second audio acquisition unit. The recording control device according to claim 1 ,
the sound acquisition device includes a microphone array having a plurality of microphones;
The first voice acquisition unit is
A recording control device that acquires, as the first audio signal, at least one of the sound inside the vehicle and outside the vehicle that is acquired by any one of the plurality of microphones. The recording control device according to claim 1 ,
The recording period of the event recording information recorded in the recording device when the occurrence of the specific event is detected by the event detection unit is set when the occurrence of the specific event is not detected by the event detection unit. recording control device longer than the recording period of the event recording information recorded in the recording device. A recording control method for controlling recording of a recording device, comprising:
The outside image acquisition unit acquires an image around the vehicle as an outside image signal,
A driver state acquisition unit acquires a driver state, which is the state of the driver of the vehicle, as a driver state signal,
A first sound acquisition unit acquires at least one of the sound inside the vehicle and outside the vehicle as a first sound signal,
a second voice acquisition unit using a voice acquisition device having directivity for voice acquisition to acquire, as a second voice signal, a voice having higher precision of the voice of the driver than the first voice signal;
an event detection unit detecting occurrence of a specific event in the vehicle;
When the occurrence of the specific event is detected, the recording control unit outputs the outside video signal, the driver status signal, and the second signal within a predetermined period including the occurrence of the specific event. performing control to record the first audio signal and the second audio signal as event recording information in the recording device;
The recording control method, wherein the driver status signal is included in the vehicle exterior video signal by electronic watermarking and recorded in the recording device.

Images