JP2022056922A - Vehicle recording control device, recording control method, and program - Google Patents

Abstract

To suppress dangerous driving by protecting a passenger's privacy.SOLUTION: A control device 100 comprises a vehicle information acquiring part 130 for acquiring vehicle information indicating behavior of a vehicle, that includes at least either driving operation information indicating a driving operation for the vehicle or a state information indicating a state of the vehicle; a photographic data acquisition part 120 for acquiring in-vehicle video data as photographic data; a mental condition determining part 131 for determining whether or not a mental condition of a passenger is good, based on the behavior of the vehicle recognized from at least either the photographic data or the vehicle information; a sound-recording setting part 132 for setting a sound-recording object person from the photographic data, when the mental condition determining part 131 determines that the mental condition of the passenger is bad; and a recording control part 123 for storing voice data of the sound-recording object person set by the sound-recording setting part 132, when the mental condition determining part 131 determines that the mental condition of the passenger is bad.SELECTED DRAWING: Figure 1

Description

The present invention relates to a vehicle record control device, a record control method and a program.

A technique for controlling the running of a vehicle based on emotions estimated by using biometric information of the driver or passenger of the vehicle is disclosed (see, for example, Patent Document 1). A drive recorder discloses a technique for stopping recording for a call made by a mobile phone in a vehicle for privacy protection (see, for example, Patent Document 2).

Japanese Unexamined Patent Publication No. 2017-136922 Japanese Unexamined Patent Publication No. 2019-133420

For example, if the occupant of the vehicle becomes irritated or the mental condition deteriorates, there is a risk of driving in a so-called dangerous manner such as driving in a hurry or intentionally interrupting. Dangerous driving may induce an accident in the own vehicle or another vehicle, so it is desirable to suppress dangerous driving. In addition, if the inside of the vehicle can be photographed, it is desirable to protect the privacy of the occupants.

The present invention has been made in view of the above, and an object of the present invention is to protect the privacy of occupants and suppress dangerous driving.

In order to solve the above-mentioned problems and achieve the object, the vehicle record control device according to the present invention provides at least one of driving operation information indicating a driving operation for a vehicle and state information indicating the state of the vehicle. A vehicle information acquisition unit that acquires vehicle information indicating the behavior of the vehicle, an in-vehicle video data that is an image of the inside of the vehicle, and an in-vehicle audio data that is an in-house voice of the vehicle are acquired as shooting data. The mental state of the occupant based on the behavior of the vehicle recognized from at least one of the data acquisition unit, the photography data acquired by the photography data acquisition unit, and the vehicle information acquired by the vehicle information acquisition unit. The mental state determination unit for determining the quality of the vehicle, the recording setting unit for changing the recording setting of the in-vehicle voice data when the mental state determination unit determines that the mental state of the occupant is bad, and the mental state determination unit When it is determined that the mental state of the occupant is bad, the recording control unit for storing the in-vehicle voice data based on the recording setting set by the recording setting unit is provided.

The recording control method according to the present invention includes at least one of driving operation information indicating a driving operation for a vehicle and state information indicating the state of the vehicle, and acquires vehicle information indicating the behavior of the vehicle. The step, the shooting data acquisition step of acquiring the in-vehicle video data which is the image inside the vehicle of the vehicle, and the in-vehicle audio data which is the sound of the inside of the vehicle as shooting data, and the shooting acquired by the shooting data acquisition step. A mental state determination step for determining the quality of the occupant's mental state based on the behavior of the vehicle recognized from at least one of the data and the vehicle information acquired by the vehicle information acquisition step, and the mental state. When the determination step determines that the mental state of the occupant is bad, the recording setting step for changing the recording setting from the shooting data, and the mental state determination step determines that the mental state of the occupant is bad, the recording. The setting step includes a recording control step of saving the sound of the shooting data based on the recording setting set.

The program according to the present invention includes at least one of driving operation information indicating a driving operation for a vehicle and state information indicating the state of the vehicle, and a vehicle information acquisition step for acquiring vehicle information indicating the behavior of the vehicle. , The shooting data acquisition step of acquiring the vehicle interior video data which is the image inside the vehicle and the vehicle interior audio data which is the voice of the vehicle inside the vehicle as shooting data, and the shooting data acquired by the shooting data acquisition step. , A mental state determination step for determining the quality of the occupant's mental state based on the behavior of the vehicle recognized from at least one of the vehicle information acquired by the vehicle information acquisition step, and the mental state determination step. When it is determined that the mental state of the occupant is bad, the recording setting step of changing the recording setting so as to record the voice of the recording target person having a bad mental state from the shooting data and the mental state determination step of the occupant When it is determined that the mental state is bad, a computer operating as a vehicle recording control device is made to execute a recording control step for storing the sound of the shooting data based on the recording setting set by the recording setting step.

According to the present invention, there is an effect that the privacy of the occupant can be protected and dangerous driving can be suppressed.

FIG. 1 is a block diagram showing a configuration example of a vehicle recording device having a control device according to the first embodiment. FIG. 2 is a flowchart showing an example of a processing flow in the control device according to the first embodiment. FIG. 3 is a flowchart showing an example of the flow of event detection processing in the control device according to the first embodiment. FIG. 4 is a block diagram showing a configuration example of a vehicle recording device having the control device according to the second embodiment. FIG. 5 is a flowchart showing an example of a processing flow in the control device according to the second embodiment. FIG. 6 is a flowchart showing an example of the processing flow in the control device according to the third embodiment.

Hereinafter, embodiments of a vehicle record control device, a record control method, and a program according to the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited to the following embodiments.
[First Embodiment]
<Vehicle recording device>
FIG. 1 is a block diagram showing a configuration example of a vehicle recording device 10 having a vehicle recording control device (hereinafter, referred to as “control device”) 100 according to the first embodiment. The vehicle recording device 10 is a so-called drive recorder that records an event that occurs in a vehicle. The vehicle recording device 10 changes the recording setting of the in-vehicle voice data based on the mental state of the driver or the passenger of the vehicle.

The vehicle recording device 10 may be a portable device that can be used in the vehicle, in addition to the device mounted on the vehicle. Further, the vehicle recording device 10 may be realized by including a function or a configuration of a device, a navigation device, or the like installed in advance in the vehicle. The vehicle recording device 10 includes a first camera (peripheral photographing unit) 211, a second camera (in-vehicle photographing unit) 212, a microphone (sound collecting unit) 213, a recording unit 214, an operation unit 215, and a display unit. 216, speaker 217, accelerometer 218, GPS (Global Positioning System) receiving unit 219, which is an example of GNSS (Global Navigation Satellite System), and CAN (Control Navigation System) interface unit (hereinafter referred to as "IF"). It has 220 and a control device 100.

The first camera 211 is a camera that photographs the front of the vehicle. In the present embodiment, the first camera 211 will be described using a single camera facing in one direction in front of the vehicle, but may be a plurality of camera groups. The first camera 211 is arranged at a position where the front of the vehicle can be photographed, for example, in front of the inside of the vehicle. In the present embodiment, the first camera 211 constantly captures an image while the accessory power of the vehicle is on. The same applies to the second camera 212, which will be described later. The first camera 211 outputs the captured peripheral video data to the imaging data acquisition unit 120 of the control device 100. The peripheral video data is, for example, a moving image composed of an image of 30 frames per second.

The second camera 212 is arranged inside the vehicle so as to face rearward, and photographs the inside of the vehicle. The second camera 212 captures a wide-angle lens, for example, a shooting range of about 180 ° in the horizontal direction. In the present embodiment, the second camera 212 will be described using a single camera facing one direction behind the vehicle, but may be a plurality of camera groups. The second camera 212 outputs the captured vehicle interior video data to the captured data acquisition unit 120 of the control device 100.

When it is not particularly necessary to distinguish between the peripheral video data and the vehicle interior video data, the data will be described as video data. Further, in the present embodiment, the peripheral image is acquired by the first camera 211 and the in-vehicle image is acquired by the second camera 212, but the peripheral image is acquired by one camera, for example, a camera capable of photographing all directions (360 °) of the surroundings. You may acquire the image inside the car.

The microphone 213 collects the sound inside the vehicle. For example, the microphone 213 is arranged in a housing constituting the vehicle recording device 10. The vehicle recording device 10 arranges a plurality of microphones 213 for each occupant, and each microphone 213 acquires the voice of each occupant. Further, the microphone 213 may be arranged in the vicinity of a display unit (not shown) of another in-vehicle navigation system. Further, the microphone 213 may be arranged outside the housing constituting the vehicle recording device 10. Specifically, the microphone 213 may be arranged for each seat of the occupant, for example, in the headrest of the seat or in the structure around the seat. The microphone 213 outputs the collected voice data in the vehicle to the photographing data acquisition unit 120 of the control device 100.

The recording unit 214 is used for temporary storage of data in the vehicle recording device 10. The recording unit 214 is, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a recording unit such as a memory card. Alternatively, it may be an external recording unit wirelessly connected via a communication device (not shown). The recording unit 214 records loop recorded video data or event recording data based on the control signal output from the recording control unit 123 of the control device 100.

The operation unit 215 can receive various operations on the vehicle recording device 10. For example, the operation unit 215 can accept an operation of manually saving the captured video data in the recording unit 214 as event recording data. For example, the operation unit 215 can accept an operation of reproducing the loop recorded video data or the event recording data recorded in the recording unit 214. For example, the operation unit 215 can accept an operation of erasing the event record data recorded in the recording unit 214. For example, the operation unit 215 can accept an operation to end the loop recording. The operation unit 215 outputs the operation information to the operation control unit 125 of the control device 100.

As an example, the display unit 216 is a display device unique to the vehicle recording device 10, a display device shared with other systems including a navigation system, and the like. The display unit 216 may be integrally formed with the first camera 211 or the second camera 212. The display unit 216 is a display including, for example, a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Organic Electro-Luminence) display. In the present embodiment, the display unit 216 is arranged in front of the driver of the vehicle, such as a dashboard, an instrument panel, and a center console. The display unit 216 displays an image based on the image signal output from the display control unit 133 of the control device 100. The display unit 216 displays the image captured by the first camera 211 or the second camera 212, or the image recorded on the recording unit 214.

As an example, the speaker 217 is a voice output device unique to the vehicle recording device 10, or a voice output device shared with other systems including a navigation system. The speaker 217 outputs voice based on the voice signal output from the voice output control unit 134 of the control device 100.

The acceleration sensor 218 is a sensor that detects the acceleration generated on the vehicle. The acceleration sensor 218 outputs the detection result to the acceleration information acquisition unit 126 of the control device 100. The acceleration sensor 218 is, for example, a sensor that detects acceleration in three axial directions. The three-axis directions are the front-rear direction, the left-right direction, and the up-down direction of the vehicle.

The GPS receiving unit 219 receives radio waves from GPS satellites (not shown). The GPS receiving unit 219 outputs the received radio wave signal to the position information acquisition unit 128 of the control device 100.

The IF unit 220 is an interface for acquiring various vehicle information via CAN. The vehicle information includes at least one of driving operation information indicating the driving operation of the vehicle and state information indicating the state of the vehicle. The vehicle information includes information capable of determining the mental state of the occupant leading to dangerous driving, such as sudden steering, sudden braking, sudden start, meandering driving, fanning driving, or intentional interruption. Crew includes the driver and passengers. The driving operation information includes, for example, operation information such as a headlight lighting operation, a horn ringing operation, a steering operation, a brake operation, and an accelerator operation. The state information includes, for example, state information such as the vehicle speed of the vehicle and the acceleration of the vehicle. Further, the state information may include the inter-vehicle distance to another vehicle measured by a sensor (not shown) or the like.
<Vehicle record control device>
The control device 100 is, for example, an arithmetic processing unit (control device) configured by a CPU (Central Processing Unit) or the like. The control device 100 loads the stored program into the memory and executes the instruction included in the program. The control device 100 includes an internal memory (not shown), and the internal memory is used for temporary storage of data in the control device 100 and the like. The control device 100 includes a shooting data acquisition unit 120, a buffer memory 121, a video data processing unit 122, a recording control unit 123, a reproduction control unit 124, an operation control unit 125, and an acceleration connected to the bus 110. Information acquisition unit 126, event detection unit 127, position information acquisition unit 128, vehicle information acquisition unit 130, mental state determination unit 131, cutout range setting unit 132, display control unit 133, and voice output control. It has a portion 134 and.

The shooting data acquisition unit 120 collects peripheral video data from the first camera 211 that shoots the periphery of the vehicle, vehicle interior video data from the second camera 212 that shoots the inside of the vehicle, and voices of vehicle occupants. At least one of the in-vehicle voice data from the microphone 213 is acquired as shooting data. The shooting data acquisition unit 120 outputs the acquired shooting data to the buffer memory 121.

The buffer memory 121 is an internal memory included in the control device 100, and is a memory for temporarily recording video data for a certain period of time acquired by the shooting data acquisition unit 120 while updating it.

The video data processing unit 122 stores the video data temporarily stored in the buffer memory 121, for example, by H.I. Convert to any file format encoded by any codec such as 264 or MPEG-4 (Moving Picture Experts Group), for example MP4 format. The video data processing unit 122 generates video data as a file for a certain period of time from the video data temporarily stored in the buffer memory 121. As a specific example, the video data processing unit 122 generates video data temporarily stored in the buffer memory 121 as a file for 60 seconds in the recording order. The video data processing unit 122 outputs the generated video data to the recording control unit 123. Further, the video data processing unit 122 outputs the generated video data to the display control unit 133 after decoding the image through the reproduction control unit 124. The period of the video data generated as a file is set to 60 seconds as an example, but the period is not limited to this. The video data referred to here may be data including audio in addition to the video captured by the first camera 211.

When the mental state determination unit 131 determines that the mental state of the occupant is bad, the video data processing unit 122 saves the in-vehicle audio data of the recording target person set by the recording setting unit 132.

The recording control unit 123 controls the recording unit 214 to record the video data filed by the video data processing unit 122. The recording control unit 123 uses the video data filed by the video data processing unit 122 as video data that can be overwritten during the period in which the loop recording process is executed, such as when the accessory power supply of the vehicle is ON, as the recording unit 214. Record in. More specifically, the recording control unit 123 continues to record the video data generated by the video data processing unit 122 in the recording unit 214 during the period in which the loop recording process is executed, and the capacity of the recording unit 214 becomes a predetermined capacity. In this case, the oldest video data is overwritten with the new video data and recorded.

Further, when the event detection unit 127 detects the occurrence of an event, the recording control unit 123 saves the shooting data corresponding to the event detection. Further, when the mental state determination unit 131 determines that the mental state of the occupant is bad, the recording control unit 123 saves the in-vehicle voice data of the recording target person set by the recording setting unit 132. The shooting data corresponding to the detection of the event is the video data for a predetermined period in the video data generated by the video data processing unit 122. The recording control unit 123 stores the video data corresponding to the event detection in the recording unit 214 as event recording data for which overwriting is prohibited.

As the event recording data stored in the recording unit 214 by the recording control unit 123, for example, video data for a predetermined period of about 10 seconds before and after the event is detected is copied from the buffer memory 121 and stored as event recording data. ..

The reproduction control unit 124 controls to reproduce the loop recorded video data or the event recording data recorded in the recording unit 214 based on the control signal of the reproduction operation output from the operation control unit 125. The reproduction control unit 124 includes a decoder (not shown) and reproduces various data by decoding the supplied compressed data.

The operation control unit 125 acquires the operation information of the operation received by the operation unit 215. For example, the operation control unit 125 acquires the save operation information indicating the manual save operation of the video data, the reproduction operation information indicating the reproduction operation, or the erasure operation information indicating the erasure operation of the video data, and outputs the control signal. For example, the operation control unit 125 acquires end operation information indicating an operation for ending loop recording and outputs a control signal.

The acceleration information acquisition unit 126 acquires acceleration information indicating the acceleration applied to the vehicle from the detection result of the acceleration sensor 218.

The event detection unit 127 detects an event based on the detection result of the acceleration sensor 218 acquired by the acceleration information acquisition unit 126. More specifically, the event detection unit 127 detects that an event has occurred when the acceleration of the vehicle included in the acceleration information acquired by the acceleration information acquisition unit 126 is equal to or greater than the threshold value.

The position information acquisition unit 128 acquires position information indicating the current position of the vehicle. The position information acquisition unit 128 calculates the position information of the current position of the vehicle by a known method based on the radio wave received by the GPS reception unit 219. The position information acquisition unit 128 may calculate the position information of the current position of the vehicle by using another positioning system including the quasi-zenith satellite.

The vehicle information acquisition unit 130 acquires the vehicle information acquired by the IF unit 220. More specifically, the vehicle information acquisition unit 130 acquires vehicle information indicating the behavior of the vehicle, including at least one of driving operation information indicating a driving operation for the vehicle and state information indicating the state of the vehicle.

The mental state determination unit 131 determines the mental state of the occupant based on the behavior of the vehicle recognized from at least one of the shooting data acquired by the shooting data acquisition unit 120 and the vehicle information acquired by the vehicle information acquisition unit 130. Judge good or bad. When the behavior of the vehicle as shown below is recognized more than a predetermined number of times within a predetermined time, the mental state determination unit 131 may determine that the mental state of the occupant is bad or the mental state is good. The mental state determination unit 131 may determine the quality of the occupant's mental state by combining a plurality of vehicle behaviors as shown below.

The mental state determination unit 131 recognizes the voice of the occupant from the voice data included in the shooting data, and determines whether the occupant's mental state is good or bad. More specifically, the mental state determination unit 131 determines that the mental state of the occupant is bad, for example, when the loudness of the voice of the occupant recognized from the voice data is larger than the loudness of the voice in the calm mental state. do. For example, when the pitch of the voice of the occupant recognized from the voice data is higher or lower than the pitch of the voice in the calm mental state, the mental state determination unit 131 determines that the mental state of the occupant is bad. For example, when the wording issued by the occupant recognized from the voice data corresponds to a predetermined wording stored in the storage unit in advance, the mental state determination unit 131 determines that the mental state of the occupant is bad. The prescribed wording is a wording issued when the mental state is bad. The predetermined wording is, for example, a wording indicating an angry feeling, a wording issued in a state of excitement, a dirty wording, or a wording to curse. The mental state determination unit 131 determines that the mental state is good, for example, when the wording issued by the occupant recognized from the voice data corresponds to the exclusion wording stored in the storage unit in advance. Exclusion words are words that are issued when the mental state is not bad, in other words, when the mental state is calm. The excluded wording is, for example, a thank-you wording or a wording issued when giving way.

For example, the mental state determination unit 131 recognizes the occupant from the in-vehicle video data included in the shooting data, and determines whether the occupant's mental state is good or bad. More specifically, the mental state determination unit 131 is, for example, a facial expression indicating that the occupant's facial expression recognized from the in-vehicle video data is an angry emotion, an excited state, or a yelling facial expression. If so, it is determined that the occupant's mental condition is poor. The mental state determination unit 131 determines that the mental state is good, for example, when the facial expression of the occupant recognized from the in-vehicle video data is a smile or does not express emotions. For example, the mental state determination unit 131 indicates that the shape of the occupant's finger recognized from the in-vehicle video data is that only a specific finger is raised, that only the specific finger is pointed downward, or that the finger is pointed at. If so, it is determined that the occupant's mental condition is poor. For example, when the shape of the occupant's finger recognized from the in-vehicle video data is raising the hand for gratitude, or when the mental state determination unit 131 is moving to convey the intention to give way, the mental state determination unit 131 is mental. Judge that the condition is good. The mental state determination unit 131 determines that the mental state is good, for example, when the inclination of the occupant's head or upper body recognized from the in-vehicle video data is performing a bowing motion.

When the mental state determination unit 131 indicates that the operation information of the lighting operation of the headlight included in the driving operation information indicates that the high beam is lit for the lighting time threshold value or more in the state where another vehicle is present in the vicinity. Judge that the occupant's mental condition is bad. When the operation information of the lighting operation of the headlight included in the driving operation information indicates that the passing is performed in the state where another vehicle is present in the vicinity of the driving operation information, the mental state determination unit 131 indicates that the passing is performed by the lighting time threshold value or more. Judge that the mental state of is bad. Whether or not there is another vehicle in the vicinity can be determined by any known method, and the method is not limited. For example, if another vehicle is recognized by performing image processing on the peripheral video data acquired by the shooting data acquisition unit 120, it may be determined that the other vehicle exists in the vicinity. For example, it may be determined that another vehicle exists in the vicinity based on the detection information from the sensor that detects an obstacle in the vicinity of the vehicle (not shown). For example, when traveling in an urban area, it may be determined that another vehicle exists in the vicinity from the map information (not shown) and the position information of the current position of the vehicle.

When the operation information of the horn ringing operation included in the driving operation information indicates that the horn is sounding at least the ringing time threshold value or at least the ringing frequency threshold value, the mental state determination unit 131 determines that the mental state of the occupant is bad. judge.

When the operation information of the steering operation, the brake operation, or the accelerator operation included in the driving operation information indicates, for example, a sudden steering wheel, a sudden braking, a sudden start, a meandering operation, or the like, the mental state determination unit 131 determines the mental state of the occupant. Judge as bad. More specifically, the mental state determination unit 131 determines, for example, that the steering wheel is steep when the steering operation amount included in the driving operation information is equal to or greater than the steering threshold value. For example, when the brake depression amount included in the driving operation information is equal to or larger than the brake depression amount threshold value, the mental state determination unit 131 determines that the brake is suddenly braked.

For example, when the speed information included in the state information is equal to or higher than the speed threshold value, the mental state determination unit 131 determines that the vehicle has started suddenly. For example, when the speed information included in the state information is equal to or higher than the speed threshold value, or when the acceleration information included in the state information is equal to or higher than the acceleration threshold value, the mental state determination unit 131 determines that the vehicle has started suddenly. For example, when the amount of change in the azimuth angle of the vehicle included in the driving operation information is equal to or greater than the azimuth angle threshold value, the mental state determination unit 131 determines that the vehicle is meandering.

The mental state determination unit 131 determines that the mental state of the occupant is bad when the state in which the inter-vehicle distance included in the state information is equal to or less than the distance threshold value continues for the time threshold value or more. Further, the mental state determination unit 131 may execute determination as to whether or not the inter-vehicle distance is equal to or less than the distance threshold value when the speed is constant or higher. As a result, it is possible to determine the state during traveling, and it is possible to prevent the determination that the mental state is bad, for example, when the distance between vehicles is narrow due to traffic congestion.

When the mental state determination unit 131 determines that the mental state of the occupant is bad, the recording setting unit 132 sets the occupant to the recording target person to save the in-vehicle voice data. More specifically, when the mental state determination unit 131 determines that the mental state of the occupant is bad, the recording setting unit 132 saves only the voice of the occupant with a bad mental state so as to protect the privacy of other occupants. Set to. The recording setting unit 132 cancels the setting of the recording target person to save the in-vehicle voice data when the mental state determination unit 131 determines that the mental state of the occupant is bad and then does not determine the bad condition. That is, the recording setting unit 132 sets not to record the in-vehicle voice of the occupant who has canceled the setting of the recording target person.

The recording setting unit 132 sets a recording target person when the mental state determination unit 131 determines that the occupants in the vehicle include an occupant with a bad mental state. For example, when the driver's mental state is poor among the occupants in the vehicle, the recording setting unit 132 sets only the driver as the recording target person. At this time, as a method of recording only the voice of the driver, there is a method of recording only the voice acquired from the microphone of the driver among the microphones 213 arranged for each occupant. Alternatively, if the microphone 213 is not installed in each occupant, so-called beamforming technology that extracts the voice only in the direction of the driver may be used. Since the voice of occupants other than the driver is not included in the stored voice data in the vehicle, the privacy of other occupants is protected.

Even if the mental state of the occupant other than the driver deteriorates, the recording setting unit 132 may be set to record the voice of only the occupant other than the corresponding driver, as in the case of the driver described above. .. However, when the mental state of the occupant other than the driver deteriorates, the recording setting unit 132 may set the occupant and the driver other than the corresponding driver as the recording target person. This is because when dangerous driving occurs due to an occupant other than the driver, the occupant other than the driver may be involved in the driver, or may be involved in the steering, accelerator, brake, etc. of the driver's seat. That is, it may become clear from the driver's remarks that the driving is dangerous due to the steering being operated by an occupant other than the driver, the brake or the accelerator being stepped on, or the like. Is.

When the recording setting unit 132 determines that the mental state determination unit 131 includes an occupant having a bad mental state, all the occupants in the vehicle may be set as recording target persons. This is because there are scenes in which the occupants involved in dangerous driving and the occupants who were not involved can be clarified from each person's remarks by recording the voices of all the occupants.

The recording setting unit 132 may set the recording target person to be applied only to loop recording or event recording, or may be applied to both of them.

The display control unit 133 controls the display of video data on the display unit 216. The display control unit 133 outputs a video signal for outputting the video data to the display unit 216. More specifically, the display control unit 133 displays a video signal captured by the first camera 211 or the second camera 212, or a video signal displayed by playing back loop recorded video data or event recording data recorded in the recording unit 214. Is output.

When the mental state determination unit 131 determines that the mental state of the occupant is bad, the display control unit 133 may display character information or an icon indicating that the recording setting is changed on the display unit 216. The display control unit 133 may display a message for suppressing dangerous driving on the display unit 216.

The audio output control unit 134 outputs an audio signal for outputting the audio data corresponding to the video data displayed on the display unit 216 from the speaker 217. The voice output control unit 134 transmits voice data corresponding to the event recording data or the loop recording data recorded in the recording unit 214 from the speaker 217 based on the control signals of the selection operation and the playback operation output from the operation control unit 125. Output the audio signal to be output.

When the mental state determination unit 131 determines that the mental state of the occupant is bad, the voice output control unit 134 may output voice information or a notification sound indicating that the recording setting is changed from the speaker 217. The voice output control unit 134 may output voice information or a notification sound for suppressing dangerous driving from the speaker 217.
<Information processing in vehicle record control device>
Next, the flow of processing in the control device 100 will be described with reference to FIGS. 2 and 3. In the present embodiment, when it is determined that the mental state of the occupant of the vehicle is bad, the recording setting is changed. While the vehicle recording device 10 is running, the processing of the flowchart shown in FIG. 2 and the processing of the flowchart shown in FIG. 3 are executed.

First, FIG. 2 is a flowchart showing an example of a processing flow in the control device 100 according to the first embodiment. While the vehicle recording device 10 is activated, the mental state of the occupant is constantly determined by the processing of the flowchart shown in FIG.

The control device 100 determines whether or not the occupant's mental state is bad by the mental state determination unit 131 (step S101). More specifically, the occupant is based on the behavior of the vehicle recognized by at least one of the shooting data acquired by the shooting data acquisition unit 120 and the vehicle information acquired by the vehicle information acquisition unit 130 by the mental state determination unit 131. Judge the quality of the mental state of. When the mental state determination unit 131 determines that the mental state of the occupant is bad (Yes in step S101), the process proceeds to step S102. If the mental state determination unit 131 does not determine that the mental state of the occupant is bad (No in step S101), the process proceeds to step S104.

When it is determined that the mental state of the occupant is bad (Yes in step S101), the control device 100 sets the recording target person to save the in-vehicle voice by the recording setting unit 132 (step S102). For example, when a plurality of people are on board and the mental state determination unit 131 determines that the driver's mental state is bad, the recording setting unit 132 sets all the occupants including the driver as recording target persons. The control device 100 proceeds to step S103.

The control device 100 notifies the occupant that the recording setting has been made (step S103). More specifically, the display control unit 133 causes the display unit 216 to display character information or an icon indicating that the recording setting has been made. The voice output control unit 134 outputs voice information or a notification sound indicating that the recording setting has been made from the speaker 217. The control device 100 ends the process.

When it is not determined that the occupant's mental state is bad (No in step S101), the control device 100 cancels the recording setting for the in-vehicle voice data by the recording setting unit 132 (step S104). The control device 100 ends the process.

In this way, the recording is set according to the mental state of the occupant of the vehicle.

Next, FIG. 3 is a flowchart showing an example of the flow of the event detection process in the control device 100 according to the first embodiment.

The control device 100 starts loop recording and event detection (step S201). More specifically, the video data processing unit 122 starts loop recording. Further, the detection of acceleration by the acceleration sensor 218 is started. The event detection unit 127 starts event detection based on the detected acceleration. The control device 100 proceeds to step S202.

The control device 100 determines whether or not an event has been detected based on the detection result of the event detection unit 127 (step S202). If the detected acceleration is equal to or greater than the threshold value by the event detection unit 127, it is determined that the event has been detected (Yes in step S202), and the process proceeds to step S203. Alternatively, if the event detection unit 127 determines that the detected acceleration is not equal to or greater than the threshold value, it is determined that no event is detected (No in step S202), and the process proceeds to step S204.

When it is determined that an event has been detected (Yes in step S202), the recording control unit 123 prohibits the recording unit 214 from overwriting the captured video data as event recording data (step S203), and causes step S204 to save the captured video data. move on. When the process of step S102 is executed, the in-vehicle video data included in the event recording data is saved as video data including audio by the video data processing unit 122. When the process of step S104 is executed, the video data processing unit 122 does not include audio in the in-vehicle video data included in the event recording data. The event recording data is video data including the time before and after the event detection time.

The control device 100 determines whether or not to end loop recording and event detection (step S204). For example, it is determined that the loop recording and the event detection are terminated when the power or power of the vehicle is turned off or the operation unit 215 is operated. When it is determined that the loop recording and the event detection are finished (Yes in step S204), the control device 100 ends this process. If it is not determined that the loop recording and event detection are completed (No in step S204), the control device 100 re-executes the process of step S202.
<Effect>
As described above, in the present embodiment, when it is determined that the mental state of the occupant is bad, the recording setting of the voice data in the vehicle is set. In the present embodiment, when an event is detected in a state where it is determined that the mental state of the occupant is bad, the event record data including the voice data in the vehicle of the occupant is saved. In the present embodiment, it is possible to protect the privacy of the occupant who has a good mental condition by recording the voice data in the vehicle only for the occupant who has a bad mental condition. In the present embodiment, the storage of in-vehicle voice data is a deterrent to the occupant from performing dangerous driving. In this way, the present embodiment can protect the privacy of the occupant and deter dangerous driving.

In the present embodiment, when it is determined that the mental state of the occupant is bad, the occupant is notified by display or voice that the recording setting is set. According to this embodiment, dangerous driving can be suppressed.
[Second Embodiment]
The vehicle recording device 10A according to the present embodiment will be described with reference to FIGS. 4 to 5. FIG. 4 is a block diagram showing a configuration example of a vehicle recording device 10A having the control device 100A according to the second embodiment. FIG. 5 is a flowchart showing an example of the processing flow in the control device 100A according to the second embodiment. The basic configuration of the vehicle recording device 10A is the same as that of the vehicle recording device 10 of the first embodiment. In the following description, the same components as those of the vehicle recording device 10 are designated by the same reference numerals or the corresponding reference numerals, and detailed description thereof will be omitted. In the present embodiment, the point that the control device 100A has the occupant determination unit 135A and the processing in the control device 100A are different from those in the first embodiment.

The occupant determination unit 135A determines the position of the occupant of the vehicle. More specifically, the occupant determination unit 135A performs image processing on the in-vehicle video data acquired by the shooting data acquisition unit 120 to determine the seating position of the occupant. When the seating position of the occupant is determined by image processing the in-vehicle video data, it is possible to determine the seating position in a poor mental state in combination with the determination result of the mental state determination unit 131A.

The occupant determination unit 135A may determine the seating position of the occupant from the voice data in the vehicle acquired by the shooting data acquisition unit 120. When the voice included in the voice data is the voice of one person, it may be determined that only the driver is on board and the seating position is only the driver's seat. For example, the position of the driver's seat can be specified according to the information including the position of the driver's seat of the vehicle contained in the vehicle information or the information indicating whether the vehicle is a right-hand drive vehicle or a left-hand drive vehicle. be. When the voice included in the voice data is the voice of a plurality of people, all the seats may be determined as the seating positions.

The occupant determination unit 135A may determine the seating position from the sensor data detected by the seating sensor arranged in each seat.

When the mental state determination unit 131A determines that the mental state of the occupant is bad, the recording setting unit 132A sets a recording target person to save the in-vehicle voice according to the seating position of the occupant determined by the occupant determination unit 135A. More specifically, when the mental state determination unit 131A determines that the mental state of the occupant is bad, the recording setting unit 132A sets only the occupant with a bad mental state as the recording target person.

In the present embodiment, when the mental state of the occupant is poor, the driver may be included in the recording target in addition to the corresponding occupant. This is because when dangerous driving occurs due to an occupant other than the driver, the occupant other than the driver may be involved in the driver, or may be involved in the steering, accelerator, brake, etc. of the driver's seat. That is, it may become clear from the driver's remarks that the driving is dangerous due to the steering being operated by an occupant other than the driver, the brake or the accelerator being stepped on, or the like. Is.

In the present embodiment, in order to prevent the voices from overlapping and being difficult to hear when there are a plurality of recording target persons, a mode may be included in which the recording target person is limited to one person. For example, when the driver is included in the recording target person, the recording setting unit 132 may set the driver as the recording target person. The recording setting unit 132 may set the occupant closest to the driver's seat as the recording target person. The recording setting unit 132 sets the recording target person in priority based on the distance to the driver's seat (easiness of interference with the driver's seat) such as the passenger seat, the driver side of the rear seat, and the passenger seat side of the rear seat. You may decide. The recording setting unit 132 may record an occupant with the worst mental condition. The recording setting unit 132 may set the occupant who has the largest or longest number of times the mental state determination unit 131 determines that the person is mentally ill in an arbitrary period as the recording target person.

Next, the flow of processing in the control device 100A will be described with reference to FIG. The processing of step S112, step S114, and step S115 is the same as that of step S101, step S103, and step S104 of the flowchart shown in FIG.

The control device 100A specifies the position of the occupant by the occupant determination unit 135A (step S111). More specifically, the occupant determination unit 135A performs image processing on the in-vehicle video data acquired by the shooting data acquisition unit 120 to specify the seating position of the occupant. The control device 100A proceeds to step S112.

When it is determined that the driver's mental state is bad (Yes in step S112), the control device 100A sets the recording target person by the recording setting unit 132A (step S113). More specifically, the recording setting unit 132A sets the recording target person to record the voice according to the seating position of the occupant determined by the occupant determination unit 135A. For example, the recording setting unit 132A may record only a driver having a bad mental state. For example, the recording setting unit 132A may record all the occupants other than the driver and the driver. The control device 100A proceeds to step S114.

As described above, in the present embodiment, when it is determined that the mental state of the driver is bad, the recording target person is set according to the position of the occupant. In this embodiment, the recording target person can be set to include not only the driver but also the occupant. According to the present embodiment, even when a occupant other than the driver intervenes in the driving operation, the voice of the occupant can be stored as in-vehicle voice data.
[Third Embodiment]
The vehicle recording device 10A according to the present embodiment will be described with reference to FIG. FIG. 6 is a flowchart showing an example of the processing flow in the control device 100A according to the third embodiment. In the present embodiment, the processing in the occupant determination unit 135A of the control device 100A is different from that in the second embodiment.

The occupant determination unit 135A further determines whether or not the occupant of the vehicle is only the driver. More specifically, the occupant determination unit 135A performs image processing on the in-vehicle video data acquired by the shooting data acquisition unit 120 to determine whether or not there is a passenger other than the driver. When the occupant determination unit 135A does not recognize an occupant sitting in a seat other than the driver's seat in the in-vehicle video data, the occupant determination unit 135A determines that the occupant of the vehicle is only the driver.

The occupant determination unit 135A may determine whether or not there is a passenger other than the driver from the voice data in the vehicle acquired by the shooting data acquisition unit 120. By comparing the voice included in the voice data with the voice of the driver stored in the storage unit, it may be determined whether or not there is a passenger other than the driver.

The occupant determination unit 135A may determine whether or not there is a passenger other than the driver from the sensor data detected by the seating sensor arranged in each seat.

The recording control unit 123A records the shooting data when the occupant determination unit 135A determines that the occupant is only the driver and the mental state determination unit 131 determines that the driver's mental state is bad. Save to. If the driver's mental condition is poor, there is no passenger who can suppress dangerous driving, so there is a risk of driving dangerously immediately. Therefore, the shooting data is saved regardless of the presence or absence of event detection.

Next, the flow of processing in the control device 100A will be described with reference to FIG. The processing of step S121, step S128 to step S131 is the same as that of step S111, step S112 to step S115 of the flowchart shown in FIG.

The control device 100A determines whether or not the occupant is only the driver by the occupant determination unit 135A (step S122). When the occupant determination unit 135A determines that the occupant is only the driver (Yes in step S122), the process proceeds to step S123. If the occupant determination unit 135A does not determine that the occupant is only the driver (No in step S122), in other words, if it is determined that there is an occupant other than the driver, the process proceeds to step S128. When it is determined that there is an occupant other than the driver, the same process as in the second embodiment is executed.

When it is determined that the occupant is only the driver (Yes in step S122), the control device 100A determines whether or not the driver's mental state is bad (step S123). When the mental state determination unit 131A determines that the driver's mental state is bad (Yes in step S123), the process proceeds to step S124. When the mental state determination unit 131A does not determine that the driver's mental state is bad (No in step S123), the setting of the recording target person is canceled (step S125), that is, the setting is set so that recording is not performed, and the process is performed. finish.

When it is determined that the mental state of the driver is bad (Yes in step S123), the recording setting unit 132A sets the driver as the recording target (step S124). The control device 100A proceeds to step S126.

The control device 100A starts recording (step S126). More specifically, the recording control unit 123 saves the shooting data in the recording unit 214. The control device 100A proceeds to step S127.

The control device 100A notifies the driver that recording has started (step S127). More specifically, the display control unit 133 causes the display unit 216 to display character information or an icon indicating that recording has started. The voice output control unit 134 outputs voice information or a notification sound indicating that recording has started from the speaker 217. The control device 100A ends the process.

As described above, in the present embodiment, when it is determined that the mental state of the driver is bad, the shooting data is stored in the recording unit 214 regardless of the presence or absence of event detection. In this embodiment, the recording frequency of event recording can be increased. This embodiment can suppress dangerous driving.

The vehicle recording device 10 according to the present invention may be implemented in various different forms other than the above-described embodiment.

Each component of the illustrated vehicle recording device 10 is functionally conceptual and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of each device is not limited to the one shown in the figure, and all or part of the device is functionally or physically dispersed or integrated in an arbitrary unit according to the processing load and usage status of each device. You may.

The configuration of the vehicle recording device 10 is realized, for example, by a program loaded in a memory as software. In the above embodiment, it has been described as a functional block realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The above-mentioned components include those that can be easily assumed by those skilled in the art and those that are substantially the same. Further, the above configurations can be combined as appropriate. Further, various omissions, substitutions or changes of the configuration can be made without departing from the gist of the present invention.

In the third embodiment, when the recording control unit 123A determines that the occupant determination unit 135A determines that the occupant is only the driver, and the mental state determination unit 131A determines that the driver's mental state is bad. Only the in-vehicle video data taken by the second camera 212 may be saved, and the peripheral video data taken by the first camera 211 may be saved only at the time of event detection.

When the mental state determination unit 131 determines that the mental state of the occupant is bad, the setting of the recording target is performed by lighting an indicator provided around the display unit 216 or on the dashboard, instrument panel, center console, or the like. May be shown.

10 Vehicle recording device 100 Control device (Vehicle recording control device)
110 Bus 120 Shooting data acquisition unit 121 Buffer memory 122 Video data processing unit 123 Recording control unit 124 Playback control unit 125 Operation control unit 126 Acceleration information acquisition unit 127 Event detection unit 128 Position information acquisition unit 130 Vehicle information acquisition unit 131 Mental state determination Unit 132 Recording setting unit 133 Display control unit 134 Audio output control unit 211 First camera 212 Second camera (in-vehicle shooting unit)
213 Microphone 214 Recording unit 215 Operation unit 216 Display unit 217 Speaker 218 Accelerometer 219 GPS receiver unit 220 IF unit (CAN interface unit)

Claims (4)

A vehicle information acquisition unit that includes at least one of driving operation information indicating a driving operation for a vehicle and state information indicating the state of the vehicle and acquires vehicle information indicating the behavior of the vehicle.
A shooting data acquisition unit that acquires in-vehicle video data that is an image of the inside of the vehicle and in-vehicle audio data that is the sound of the inside of the vehicle as shooting data.
Based on the behavior of the vehicle recognized from at least one of the shooting data acquired by the shooting data acquisition unit and the vehicle information acquired by the vehicle information acquisition unit, the quality of the mental state of the occupant is determined. Mental state judgment department and
When the mental state determination unit determines that the mental state of the occupant is bad, the recording setting unit for changing the recording setting of the in-vehicle voice data and the recording setting unit
When the mental state determination unit determines that the mental state of the occupant is bad, the recording control unit that saves the shooting data based on the recording setting set by the recording setting unit, and the recording control unit.
A vehicle record control device. When the mental state determination unit determines that the mental state of the occupant is bad, the recording setting unit sets only the occupant with a bad mental state as a recording target.
The vehicle record control device according to claim 1. A vehicle information acquisition step for acquiring vehicle information indicating the behavior of the vehicle, including at least one of driving operation information indicating a driving operation for the vehicle and state information indicating the state of the vehicle.
A shooting data acquisition step for acquiring in-vehicle video data, which is an image of the inside of the vehicle, and in-vehicle audio data, which is the sound of the inside of the vehicle, as shooting data.
Based on the behavior of the vehicle recognized from at least one of the shooting data acquired by the shooting data acquisition step and the vehicle information acquired by the vehicle information acquisition step, the quality of the occupant's mental state is determined. The mental state judgment step to judge and
When it is determined by the mental state determination step that the mental state of the occupant is bad, the recording setting step for setting the recording setting of the in-vehicle voice data and the recording setting step.
When it is determined by the mental state determination step that the mental state of the occupant is bad, the recording control step for saving the shooting data based on the recording setting set by the recording setting step, and the recording control step.
A record control method performed by vehicle record control, including. A vehicle information acquisition step for acquiring vehicle information indicating the behavior of the vehicle, including at least one of driving operation information indicating a driving operation for the vehicle and state information indicating the state of the vehicle.
A shooting data acquisition step for acquiring in-vehicle video data, which is an image of the inside of the vehicle, and in-vehicle audio data, which is the sound of the inside of the vehicle, as shooting data.
Based on the behavior of the vehicle recognized from at least one of the shooting data acquired by the shooting data acquisition step and the vehicle information acquired by the vehicle information acquisition step, the quality of the occupant's mental state is determined. The mental state judgment step to judge and
When it is determined by the mental state determination step that the mental state of the occupant is bad, the recording setting step for setting the recording setting of the in-vehicle voice data and the recording setting step.
When it is determined by the mental state determination step that the mental state of the occupant is bad, the recording control step for saving the shooting data based on the recording setting set by the recording setting step, and the recording control step.
A program to execute.

Images