JP2019121314A - Determination device, information recording device, determination method, and program for determination - Google Patents

Abstract

To determine a state of an automatic operation control of a moving body without directly acquiring information indicating the state of the automatic operation control from the moving body.SOLUTION: A determination device detects a notification that indicates a state of an automatic operation control of a moving body and that can be sensed by a person present in a room of the moving body through detection means including a room camera of a camera 12, a microphone 16 and a vibration sensor 20, and determines the state of the automatic operation control via at least any one of the detection means and an image processing unit 22 on the basis of the detected notification.SELECTED DRAWING: Figure 2

Description

  The present application belongs to the technical field of an information recording apparatus for recording information indicating a situation in which a movable body capable of automatic driving is placed, for example, an image obtained by photographing the outside of the movable body on a recording medium.

  In recent years, a so-called drive recorder for capturing and recording, for example, the front of a moving vehicle is provided for the purpose of later verifying the situation etc. when an accident or the like occurs while moving the vehicle, for example. It is generalized.

  As a prior art regarding the drive recorder which takes the picture used for the said verification, there exists a prior art described, for example in following patent document 1. FIG. In this prior art, when the acceleration of the vehicle exceeds a threshold at which it is determined whether an accident has occurred, the same time as the imaging time with respect to the image for a predetermined time taken before the accident occurs. The altitude data and the gradient data acquired in the above are superimposed and recorded in the image memory.

  On the other hand, Patent Document 2 automatically selects an automatic driving function or an automatic driving level according to at least one of a surrounding condition of the vehicle and a driving condition of the vehicle from each automatic driving function or each automatic driving level of the vehicle. An automatic driving system is disclosed that automatically performs part or all of the driving operation of a vehicle and stores the time during automatic driving and the automatic driving function or the automatic driving level selected at that time.

JP, 2013-206417, A WO 2016/080070 pamphlet

  The video recorded by the technique disclosed in Patent Document 1 is useful for determining the fault liability sharing of the accident vehicle or its driver. On the other hand, in a vehicle capable of autonomous driving, an accident may occur during autonomous driving. In such a case, driving operation by automatic driving may cause an accident, which may affect the rate of driver's fault. However, with the technology disclosed in Patent Document 1, it is not possible to determine from the recorded video whether there is an accident cause in the driving operation by the driver or whether there is an accident cause in the driving operation by the automatic driving.

  On the other hand, according to the technology disclosed in Patent Document 2, when an accident or a failure occurs in a vehicle, it is necessary to grasp after that what automatic driving function or automatic driving level was selected at that time it can. However, this is possible because the vehicle itself is already equipped with a system capable of storing the selected automatic driving function or the automatic driving level. When the information recording apparatus is retrofitted to the vehicle, it may be difficult for the information recording apparatus to directly acquire the selected automatic driving function or automatic driving level from the vehicle.

    The present application has been made in view of the above points, and one example of the problem is to determine the state of the automatic operation control of the moving body without obtaining the information indicating the state of the automatic operation control directly from the moving body. An object of the present invention is to provide a determination device, a determination method, and a program for the determination device that can be performed.

  In order to solve the above-mentioned subject, the invention according to claim 1 is a notice of a state of automatic operation control of a mobile, and detects a notice perceivable by a person in the room of the mobile in the room. A determination apparatus comprising: detection means; and determination means for determining the state of the automatic driving control based on the detected notification.

  The invention according to claim 7 is a situation information acquiring means for acquiring situation information which is at least one of the current date and time, the position of the moving body, and an external image obtained by imaging the outside of the moving body from the moving body, A state of the automatic driving control determined by the determination device according to any one of items 1 to 6, wherein the state of the automatic driving control when the state information is acquired by the state information acquiring unit is An information recording apparatus comprising: an automatic driving information acquiring unit to acquire; and a recording control unit configured to record, on a recording medium, recording information including the acquired status information and the acquired status of the automatic driving control. It features.

  The invention according to claim 8 is, in the determination method executed by a computer, a notification of a state of automatic operation control of a moving object, wherein a notification perceptible to a person in the room of the moving object is detected in the room And determining the state of the automatic driving control based on the detected notification.

  The invention according to claim 9 is a determination program that causes a computer to function as the determination device.

It is a block diagram showing an example of outline composition of an information recording device concerning an embodiment. It is a block diagram showing an example of outline composition of a drive recorder concerning an example. It is a flowchart which shows an example of the recording process of the display image data in the drive recorder which concerns on an Example. It is a figure which illustrates the image recorded by the recording process by the drive recorder which concerns on an Example. It is a flow chart which shows an example of acquisition processing of a state of automatic operation control in a drive recorder concerning a modification.

  First, a mode for carrying out the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing an example of a schematic configuration of a determination apparatus according to the embodiment.

  As shown in FIG. 1, the determination apparatus S according to the embodiment includes a detection unit 1 and a determination unit 2.

  In this configuration, the detection means 1 is a notification of the state of automatic operation control of the moving body, and detects a notice that can be perceived by a person in the room of the moving body in the room. Examples of mobile objects include vehicles, motorcycles, bicycles, aircrafts, and ships.

  The determination unit 2 determines the state of the automatic driving control based on the notification detected by the detection unit 1.

  As described above, according to the operation of the determination apparatus S according to the embodiment, the notification that can be perceived by the person in the room of the mobile object is detected based on the detected notification about the state of the automatic driving control of the mobile object. State of automatic operation control. Therefore, even if it does not acquire the information which shows the state of automatic driving | operation control directly from a moving body, the state of automatic driving | running control of a moving body can be determined.

  Next, specific examples corresponding to the above-described embodiment will be described using FIGS. 2 to 4. The embodiment described below is an embodiment in the case where the embodiment is applied to a recording process of image data and the like in a drive recorder mounted on a vehicle. FIG. 2 is a block diagram showing an example of a schematic configuration of the drive recorder according to the embodiment. FIG. 3 is a flowchart showing an example of recording processing of display image data in the drive recorder according to the embodiment. FIG. 4 is a view exemplifying an image recorded by the recording process by the drive recorder according to the embodiment.

  The drive recorder according to the embodiment is installed in the interior of the vehicle so as to be able to capture the front view of the vehicle on which the drive recorder is mounted. In the following description, a vehicle in which the drive recorder according to the embodiment is used is simply referred to as "own vehicle". The host vehicle is a vehicle compatible with automatic driving. At least one automatic operation of acceleration, steering, and braking of the vehicle is performed by the automatic driving control system mounted on the vehicle. The host vehicle is compatible with a plurality of automatic driving levels. The plurality of automatic driving levels are defined, for example, as follows.

  Level 0: The driver performs all operations of acceleration, steering and braking.

  Level 1: In this state, the automatic driving control system performs any one operation of acceleration, steering and braking.

  Level 2: The automatic driving control system performs any two operations of acceleration, steering and braking.

  Level 3: The automatic operation control system performs all operations of acceleration, steering and braking, and the driver performs the operation when the automatic operation control system requests it.

  Level 4: The automatic driving control system performs all operations of acceleration, steering and braking, and the driver is not involved in driving at all.

  In addition, the definition of an automatic driving | operation level is not limited to the levels 0-4 mentioned above. Also, the number of automatic driving levels is not limited to five.

  As shown in FIG. 2, the drive recorder DR according to the embodiment includes an operation unit 11 including operation buttons and a touch panel, a display 18 for outputting various information, and a speaker 19 for emitting necessary operation guidance sound and the like. ing. In addition to these, the drive recorder DR according to the embodiment includes a CPU, a read only memory (ROM), a random access memory (RAM), and the like, and controls the entire drive recorder DR to exhibit its function. For example, a storage unit 10 formed of a solid state drive (SSD) or the like, a camera 12 formed of a complementary metal-oxide-semiconductor (CMOS) imaging device or the like, a map data recording unit 13 formed of an SSD or the like, and a GPS (global positioning sensor) A sensor 14, an acceleration sensor 15, a microphone 16, a vehicle interface 17, a vibration sensor 20, a memory SD such as a removable SD (Secure Digital) card, a camera 12, a map data recording unit 13, a GPS Sensor 14, acceleration sensor 15, microphone 16, vehicle interface 17, vibration sensor 20 and processing unit 9 An interface 21 for connection, and is made of. The processing unit 9 is configured of an image processing unit 22 and a recording unit 23. The image processing unit 22 and the recording unit 23 may be realized by a hardware logic circuit including the CPU and the like constituting the processing unit 9, and a program corresponding to a recording process of display image data according to an embodiment described later. May be realized as software by reading and executing the CPU.

  In the above configuration, the operation unit 11 is configured with an operation button, a touch panel, and the like in a part of the drive recorder DR, receives an operation by the user, and outputs an operation signal corresponding to the operation to the processing unit 9. In addition, the display 18 displays information to be presented to the user as a function of the drive recorder DR under the control of the processing unit 9 using, for example, an image or an icon. Further, under the control of the processing unit 9, the speaker 19 emits operation guidance sound and the like necessary for performing the function of the drive recorder DR.

  Next, the camera 12 captures an image of the outside of the host vehicle. For example, the camera 12 uses the surroundings including the moving direction ahead of the host vehicle as its imaging range. Then, the camera 12 generates imaging data corresponding to an external image obtained by imaging the inside of the imaging range, and outputs the imaging data to the image processing unit 22 of the processing unit 9 via the interface 21. In the embodiment, the external image is a moving image, but the external image may be a still image. On the other hand, the map data recording unit 13 records general road data and intersection data as map data. The map data recording unit 13 outputs map data such as road data to the image processing unit 22 via the interface 21 in response to a request from the image processing unit 22.

  On the other hand, the GPS sensor 14 outputs position data indicating the current position of the drive recorder DR to the image processing unit 22 via the interface 21 based on the result of receiving navigation radio waves from a navigation satellite not shown. Here, the current position of the drive recorder DR is, in other words, the current position of the vehicle. Next, the acceleration sensor 15 detects the acceleration applied to the vehicle, for example, in the traveling direction, the left and right direction, and the vertical direction of the vehicle, and detects the acceleration data as the detection result through the interface 21. Output to 22 The image processing unit 22 may generate a map image around the vehicle or a navigation image for guiding the vehicle to a destination based on the current position of the vehicle, road data, etc. .

  On the other hand, the microphone 16 collects, for example, sound or voice of the own vehicle, for example, in the car, and outputs sound data as a result of the sound collection to the image processing unit 22 through the interface 21. At this time, the microphone 16 may be able to collect sounds or sounds outside the vehicle. The vibration sensor 20 detects the vibration of a seat such as the driver's seat of the host vehicle. Then, the vehicle interface 17 acquires automatic driving data indicating the state of automatic driving control via an in-vehicle network such as CAN (Controller Area Network), for example, and outputs the automatic driving data to the image processing unit 22 via the interface 21. The state of the automatic driving control is, for example, which of the acceleration, steering, and braking targets is under automatic driving control, or at which self-driven driving level automatic driving control is performed.

  The image processing unit 22 performs automatic driving control indicating at least a part of the external image and the state of automatic driving control at the photographing time of the external image based on the imaging data and the automatic driving data and the like acquired through the interface 21. Display image data for simultaneously displaying the control state image is acquired. The recording unit 23 records the display image data acquired by the image processing unit 22 in the memory SD. In this case, the recording unit 23 records the display image data in the memory SD in a general-purpose format. The general-purpose format is, for example, an MP4 (Moving Picture 4) format, an AVI (Audio video Interleave) format, an MOV format, or the like. The image processing unit 22 and the recording unit 23 record audio data in the memory SD in synchronization with the corresponding display image data in parallel with the recording of the display image data in the memory SD. The display image data and the audio data and the like recorded in the memory SD are displayed or reproduced by an external reproduction device (not shown) or the like, for example, at the time of post-testing of an accident or the like.

  Next, a process of recording display image data according to the embodiment, which is performed mainly by the image processing unit 22 in the drive recorder DR according to the embodiment including the above-described configuration, will be described with reference to FIG.

  The recording process of display image data is started, for example, at the timing when the power of the drive recorder DR is turned on.

  The image processing unit 22 first acquires imaging data via the interface 21. That is, the interface 21 acquires imaging data of an external image captured by the camera 12 and outputs the imaging data to the image processing unit 22 (step S10).

  Next, the image processing unit 22 acquires automatic driving data and the like through the interface 21. The autonomous driving data may be data indicating whether each operation of acceleration, steering and braking is automatic or manual, or may be data indicating an autonomous driving level. That is, the interface 21 acquires information on automatic driving data, current date and time, traveling speed, and current position of the vehicle via the vehicle interface 17 and outputs the information to the image processing unit 22 (step S20).

  Next, the image processing unit 22 performs processing such as generation of display image data to be recorded in the memory SD, based on the information acquired in steps S10 and S20. For example, the image processing unit 22 generates an automatic driving control state image indicating the current state of the automatic driving control based on the automatic driving data and the like. In addition to the state of automatic driving control, the image processing unit 22 indicates at least one of information indicating the situation in which the vehicle is placed, such as the current date and time, the traveling speed, and the current position of the vehicle. The automatic driving control image may be generated. The image processing unit 22 may generate display image data by embedding an automatic driving control state image indicating a state of automatic driving control at the shooting time of the external image in the external image by image combination, for example. Alternatively, the image processing unit 22 may generate display image data including, as separate layers, an external image and an automatic driving control state image indicating a state of automatic driving control at the shooting time of the external image. The image processing unit 22 temporarily stores the generated display image data in the RAM or the like of the processing unit 9 (step S30).

  Then, when the acceleration sensor 15 detects an acceleration larger than a predetermined value, the recording unit 23 acquires a display image which is acquired for a predetermined time before and after the timing at which such an acceleration is detected and temporarily stored in the RAM or the like. The data is recorded on the recording medium SD as recording information. When an acceleration larger than a predetermined value is detected, there is a possibility that the vehicle is suddenly braked or an impact is applied to the vehicle. If the acceleration larger than the predetermined value is not detected, the recording on the recording medium SD is not performed (step S40). In addition to the case where the acceleration larger than the predetermined value is detected, the recording unit 23 may record the display image data in the memory SD when the automatic driving level changes.

  Thereafter, the image processing unit 22 determines whether or not the recording process of display image data according to the embodiment is ended by turning off the power of the drive recorder DR or the like (step S50). When the recording process is ended in the determination of step S50 (step S50: YES), the image processing unit 22 ends the process as it is. On the other hand, if it is determined in step S50 that the recording process is to be continued (step S50: NO), the image processing unit 22 returns to step S10 and repeats each process described above.

  Next, an image displayed using display image data recorded in the memory SD as a result of recording processing of display image data according to the embodiment described with reference to FIG. 3 will be described with reference to FIG. As shown in FIG. 4, the image 100 displayed by the reproduction apparatus described above using display image data includes an external image 110 and an automatic driving control state image 120. For example, the image 100 is generated by embedding the automatic driving control state image 120 in the external image 110. In FIG. 4, the automatic driving control state image 120 shows whether each operation of acceleration, steering and braking is automatic or manual. However, the autonomous driving control state image 120 may indicate an autonomous driving level. Alternatively, the automatic driving control state image 120 may indicate a control target of which operation is automatic among acceleration, steering, and braking. The automatic driving control state image 120 may indicate at least one of the current date and time, the traveling speed, and the latitude and longitude of the current position, in addition to indicating the state of the automatic driving control. When displaying image 100 using display image data including an external image and an automatic driving control state image as separate layers, display and non-operation of automatic driving control state image 120 is performed based on the user's operation on the playback device. Switching of display may be possible. When the automatic driving control state image 120 is not displayed, the entire external image 110 is displayed. Thereby, it is possible to confirm a portion hidden by the automatic driving control state image 120 in the external image 110. Alternatively, the external image 110 and the automatic driving control state image 120 may be displayed separately. For example, when an area for displaying the external image 110 and an area for displaying the automatic driving control state image 120 are set on the screen of the playback device, and the external image 110 is displayed, the user can When the display operation is performed, the automatic driving control image 120 indicating the state of the automatic driving control at the photographing time of the external image 110 is displayed in an area different from the area of the external image 110.

  As described above, according to the operation according to the embodiment, at least a part of the external image based on the external image obtained by imaging the outside of the own vehicle from the own vehicle and the state of the automatic driving control of the own vehicle Display image data for simultaneously displaying the automatic driving control image showing the state of the automatic driving control at the imaging time of the external image is stored in the memory SD. In the above, it is possible to record an image etc. so that the situation etc. can be confirmed together with the state of the automatic driving control of the own vehicle.

  In addition, when acquiring display image data corresponding to an image generated by embedding an automatic driving state image in an external image, it is possible to easily associate the state of automatic driving control with the external condition etc. The display image data to be displayed on can be recorded in the memory SD.

  In addition, when the automatic driving state image indicates at least a target under automatic driving control among the acceleration, steering and braking of the own vehicle, it is easily confirmed which operation of the acceleration, steering and braking is automatic. can do.

  In addition, when the automatic driving state image indicates the automatic driving level, it can be easily confirmed at what automatic driving level the automatic driving control has been performed.

  Further, when recording of display image data in the memory SD is performed using a general-purpose format, versatility in reproduction or display of the display image data itself can be enhanced.

  The recording medium on which the display data is recorded is not limited to the memory SD provided in the drive recorder DR or directly accessible by the drive recorder DR. For example, the recording medium may be a storage unit such as a hard disk provided in a server device (not shown) outside the host vehicle. For example, the drive recorder DR has a communication unit for performing wireless communication with the server device. Drive recorder DR may be made to memorize in a storage part of a server apparatus by transmitting display image data to a server apparatus through a communication part triggered by having detected sudden braking of a self-vehicle, or an impact. In such a mode, for example, by transmitting the display image data to the server device of the automobile insurance company, the automobile insurance company can more quickly determine the negligence rate when the accident occurs.

[Modification]
In the above embodiment, the automatic driving data indicating the state of the automatic driving control of the own vehicle is acquired through the vehicle interface 17. However, information indicating the state of automatic driving control may be acquired by the methods described below. According to the modification described below, for example, information indicating the state of automatic driving control of the own vehicle can not be obtained directly from the own vehicle via the vehicle interface 17 by retrofitting the drive recorder DR to the own vehicle or the like. Even in this case, information indicating the state of automatic operation control can be acquired.

[Modification 1]
The interface 21 may acquire information indicating the state of automatic driving control based on the relationship between the driving operation performed by the driver of the host vehicle and the behavior of the host vehicle at the time of the driving operation.

  For example, the camera 12 includes an external camera that captures the outside of the host vehicle, and one or more indoor cameras that capture the interior of the host vehicle. The indoor camera is installed indoors so as to provide an imaging range for steering and its surroundings, and an accelerator pedal and a brake pedal and their surroundings. The image processing unit 22 analyzes the image captured by the indoor camera, and recognizes the driver's hands and feet from the image using pattern recognition. Then, the image processing unit 22 specifies the driving operation by the driver based on the movement of the recognized limb.

(1) Acceleration The image processing unit 22 compares, for example, the operation of depressing the accelerator pedal by the driver with the change in the longitudinal acceleration of the host vehicle detected by the acceleration sensor 15, and the operation of stepping on the accelerator pedal and the host vehicle It is determined whether or not there is a relationship between the change of the acceleration in the front-rear direction and the change of the acceleration in the front-rear direction. For example, when a positive acceleration occurs in the traveling direction of the host vehicle when the accelerator is depressed, it is determined that there is a relationship between the two. On the other hand, when no positive acceleration occurs in the traveling direction of the host vehicle when the accelerator is depressed, it is determined that there is no relationship between the two. Alternatively, if a positive acceleration occurs in the traveling direction of the host vehicle despite the fact that the accelerator is not depressed, it is determined that there is no relationship between the two. The image processing unit 22 determines that the control of acceleration is manually performed when the relationship is recognized, and determines that the control of acceleration is automatically performed when the relationship is not recognized. .

  The image processing unit 22 detects the acceleration of the vehicle based on the change in the moving speed of an external feature (for example, a building, a telephone pole, a road surface, a traffic sign, a plant, etc.) captured by an external camera. You may calculate Alternatively, instead of the acceleration, the image processing unit 22 may compare the change in the speed of the host vehicle with the operation of depressing the accelerator pedal.

(2) Steering If, for example, the image processing unit 22 compares the steering operation by the driver with the change in the traveling direction of the host vehicle, is there any relation between the operation of the steering and the change in the traveling direction of the host vehicle? It is determined whether or not. For example, when the traveling direction of the host vehicle changes to the right when the steering is turned to the right by the driver, it is determined that there is a relationship between the two. Further, when the traveling direction of the vehicle changes to the left when the steering is turned left by the driver, it is determined that there is a relation between the two. On the other hand, when the traveling direction of the host vehicle changes even though the driver does not operate the steering, it is determined that there is no relationship between the two. The image processing unit 22 determines that steering control is manually performed when the relationship is recognized, and determines that steering control is automatically performed when the relationship is not recognized. .

  The image processing unit 22 may specify a change in the traveling direction of the vehicle based on, for example, the acceleration in the left-right direction of the host vehicle detected by the acceleration sensor 15. For example, the image processing unit 22 determines that the traveling direction of the host vehicle has changed to the right when the acceleration in the right direction is generated, and the host vehicle proceeds in the left direction when the acceleration in the left direction is generated. Determine that it has changed. Alternatively, the image processing unit 22 may specify a change in the traveling direction of the vehicle based on the movement direction of the external feature captured by the external camera in the external image of the external feature. Alternatively, the drive recorder DR may be provided with an angular velocity sensor for detecting the rate of change of the traveling direction of the host vehicle. The change in the traveling direction of the host vehicle may be specified based on the angular velocity detected by the angular velocity sensor.

(3) Braking The image processing unit 22 compares, for example, the operation of depressing the brake pedal by the driver with the change in the longitudinal acceleration of the host vehicle detected by the acceleration sensor 15, and the operation of stepping on the brake pedal and the host vehicle It is determined whether or not there is a relationship between the change of the acceleration in the front-rear direction and the change of the acceleration in the front-rear direction. For example, when a negative acceleration occurs in the traveling direction of the host vehicle when the brake is depressed, it is determined that there is a relationship between the two. On the other hand, if no negative acceleration occurs in the traveling direction of the host vehicle when the accelerator is depressed, it is determined that there is no relationship between the two. Alternatively, in the case where a negative acceleration occurs in the traveling direction of the host vehicle despite the fact that the brake is not depressed, it is determined that there is no relationship between the two. The image processing unit 22 determines that the control of braking is manually performed when the relationship is recognized, and determines that the control of braking is automatically performed when the relationship is not recognized. .

  The image processing unit 22 may calculate the acceleration of the host vehicle from the change in the moving speed in the external image of the external feature captured by the external camera. Alternatively, the image processing unit 22 may compare the change in the speed of the host vehicle with the operation of depressing the brake pedal, instead of the acceleration.

  In the present embodiment, the driver's driving operation is specified by analyzing the image captured by the indoor camera. However, instead of the indoor camera, sensors may be installed on the steering and operation pedals. Then, the image processing unit 22 may specify the driving operation by the driver based on the output from the sensor. For example, an angular velocity sensor and a contact sensor are installed in the steering. When the contact sensor detects that the driver is in contact with the steering, and the angular velocity sensor detects that an angular velocity is generated in the steering, the image processing unit 22 causes the driver to operate the steering. It is determined that there is. At this time, the image processing unit 22 determines the rotation direction of the steering based on the positive or negative of the angular velocity. In addition, a load sensor is installed on the operation pedal. The image processing unit 22 determines that the driver depresses the operation pedal when the load detected by the load sensor increases.

[Modification 2]
At least one of the indoor camera of the camera 12, the microphone 16, and the vibration sensor 20 is a notification of a state of automatic driving control of the own vehicle, and a notice perceptible to a person in the room of the own vehicle can be It may be detected. In the room of the host vehicle, the state of automatic driving control is notified by the automatic driving control system. This notification is perceptible to a person such as a driver in the room of the host vehicle (recognizable through stimulation to a sense). Examples of the notification method include a display on an instrument panel provided in the room of the host vehicle, an output of notification sound from a speaker provided in the host vehicle, and vibration of a seat such as a steering wheel or a driver's seat.

  The image processing unit 22 may determine the state of the automatic driving control based on the notification detected by the indoor camera of the camera 12, the microphone 16, or the vibration sensor 20.

  In the present modification, each of the indoor camera of the camera 12, the microphone 16, and the vibration sensor 20 corresponds to an example of the detection unit 1 according to the embodiment, and the image processing unit 22 corresponds to an example of the determination unit 2 according to the embodiment. . Then, an example of the determination device S according to the embodiment is configured by the image processing unit 22 and at least one of the indoor camera of the camera 12, the microphone 16, and the vibration sensor 20.

  Here, the storage unit 10 may store, as an example of the type of mobile object, a notification mode corresponding to the vehicle type of the host vehicle for each state of the automatic driving control. Then, the image processing unit 22 may determine the state of the automatic driving control based on the comparison between the detected notification mode and the notification mode stored in the storage unit 10.

(1) Display of Instrument Panel The instrument panel is provided with a display for displaying, for example, the state of automatic operation control. An automatic driving level may be displayed on this display. Alternatively, information may be displayed indicating whether each operation of acceleration, steering and braking is automatic or manual. Alternatively, among acceleration, steering, and braking, a control target whose operation is automatic may be displayed.

  The indoor camera is installed indoors to set the display as an imaging range. The indoor camera detects the display of the state of automatic driving control on the instrument panel as the above notification. The image processing unit 22 acquires information indicating the state of the automatic driving control based on the imaging result by the indoor camera. Specifically, the image processing unit 22 analyzes the image captured by the indoor camera to acquire information indicating the state of the automatic driving control of the host vehicle displayed on the display. Specifically, the image processing unit 22 recognizes the state of the automatic driving control of the own vehicle displayed on the display from the image. For example, in the storage unit 10, a feature amount of an image showing a display of an instrument panel corresponding to a vehicle type of the host vehicle may be stored in advance as a notification mode for each state of automatic driving control. For example, the image processing unit 22 compares the feature amount extracted from the captured image of the display of the instrument panel with the feature amount stored in the storage unit 10, so that the captured image is automatically detected. It may be determined whether it corresponds to the operation control state.

  Although the display of the automatic control state on the instrument panel is always performed, this display may be performed discontinuously. For example, the display of the automatic control state is performed at least at the start of driving of the host vehicle and when the state of the automatic driving control changes. When the automatic control state is not displayed, it is necessary to determine the state of automatic operation control. For example, when the display of the automatic control state is not detected by the indoor camera, the image processing unit 22 may determine that the state of the automatic driving control determined last time by the image processing unit 22 continues.

(2) Notification Sound The automatic driving control system of the own vehicle causes the speaker provided on the instrument panel or the like of the own vehicle to intermittently output the notification sound indicating the automatic driving control state. For example, the notification sound is output at least at the start of driving of the host vehicle and at the time of change of the state of automatic driving control. The notification sound may be, for example, a voice indicating an automatic driving level such as "the automatic driving level 2" or the like, or may be a buzzer sound corresponding to the automatic driving level such as "piping".

  The microphone 16 detects an alarm sound indicating the state of the automatic driving control as the above-mentioned alarm by collecting the room sound of the host vehicle. The image processing unit 22 acquires information indicating the state of automatic driving control based on the sound collection result of the notification sound by the microphone 16. Specifically, the image processing unit 22 determines the state of the automatic driving control based on the room sound collected by the microphone 16. For example, in the storage unit 10, a feature amount or the like of a notification sound corresponding to the vehicle type of the host vehicle may be stored in advance as a notification mode for each state of automatic driving control. The image processing unit 22 compares, for example, the feature amount extracted from the collected alarm sound with the feature amount stored in the storage unit 10, so that any automatic driving control state of the collected alarm sound is detected. It may be determined whether it corresponds to

  Since the notification sound is output discontinuously, it is necessary to determine the state of the automatic driving control even when the notification sound is not output. For example, when the notification sound is not detected by the microphone 16, the image processing unit 22 may determine that the state of the automatic driving control determined last time by the image processing unit 22 continues.

(3) Steering or vibration of the seat A vibrator is provided to the seat such as the steering or driver's seat. The autonomous driving control system of the host vehicle causes the steering or the seat to intermittently generate vibrations indicating the state of the autonomous driving control by driving the vibrator. For example, the steering or the seat vibrates at least at the start of driving of the host vehicle and when the state of automatic driving control changes. The steering or seat vibrates in a manner corresponding to the automatic driving control state. For example, the length of time for which the vibration continues, the strength of the vibration, the pattern of the vibration, etc. correspond to the automatic driving control state.

  The vibration sensor 20 detects a vibration indicating a state of steering or automatic operation control of a seat as the above notification. The image processing unit 22 acquires information indicating the state of the automatic driving control based on the detection result of the vibration by the vibration sensor 20. Specifically, the image processing unit 22 determines the state of the automatic driving control based on the vibration detected by the vibration sensor 20. For example, in the storage unit 10, information indicating the mode of vibration of the steering or the seat corresponding to the vehicle type of the host vehicle may be stored in advance as the notification mode for each state of automatic driving control. The image processing unit 22 compares the mode of the detected vibration with the mode of the vibration indicated by the information stored in the storage unit 10, for example, to which automatic driving control state the detected vibration corresponds May be determined.

  Since notification by vibration is performed discontinuously, it is necessary to determine the state of automatic driving control even when the steering or the seat is not vibrating. For example, when the vibration is not detected by the vibration sensor 20, the image processing unit 22 may determine that the state of the automatic driving control determined last time by the image processing unit 22 continues.

  Next, an acquisition process of information indicating the state of the automatic driving control of the own vehicle, which is executed mainly by the image processing unit 22 in the drive recorder DR according to the present modification, will be described with reference to FIG. Acquisition processing of information which shows a state of automatic operation control is performed at Step S20 of recording processing shown in FIG.

  As shown in FIG. 5, the image processing unit 22 determines whether the state of the automatic driving control has been detected (step S110). At this time, the indoor camera of the camera 12, the microphone 16 or the vibration sensor 20 functions as the detection means 1 of the determination device S. For example, the image processing unit 22 may use the feature amount of the image of the instrument panel captured by the indoor camera and the feature amount of the image of the state of automatic driving control by the instrument panel stored in the storage unit 10. If it is determined that the state of automatic operation control is displayed on the instrument panel based on the comparison with the above, it is determined that the state of automatic operation control is detected. Alternatively, the image processing unit 22 may notify the user of the room sound collected based on the comparison between the feature amount of the room sound collected by the microphone 16 and the feature amount of the notification sound stored in the storage unit 10. If it is determined that the state of the automatic driving control is detected. Alternatively, when the vibration of the steering or the seat is detected by the vibration sensor 20, the image processing unit 22 determines that the state of the automatic driving control is detected. If the state of the automatic driving control is detected, the image processing unit 22 advances the process to step S120. In addition, when the display of the state of the automatic driving | operation control by an instrument panel is always performed, the image processing part 22 should just perform step S120, without performing determination of step S110. On the other hand, if the state of the automatic driving control is not detected, the image processing unit 22 proceeds with the process to step S130.

  In step S120, the image processing unit 22 determines the state of the automatic driving control based on the detected notification. That is, the image processing unit 22 functions as the determination unit 2 of the determination device S. For example, in the case of the display by the instrument panel, the image processing unit 22 determines the state of the automatic driving control based on the comparison of the feature amounts of the image executed in step S110. In the case of the notification sound, the image processing unit 22 determines the state of the automatic driving control based on the comparison of the feature amounts of the sound performed in step S110. In the case of vibration, the image processing unit 22 determines the state of automatic driving control based on the comparison between the mode of vibration detected by the vibration sensor and the information indicating the mode of vibration stored in the storage unit 10. The image processing unit 22 acquires information indicating the determined state of automatic driving control as information for generating display image data to be recorded in the memory SD. Further, the image processing unit 22 temporarily stores information indicating the determined state of automatic driving control in the RAM or the like.

  In step S130, the image processing unit 22 determines that the state of the automatic driving control determined previously is continuing. Then, the image processing unit 22 acquires information indicating the state of automatic driving control temporarily stored in the RAM as information for generating display image data.

  When step S120 or S130 is executed, the image processing unit 22 ends the process of acquiring information indicating the state of automatic driving control, and advances the process to step S30 shown in FIG. In steps S30 and S40 shown in FIG. 3, the display image data is generated and recorded, whereby the state of the automatic driving control determined by the image processing unit 22, the current date and time, the position of the own vehicle, and the external image data Is recorded as recording information. In the present modification, the interface 21 may obtain status information on at least one of the current date, the position of the vehicle, and the external image data. Then, the interface 21 acquires the state of the automatic driving control determined by the image processing unit 22 when the state information is acquired, and the recording unit 23 acquires the acquired state information and the acquired state of the automatic driving control. And recording information including the information may be recorded in the memory SD.

  As described above, according to the operation according to the second modification, the notification of the state of the automatic driving control of the own vehicle is detected and detected in the room that can be perceived by the person in the room of the own vehicle. Since the state of the automatic driving control is determined based on the notification, it is possible to determine the state of the automatic driving control without directly acquiring information indicating the automatic driving state from the host vehicle.

  Moreover, when the display of the state of the automatic driving | operation control in an instrument panel is detected as said alerting | reporting, the state of automatic driving | operation control can be determined based on the display in an instrument panel.

  Moreover, when the alerting | reporting sound which shows the state of automatic driving | operation control is detected as said alerting | reporting, the state of automatic driving | operation control can be determined based on an alerting | reporting sound.

  Moreover, when the vibration which shows the state of automatic driving | operation control of the operating handle or seat of the own vehicle is detected as said alerting | reporting, the state of automatic driving control can be determined based on a vibration.

  Moreover, when the notification of the state of the automatic driving control is not detected and the notification of the state of the automatic driving control determined last time is determined to be continuing, the notification is not detected. Even when the operation is continuously performed, the state of the automatic driving control can be appropriately determined.

  When the state of the automatic driving control is determined based on the comparison between the detected state of the notification and the mode of the notification stored in the storage unit 10, the state of the automatic driving control may be easily determined. it can.

  In addition, it is a state of automatic driving control determined by acquiring status information that is at least one of the current date and time, the position of the own vehicle, and an external image, and the state of automatic driving control when the status information is acquired When acquiring and recording information including status information and the state of automatic driving control in the memory SD, the situation in which the host vehicle is placed may be checked after the state of automatic driving control of the host vehicle. it can.

1 detection means 2 determination means 9 processing unit 10 storage unit 12 camera 14 GPS sensor 15 acceleration sensor 16 microphone 17 vehicle interface 20 vibration sensor 21 interface 22 image processing unit 23 recording unit SD recording medium, memory S determination device DR drive recorder

Claims (9)

Detection means for detecting a notification that can be perceived by a person who is in the room of the mobile object, the notification being a notification of the state of automatic driving control of the mobile object, and
A determination unit that determines a state of the automatic driving control based on the detected notification;
A determination apparatus comprising:   The determination apparatus according to claim 1, wherein the detection unit detects a display of a state of automatic operation control on an instrument panel as the notification.   The determination apparatus according to claim 1, wherein the detection unit detects a sound indicating a state of the automatic driving control as the notification.   The determination apparatus according to claim 1, wherein the detection unit detects, as the notification, a vibration indicating a state of the automatic driving control of an operation handle or a seat of the moving body. The information processing apparatus further comprises storage means for storing an aspect of the notification corresponding to the type of the moving body for each state of the automatic driving control,
The determination means determines the state of the automatic driving control based on a comparison between the mode of the notification detected by the detection means and the mode of the notification stored in the storage means. The determination apparatus according to any one of claims 1 to 4. The detection means detects the notification that is performed discontinuously;
The determination means is characterized in that when the notification is not detected by the detection means, it is determined that the state of the automatic driving control determined last time by the determination means is continuing. The determination apparatus according to any one of the above. Situation information acquisition means for acquiring situation information which is at least one of the current date, the position of the moving body, and an external image obtained by imaging the outside of the moving body from the moving body;
It is a state of the said automatic driving | operation control determined by the determination apparatus in any one of Claim 1 thru | or 6, Comprising: The state of the said automatic driving control when the said status information is acquired by the said status information acquisition means Automatic driving information acquisition means for acquiring
Recording control means for recording recording information including the acquired status information and the acquired state of automatic driving control on a recording medium;
An information recording apparatus comprising: In the determination method executed by a computer,
A notification of a state of automatic driving control of the mobile body, wherein a detection step of detecting a notification perceivable by a person in the room of the mobile body in the room;
A determination step of determining a state of the automatic driving control based on the detected notification;
A determination method comprising:   A program for determination that causes a computer to function as the determination device according to any one of claims 1 to 6.

Images