JP2023110267A - On-vehicle device and control method of on-vehicle device - Google Patents

Abstract

To provide an on-vehicle device that can sense wrong operation by a driver to an accelerator with good accuracy, and a control method of an on-vehicle device.SOLUTION: An on-vehicle device according to one aspect of an embodiment comprises a control part. The control part detects motion of a foot part of a driver on the basis of an image of a vehicle interior having a condition of the vehicle interior of a vehicle photographed. The control part determines whether or not the vehicle is in a vehicle start-prohibited situation where the vehicle is prohibited from starting, on the basis of situation information showing a situation of the vehicle. The control part senses that motion of the foot part is a wrong operation to an accelerator when the detected motion of the foot part is an accelerator-motion operating the accelerator and when determining that the vehicle is in the vehicle start-prohibited situation.SELECTED DRAWING: Figure 1A

Description

The present invention relates to an in-vehicle device and a control method for the in-vehicle device.

Conventionally, there has been proposed a technique for estimating the toe positions of a person sitting on a seat inside a vehicle (see, for example, Patent Document 1). In the prior art, the positions of the toes of the feet are estimated from an image captured by an in-vehicle camera.

JP 2021-068059 A

By the way, in recent years, it has been desired to detect and notify an erroneous operation (for example, stepping on an accelerator) by a driver. However, the conventional technology has room for improvement in terms of accurately detecting an erroneous operation of the accelerator by the driver.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an in-vehicle device and a control method for the in-vehicle device that can accurately detect an erroneous operation of the accelerator by the driver.

In order to solve the above problems and achieve the object, the present invention has a controller in an in-vehicle device. The control unit detects the movement of the driver's feet based on an in-vehicle image of the interior of the vehicle. The control unit determines whether or not a start-prohibited state prohibits the vehicle from starting, based on situation information indicating the situation of the vehicle. When it is determined that the detected motion of the foot is an accelerator motion and that the vehicle is in the start prohibited state, the controller determines that the motion of the foot is an erroneous operation of the accelerator. detect something.

According to the present invention, it is possible to accurately detect an erroneous operation of the accelerator by the driver.

FIG. 1A is a diagram showing an overview of a control method according to an embodiment. FIG. 1B is a diagram illustrating an outline of a control method according to the embodiment; FIG. 1C is a diagram illustrating an outline of a control method according to the embodiment; FIG. 2 is a block diagram showing a configuration example of a vehicle system including an in-vehicle device. FIG. 3 is a diagram showing an example of accelerator operation information. FIG. 4 is a diagram showing an example of notification of an erroneous operation of the accelerator. FIG. 5 is a flowchart showing a processing procedure executed by the in-vehicle device.

Hereinafter, embodiments of an in-vehicle device and an in-vehicle device control method disclosed in the present application will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by embodiment shown below.

<Outline of control method for in-vehicle device>
First, an outline of a control method for an in-vehicle device according to an embodiment will be described below with reference to FIGS. 1A to 1C. 1A to 1C are diagrams showing an overview of the control method according to the embodiment.

As shown in FIG. 1A, the in-vehicle device 10 is mounted in a vehicle C such as an automobile. A drive recorder, for example, can be used as the in-vehicle device 10 .

The in-vehicle device 10 includes an in-vehicle camera 11 . The in-vehicle camera 11 captures an image of the interior C1 of the vehicle C. As shown in FIG. Note that the in-vehicle device 10 may include an exterior camera 12 (see FIG. 2) that captures an image of the exterior of the vehicle C. As shown in FIG.

The in-vehicle device 10 detects the movement of the foot Da of the driver D based on the in-vehicle image captured by the in-vehicle camera 11 (step S1). For example, the in-vehicle device 10 analyzes the in-vehicle image to detect the foot Da (specifically, thigh Da1) of the driver D, and detects the motion of the detected foot Da (thigh Da1).

Here, the in-vehicle image will be described with reference to FIG. 1B. FIG. 1B is an in-vehicle image A captured by the in-vehicle camera 11, and more specifically, the in-vehicle image A showing the state of the interior C1 of the vehicle C. As shown in FIG.

As shown in FIG. 1B, the in-vehicle image A includes the foot Da of the driver D of the vehicle, and more specifically, the thigh Da1 of the driver D. As shown in FIG. In the in-vehicle device 10 according to the present embodiment, by detecting the motion of the foot portion Da, the accelerator motion of the driver D operating the accelerator 100 (see FIG. 1A) is detected.

Specifically, the position of the foot Da of the driver D differs between when the brake is operated and when the accelerator 100 is operated. That is, when the foot Da (for example, the thigh Da1) is operating (depressing) the brake to the state of operating (depressing) the accelerator 100, the position of the foot Da is shifted from the brake side. Move to the accelerator 100 side.

Here, it is assumed that the brake is arranged on the left side as seen from the driver D (that is, the right side in FIG. 1B), and the accelerator 100 is arranged on the right side as seen from the driver D (that is, the left side in FIG. 1B). Further, in FIG. 1B, the foot Da (thigh Da1) in a state in which the brake is operated is indicated by a solid line, and the foot Da (thigh Da1) in a state in which the accelerator 100 is operated is indicated by a broken line. ing.

Therefore, in the in-vehicle device 10 according to the present embodiment, based on the in-vehicle image A, the movement of the foot Da such that the foot Da moves from the position when the brake is operated to the position when the accelerator is operated is detected. In this case, the motion of the foot Da is detected as an accelerator motion. It should be noted that even if the foot Da has not moved from the position at which the brake is operated, detecting the accelerator operation by simply detecting the state where the foot Da is at the position at which the accelerator is operated is also included.

Continuing with the description of FIG. 1A, the in-vehicle device 10 determines whether or not the vehicle C is in a start-prohibited state (step S2). The start-prohibited situation is a situation in which the vehicle C is prohibited from starting. As an example, the start-prohibited situation is a situation in which the display state of a traffic light existing in the traveling direction of the vehicle C is in a display state (for example, a red light) instructing the vehicle C to stop, or a situation in which the vehicle C exists in the vicinity of the vehicle C. This includes situations where the distance to an object (eg, a vehicle ahead, a wall, etc.) is relatively short, but these are examples and are not limiting. It should be noted that the determination of the start prohibition situation is performed based on, for example, an image outside the vehicle captured by the outside camera 12 (see FIG. 2), which will be described later.

When the in-vehicle device 10 determines that the detected motion of the foot Da is an accelerator motion and that the vehicle C is prohibited from starting (step S3), the detected motion of the foot Da is the accelerator 100 is detected as an erroneous operation (step S4). That is, since the driver D is performing the accelerator operation in spite of the fact that the vehicle C is in a situation where the vehicle C should not be started (the start prohibition situation), the in-vehicle device 10 detects the operation of the foot part Da as an erroneous operation of the accelerator 100. detected as

As described above, in this embodiment, by using the movement of the foot Da of the driver D detected based on the in-vehicle image A and the situation of the vehicle C (whether or not the start is prohibited), An erroneous operation of the accelerator 100 by the driver D can be detected with high accuracy.

Next, when the in-vehicle device 10 detects that the motion of the foot Da is an erroneous operation of the accelerator, it notifies the driver of the erroneous operation of the accelerator (step S5). Thus, in this embodiment, it is possible to make the driver D recognize the erroneous operation of the accelerator 100 by notifying the driver of the erroneous operation of the accelerator. In addition, although notification of an erroneous operation of the accelerator is performed, for example, via the output unit 14 (see FIG. 2) such as a speaker or a display of the in-vehicle device 10, the notification is not limited to this.

Further, the in-vehicle device 10 according to the present embodiment may detect and notify an erroneous operation of the accelerator 100 according to the moving state of the vehicle C. FIG. For example, the in-vehicle device 10 may detect and notify an erroneous operation of the accelerator 100 while the vehicle C is moving (more precisely, when the vehicle C starts moving (starts moving)). As a result, it is possible to accurately detect and notify an erroneous operation of the accelerator 100 by the driver D.

Specifically, the in-vehicle device 10 determines whether the vehicle C is moving based on movement information indicating the movement state of the vehicle C. FIG. For example, an in-vehicle image A can be used as the movement information.

Here, the in-vehicle image A used as movement information will be described with reference to FIG. 1C. As shown in FIG. 1C , in-vehicle image A may include the exterior of vehicle C through window 111 of driver's seat 110 and window 121 of passenger's seat 120 . Therefore, the in-vehicle device 10 determines whether or not the vehicle C is moving based on the appearance of the exterior of the vehicle C in the in-vehicle image A.

For example, outside the vehicle C, there are various objects such as a roadway or guardrails along the roadway, signs, lanes, sidewalks, advertisements, and the like, which become feature points when the in-vehicle image A is analyzed. Therefore, the in-vehicle device 10 analyzes the in-vehicle image A (for example, performs optical flow analysis) to detect feature points of an object existing outside the vehicle C, and when the feature points move, the vehicle C moves. It can be determined that In the above description, an example in which the in-vehicle image A is used as movement information is shown, but the present invention is not limited to this. signal, etc.) may be used.

When the in-vehicle device 10 determines that the detected motion of the foot Da is an accelerator motion, determines that the vehicle C is in a start-prohibited state, and determines that the vehicle C is moving, the foot Da is an erroneous operation of the accelerator 100 and is notified. That is, although the vehicle C is in a situation where the vehicle C should not start (start prohibition situation), the driver D is performing an accelerator operation, and the vehicle C is moving (because it is starting to move). The in-vehicle device 10 detects the motion of the foot portion Da as an erroneous operation of the accelerator 100 and notifies it.

In this way, the in-vehicle device 10 can detect the motion of the foot Da of the driver D detected based on the in-vehicle image A, the state of the vehicle C (whether or not the start is prohibited, and whether or not the vehicle C is moving). or not), an erroneous operation of the accelerator 100 by the driver D can be accurately detected and notified.

In addition, as described above, since the in-vehicle device 10 can detect an erroneous operation of the accelerator 100 using the in-vehicle image A of the in-vehicle camera 11, it can be easily retrofitted to the completed vehicle C, for example.

<Vehicle system configuration>
Next, a configuration of a vehicle system including the vehicle-mounted device 10 according to the embodiment will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration example of the vehicle system 1 including the in-vehicle device 10. As shown in FIG. In addition, in the block diagram of FIG. 2, only the components necessary for explaining the features of the present embodiment are represented by functional blocks, and the description of general components is omitted.

In other words, each component illustrated in the block diagram of FIG. 2 is functionally conceptual and does not necessarily need to be physically configured as illustrated. For example, the specific forms of distribution and integration of each functional block are not limited to those shown in the figure, and all or part of them can be functionally or physically distributed in arbitrary units according to various loads and usage conditions.・It is possible to integrate and configure.

As shown in FIG. 2 , the vehicle system 1 includes an in-vehicle device 10 and a vehicle sensor group 40 . The vehicle sensor group 40 is mounted on the vehicle and outputs vehicle information regarding the vehicle. Specifically, the vehicle sensor group 40 includes various sensors that output signals indicating various states (situations) regarding the vehicle. For example, vehicle sensor group 40 includes vehicle speed sensor 41 , accelerator sensor 42 , brake sensor 43 , movement distance sensor 44 and shift sensor 45 .

A vehicle speed sensor 41 outputs a signal indicating the vehicle speed of the vehicle. The accelerator sensor 42 outputs a signal indicating the amount of operation of an accelerator (more precisely, an accelerator pedal). The brake sensor 43 outputs a signal indicating the amount of operation of the brake (more precisely, the brake pedal). Travel distance sensor 44 outputs a signal indicating the starting distance of the vehicle. A shift sensor 45 outputs a signal indicating the position of the shift lever. Each signal of the vehicle sensor group 40 including the vehicle speed sensor 41 and the like is transmitted to the in-vehicle device 10 .

The vehicle sensor group 40 is not limited to the various sensors described above, and may include other types of sensors such as a steering angle sensor in addition to or instead of the various sensors. In the above description, the signals of the vehicle sensor group 40 are transmitted to the in-vehicle device 10. However, the present invention is not limited to this. may

The in-vehicle device 10 includes an in-vehicle camera 11 , an exterior camera 12 , an acceleration sensor 13 , an output section 14 , a control section 20 and a storage section 30 .

The in-vehicle camera 11 is installed at an appropriate position in the vehicle, and captures an image of the interior of the vehicle, for example. The exterior camera 12 is installed at an appropriate position on the vehicle, and captures an image of the exterior of the vehicle, for example. In other words, the exterior camera 12 images the surroundings of the vehicle such as the front of the vehicle. The in-vehicle camera 11 outputs the captured in-vehicle image to the control unit 20 , and the outside camera 12 outputs the captured out-vehicle image to the control unit 20 .

The in-vehicle camera 11 and the exterior camera 12 are, for example, cameras having a lens and an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), but are not limited to this. .

The acceleration sensor 13 detects acceleration acting on the vehicle and outputs a signal indicating the detected acceleration to the control unit 20 . The output unit 14 outputs various types of information, such as notification when an erroneous operation of the accelerator by the driver is detected. For example, the output unit 14 includes an audio output unit such as a speaker and a display unit such as a display, and outputs various information such as notifications to the driver of the vehicle.

The control unit 20 includes an acquisition unit 21, a detection unit 22, a determination unit 23, a detection unit 24, and a notification unit 25, and includes, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), RAM ( Random Access Memory), data flash, input/output ports, etc., including computers and various circuits.

The CPU of the computer functions as an acquisition unit 21, a detection unit 22, a determination unit 23, a detection unit 24, and a notification unit 25 of the control unit 20 by reading and executing programs stored in the ROM, for example.

In addition, at least one or all of the acquisition unit 21, the detection unit 22, the determination unit 23, the detection unit 24, and the notification unit 25 of the control unit 20 are ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), etc. It can also be configured with hardware of

The storage unit 30 is, for example, a storage unit configured by a storage device such as a nonvolatile memory, data flash, or hard disk drive. The storage unit 30 can store accelerator operation information 31, information on various programs, and the like. Note that the in-vehicle device 10 may acquire the above-described programs and various information via another computer or portable recording medium connected via a wired or wireless network.

The accelerator operation information 31 is information about the accelerator operation of the driver. For example, the accelerator motion information 31 includes information that serves as a determination criterion in processing for determining that the motion of the driver's foot is an accelerator motion. Here, the accelerator operation information 31 will be explained using FIG. FIG. 3 is a diagram showing an example of the accelerator operation information 31. As shown in FIG.

As shown in FIG. 3, the accelerator operation information 31 includes items such as "Position when braking" and "Position when accelerating", and each item is associated with each other.

"Position when braking" is information indicating the position of the driver's foot while the brake is being operated, and more specifically, information indicating the position of the driver's thigh. The position of the foot Da (thigh Da1) of the driver in the state of operating the brake is the position indicated by the solid line in FIG. 1B.

The "position during braking" of the accelerator operation information 31 described above may include, for example, information indicating the position of the foot of the driver when it is estimated that the brake is being operated. That is, as an example, the driver presses a brake setting button (not shown) while the driver is operating the brake, and the position of the driver's feet when the brake setting button is pressed is set as the "position during braking" ( stored). As another example, an instruction to perform a braking operation is output to the driver via the output unit 14 or the like, and the position of the driver's feet at that time is set (stored) as the "position during braking". may Note that the method of setting the "position during braking" described above is merely an example and is not limited, and any method can be used.

"Position at the time of accelerator" is information indicating the position of the foot of the driver who is operating the accelerator, and more specifically, information indicating the position of the thigh of the driver. The position of the foot Da (thigh Da1) of the driver who is operating the accelerator is the position indicated by the dashed line in FIG. 1B.

The "accelerator position" of the accelerator operation information 31 may include, for example, information indicating the position of the foot of the driver when it is estimated that the accelerator is being operated. That is, as an example, when the driver presses an accelerator setting button (not shown) while the driver is operating the accelerator, the position of the driver's feet when the accelerator setting button is pressed is set as the "position during accelerator operation" ( stored). As another example, an instruction to operate the accelerator is output to the driver via the output unit 14 or the like, and the position of the driver's foot at that time is set (stored) as the "position during accelerator operation". may It should be noted that the method of setting the above-described "position at the time of acceleration" is merely an example and is not limited, and any method can be used.

The accelerator operation information 31 described above is stored in advance in the storage unit 30, but is not limited to this. That is, for example, while driving a vehicle, learning using a machine learning model is performed on the tendency of the position of the foot (thigh) during acceleration or braking, and accelerator operation information 31 corresponding to the learning result is stored in the storage unit. 30 may be stored.

In the example shown in FIG. 3, the position during braking is "F1" and the position during acceleration is "G1". In the example shown in FIG. 3, for the sake of convenience, the position during braking is represented as "F1", and the position during acceleration is represented as "G1". information shall be stored.

Returning to the description of FIG. 2, the acquisition unit 21 of the control unit 20 acquires various images, information, signals, and the like. For example, the acquisition unit 21 acquires an in-vehicle image from the in-vehicle camera 11 . In addition, when the image inside the vehicle includes the appearance of the outside of the vehicle, the acquisition unit 21 acquires the image inside the vehicle as movement information indicating the movement state of the vehicle. That is, the vehicle interior image may include the exterior of the vehicle in the window 111 of the driver's seat 110 and the window 121 of the passenger seat 120 (see FIG. 1C) as described above. to get as The acquisition unit 21 outputs the acquired in-vehicle image to the detection unit 22, the determination unit 23, and the like.

In addition, the acquisition unit 21 acquires an image outside the vehicle from the camera 12 outside the vehicle. In addition, the acquisition unit 21 acquires an image outside the vehicle as situation information indicating the situation of the vehicle. Such situation information (image outside the vehicle) is information used in the process of determining whether or not the vehicle is in a start-prohibited situation, which will be described later.

Moreover, the acquisition part 21 may acquire the image outside a vehicle as movement information. That is, for example, when the vehicle interior image does not include the vehicle interior image as described above, the acquisition unit 21 may acquire the vehicle exterior image as the movement information instead of or in addition to the vehicle interior image. The acquisition unit 21 outputs the acquired outside image to the detection unit 22, the determination unit 23, and the like.

Further, the acquiring unit 21 may acquire signals from the vehicle sensor group 40, which are signals transmitted from the vehicle, as movement information. That is, for example, when the vehicle interior image does not include the above-described vehicle exterior, the acquisition unit 21 may acquire signals from the vehicle sensor group 40 as movement information instead of or in addition to the vehicle interior image. . The acquisition unit 21 outputs the acquired signals of the vehicle sensor group 40 to the detection unit 22, the determination unit 23, and the like.

In addition, the acquisition unit 21 may store the acquired in-vehicle image and the acquired outside-vehicle image in the storage unit 30 as operation records of the vehicle. In addition, when the acquisition unit 21 detects sudden braking, sudden acceleration, sudden steering of the vehicle, impact due to a contact accident, etc. based on the output of the acceleration sensor 13, the acquisition unit 21 acquires data of the vehicle interior image and the vehicle exterior image for a predetermined time before and after the detection. may be processed to prohibit overwriting.

The detection unit 22 detects the movement of the driver's feet based on the in-vehicle image. For example, the detection unit 22 analyzes the in-vehicle image and detects the driver's feet (specifically, thighs (thighs)). Then, the detection unit 22 detects the detected movement of the leg (thigh). In the above description, the detection unit 22 detects the thighs of the driver's legs, but the present invention is not limited to this. or lower legs may be detected.

Based on the detected leg (thigh) motion and the accelerator motion information 31, the detection unit 22 determines whether or not the motion of the driver's foot is an accelerator motion. For example, if the foot detected from the in-vehicle image moves from the position of the brake operation in the accelerator operation information 31 to the position of the accelerator operation, the detection unit 22 detects the detected foot (thigh). ) is determined to be an accelerator operation, that is, the operation of the foot (thigh) is detected as an accelerator operation. Conversely, if the foot detected from the in-vehicle image does not move from the brake operation position to the accelerator operation position in the accelerator operation information 31, the detected foot movement is It is determined that the operation is not an accelerator operation, that is, the operation of the foot is not detected as an accelerator operation. The detection unit 22 outputs information indicating the detection result to the detection unit 24 .

The determination unit 23 performs a process of determining whether or not the vehicle is in a start prohibited state. For example, the determination unit 23 determines whether or not the vehicle is in a start-prohibited state based on the vehicle situation information (external image). For example, the determination unit 23 analyzes the image outside the vehicle, which is the situation information, and detects the display state of a traffic signal existing in the traveling direction of the vehicle. Then, when the display state of the detected traffic signal is a display state (for example, a red light) instructing the vehicle to stop, the determination unit 23 determines that the vehicle is in a situation in which the vehicle should not start, i.e., in a start-prohibited situation. do. Conversely, when the detected display state of the traffic light is a display state (for example, a green light) indicating that the vehicle should proceed, the determination unit 23 determines that the vehicle is in a state where it is possible to start (start possible state). , that is, it is determined that the start prohibition situation does not exist. The determination unit 23 outputs information indicating the determination result to the detection unit 24 .

As described above, in the present embodiment, it is determined whether or not the vehicle is in a start-prohibited situation based on the image outside the vehicle. It is possible to accurately determine whether or not

In addition, in this embodiment, the display state of the traffic light is detected based on the image outside the vehicle to determine whether or not the vehicle is in a start-prohibited state. can judge well.

In the above description, it is determined whether or not the vehicle is in a start-prohibited state based on the display state of the traffic light, which is obtained based on the state information of the vehicle. However, the present invention is not limited to this. For example, the determination unit 23 analyzes the image outside the vehicle, which is the situation information, and calculates the distance from the vehicle to objects existing around the vehicle. The object here is, for example, a forward vehicle, a wall, or the like, but is not limited to these. Then, the determination unit 23 determines whether or not the vehicle is in the start prohibited state based on the calculated distance. For example, when the calculated distance is equal to or less than a predetermined value, the determination unit 23 determines that the vehicle is prohibited from starting. The above predetermined value is set to a value such that it is estimated that the vehicle may come into contact with an object if the vehicle starts moving, but is not limited to this and is set to an arbitrary value. It is possible. Conversely, when the calculated distance is greater than the predetermined value, the determination unit 23 determines that the vehicle is in a state where it is possible to start (start possible state), that is, it is not in a start prohibited state.

As described above, in this embodiment, the distance to an object existing around the vehicle is calculated based on the image outside the vehicle, and whether or not the vehicle is in the start prohibited state is determined based on the calculated distance. Therefore, it is possible to more accurately determine whether or not the vehicle is in the start prohibited state.

Further, the determination unit 23 performs processing for determining whether or not the vehicle is moving. For example, the determination unit 23 determines whether the vehicle is moving based on movement information indicating the movement state of the vehicle. As the movement information, as described above, an image inside the vehicle, an image outside the vehicle, signals from the vehicle sensor group 40, and the like can be used.

For example, when the image inside the vehicle includes the appearance of the outside of the vehicle, the determination unit 23 analyzes the image inside the vehicle (for example, optical flow analysis) and identifies feature points of objects (for example, guardrails, signs, etc.) that exist outside the vehicle. It is determined that the vehicle is moving when the feature point moves. Conversely, the determination unit 23 determines that the vehicle is not moving, that is, is stopped when the feature points in the vehicle interior image do not move. The determination unit 23 outputs information indicating the determination result to the detection unit 24 .

As described above, in this embodiment, it is determined whether or not the vehicle is moving based on the in-vehicle image. It is possible to accurately determine whether or not

In addition, when the image inside the vehicle does not include the appearance of the outside of the vehicle, the determination unit 23 analyzes the image outside the vehicle (for example, optical flow analysis), and determines the feature points of objects (for example, guardrails, signs, etc.) that exist outside the vehicle. is detected, and it is determined that the vehicle is moving when the feature point moves. Conversely, when the feature point in the image outside the vehicle does not move, the determination unit 23 determines that the vehicle is not moving, that is, is stopped.

As described above, in the present embodiment, it is determined whether or not the vehicle is moving based on the image outside the vehicle. It is possible to accurately determine whether or not

The determination unit 23 may determine whether the vehicle is moving based on signals from the vehicle sensor group 40 instead of or in addition to the in-vehicle image and the exterior image. As an example, the determining unit 23 determines that the vehicle is moving when a vehicle speed signal indicating that the vehicle is moving is transmitted from the vehicle speed sensor 41 of the vehicle sensor group 40 . As another example, the determination unit 23 determines that the vehicle is moving when an accelerator signal indicating that the vehicle is moving is transmitted from the accelerator sensor 42 . As another example, the determination unit 23 determines that the vehicle is moving when a moving distance signal indicating that the vehicle is moving is transmitted from the moving distance sensor 44 .

As described above, in this embodiment, it is determined whether or not the vehicle is moving based on the signals from the vehicle sensor group 40, in other words, based on the signals transmitted from the vehicle. With a simple configuration that uses signals from the vehicle sensor group 40, it is possible to accurately determine whether or not the vehicle is moving.

The detection unit 24 detects that the operation of the foot of the driver is an erroneous operation of the accelerator, based on the detection result by the detection unit 22 and the information indicating the determination result by the determination unit 23 .

For example, if the detection unit 24 determines that the detected motion of the leg (for example, the thigh) is an accelerator motion to operate the accelerator and that the vehicle is prohibited from starting, the leg (the thigh) ) is an erroneous operation of the accelerator. That is, since the driver is performing the accelerator operation in spite of the fact that the vehicle is in a start-prohibited state in which the vehicle should not start, the detection unit 24 recognizes the operation of the foot (thigh) as an erroneous operation of the accelerator. detect. The detection unit 24 outputs information indicating the detection result to the notification unit 25 .

As described above, in the present embodiment, the movement of the driver's leg (thigh) detected based on the in-vehicle image and the determination result of whether or not the vehicle is in the start-prohibited state are used. Thus, it is possible to accurately detect an erroneous operation of the accelerator by the driver.

The detection unit 24 may further detect an erroneous operation of the accelerator according to the movement state of the vehicle. For example, when the detection unit 24 determines that the detected movement of the leg (for example, the thigh) is an accelerator movement, that the vehicle is prohibited from starting, and that the vehicle is moving, It detects that the movement of the foot is an erroneous operation of the accelerator. That is, although the vehicle is in a start-prohibited state in which the vehicle should not start, the driver is performing an accelerator operation and the vehicle is moving (because it is starting to move). The movement of the buttocks (thighs) is detected as an erroneous operation of the accelerator.

As described above, in the present embodiment, the operation of the driver's leg (thigh) detected based on the in-vehicle image, the determination result of whether or not the vehicle is in the start-prohibited state, and the By using the determination result as to whether or not the is moving, it is possible to accurately detect an erroneous operation of the accelerator by the driver.

The notification unit 25 notifies the driver of the erroneous operation of the accelerator. For example, when the notification unit 25 detects that the motion of the foot is an erroneous operation of the accelerator, the notification unit 25 notifies the driver of the erroneous operation. Specifically, the notification unit 25 notifies via the output unit 14 .

Here, an example of notification when the output unit 14 is a display unit such as a display will be described with reference to FIG. FIG. 4 is a diagram showing an example of notification of an erroneous operation of the accelerator. As shown in FIG. 4 , when the output unit 14 is the display unit, the notification unit 25 causes the display column 200 to display information indicating that the driver's foot movement is an accelerator erroneous operation. Although illustration is omitted, if the output unit 14 is an audio output unit such as a speaker, the notification unit 25 outputs audio, a buzzer, or the like indicating that the movement of the driver's foot is an erroneous operation of the accelerator. output from the unit.

As described above, in the present embodiment, it is possible to make the driver recognize the erroneous operation of the accelerator by notifying the driver of the erroneous operation of the accelerator.

Continuing the description of FIG. 2, the notification unit 25 may prohibit notification of an erroneous operation according to the shift state of the vehicle. That is, the notification unit 25 may detect the shift state of the vehicle, and prohibit the notification of the erroneous operation when the detected shift state is other than drive. Specifically, the notification unit 25 may prohibit notification of an erroneous operation when the detected shift state is other than drive, that is, when the vehicle is in a shift state in which the vehicle does not travel forward (for example, neutral, parking, etc.). Conversely, the notification unit 25 may notify the erroneous operation only when the shift state of the vehicle is drive.

As a result, in this embodiment, when the shift state is other than drive and the vehicle does not travel forward, notification of an erroneous operation of the accelerator can be prevented. Notifications can be suppressed.

It should be noted that the detection of the shift state of the vehicle described above is performed, for example, based on an in-vehicle image. That is, for example, when the image of the shift lever is included in the image inside the vehicle, the notification unit 25 analyzes the image inside the vehicle, detects the position of the shift lever, and detects the shift state of the vehicle based on the detected position.

In the above description, the notification unit 25 detects the shift state of the vehicle based on the in-vehicle image, but is not limited to this. A shift state of the vehicle may be detected.

<Control processing of in-vehicle device>
Next, an example of a specific processing procedure in the in-vehicle device 10 will be described with reference to FIG. FIG. 5 is a flowchart showing a processing procedure executed by the in-vehicle device 10. As shown in FIG.

As shown in FIG. 5, the control unit 20 of the in-vehicle device 10 stores accelerator operation information 31 in the storage unit 30 (step S10). For example, the control unit 20 can determine the position of the driver's foot when the brake is being operated, which is information indicating the position of the foot of the driver when the brake is being applied, and the position of the foot of the driver when the accelerator is being operated. Information such as "position at the time of accelerator" is stored as accelerator operation information 31 in advance.

After step S11, the control unit 20 executes an erroneous accelerator operation detection process and the like. For example, the control unit 20 acquires the vehicle interior image from the vehicle interior camera 11 and acquires the vehicle exterior image from the vehicle exterior camera 12 (step S11).

Next, the controller 20 detects the motion of the driver's feet based on the in-vehicle image (step S12). Subsequently, the control unit 20 determines whether or not the detected motion of the driver's foot is an accelerator motion based on the accelerator motion information 31 (step S13).

If the controller 20 determines that the motion of the foot of the driver is not the accelerator motion (step S13, No), the control unit 20 skips the subsequent processes. On the other hand, when it is determined that the foot motion of the driver is the accelerator motion (step S13, Yes), the control unit 20 determines whether or not the vehicle is in a start-prohibited state based on the image outside the vehicle. (step S14).

When it is determined that the vehicle is not in the start prohibited state (step S14, No), the control unit 20 skips the subsequent processes. On the other hand, when it is determined that the vehicle is prohibited from starting (step S14, Yes), the control unit 20 determines whether or not the vehicle is moving based on the in-vehicle image (step S15).

When it is determined that the vehicle is not moving (step S15, No), the control unit 20 skips subsequent processing. On the other hand, when it is determined that the vehicle is moving (step S15, Yes), the controller 20 detects and notifies that the movement of the foot is an erroneous operation of the accelerator (step S16).

As described above, the in-vehicle device 10 according to the embodiment has the controller 20 . The control unit 20 detects the movement of the driver's feet based on the vehicle interior image that captures the state of the interior of the vehicle. The control unit 20 determines whether or not the vehicle is in a start-prohibited state in which the vehicle is prohibited from starting, based on the situation information indicating the vehicle situation. When it is determined that the detected foot motion is an accelerator motion and that the vehicle is prohibited from starting, the controller 20 detects that the foot motion is an accelerator erroneous operation. . As a result, it is possible to accurately detect an erroneous operation of the accelerator by the driver.

Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the invention are not limited to the specific details and representative embodiments so shown and described. Accordingly, various changes may be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and equivalents thereof.

10 In-vehicle device 11 In-vehicle camera 12 Out-of-vehicle camera 20 Control unit

Claims (8)

An in-vehicle device having a control unit,
The control unit
Detecting the movement of the driver's feet based on an image of the interior of the vehicle,
Determining whether or not the vehicle is prohibited from starting based on situation information indicating the vehicle status,
When it is determined that the detected motion of the foot is an accelerator motion and the vehicle is in the prohibited start state, it is detected that the motion of the foot is an erroneous operation of the accelerator. ,
In-vehicle device. The control unit
detecting motion of the driver's thigh based on the in-vehicle image;
When it is determined that the detected thigh motion is the accelerator motion and the vehicle is in the start prohibited state, detecting that the thigh motion is an erroneous operation of the accelerator;
The in-vehicle device according to claim 1 . The control unit
Acquiring an image outside the vehicle that captures the state of the exterior of the vehicle as the situation information, and determining whether or not the start prohibition situation exists based on the acquired image outside the vehicle;
The in-vehicle device according to claim 1 or 2. The control unit
Detecting the display state of a traffic signal existing in the direction of travel of the vehicle based on the image outside the vehicle, and determining whether or not the start prohibition situation exists based on the detected display state of the traffic signal;
The in-vehicle device according to claim 3. The control unit
calculating a distance from the vehicle to an object existing around the vehicle based on the image outside the vehicle, and determining whether or not the start prohibition situation exists based on the calculated distance;
The in-vehicle device according to claim 3 or 4. The control unit
Notifying the driver of the erroneous operation when detecting that the operation of the foot is an erroneous operation of the accelerator;
The in-vehicle device according to any one of claims 1 to 5. The control unit
detecting the shift state of the vehicle, and prohibiting notification of the erroneous operation when the detected shift state is other than drive;
The in-vehicle device according to claim 6. A control method for an in-vehicle device,
Detecting the movement of the driver's feet based on an image of the interior of the vehicle,
Determining whether or not the vehicle is prohibited from starting based on situation information indicating the vehicle status,
When it is determined that the detected motion of the foot is an accelerator motion and the vehicle is in the prohibited start state, it is detected that the motion of the foot is an erroneous operation of the accelerator. ,
A control method for an in-vehicle device.

Images