JP2023009068A - Image processing device, imaging apparatus and moving body - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device which can smooth traffic.

SOLUTION: An image processing device 10 concerning the present disclosure comprises: a communication interface 12; and a processor 13. The processor 13 detects an image of other moving body 2 from a peripheral video obtained by imaging a periphery of a moving body 1, and performs behavior determination processing for determining a behavior of the moving body 1 on the basis of a state of the detected other moving body 2. The communication interface 12 outputs information for instructing the determined behavior of the moving body 1 to the moving body 1. The processor 13 determines a first behavior of the moving body 1 on the basis of a first state of other moving body 2 detected from the peripheral video as the behavior determination processing, and outputs information for instructing the first determined behavior to the moving body 1. Then, the processor 13 determines a second behavior of the moving body 1 when it is determined that other moving body 2 performs dangerous driving on the basis of a second state of other moving body 2 detected from a peripheral video after the first behavior.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present disclosure relates to an image processing device, an imaging device, a moving object, and an image processing method.

Conventionally, an in-vehicle camera detects the state of the other vehicle, the state of pedestrians, the state of the approach road, etc., and notifies the driver only when it is possible to turn right or not. There is known a driving support device that provides truly necessary information to the driver (see, for example, Patent Literature 1).

JP 2008-243065 A

However, by simply detecting the state of the other vehicle as in the prior art, for example, there is a problem that it is not possible to accurately determine whether or not the other vehicle is driving the vehicle. be. In the future, in a society where vehicles equipped with automatic driving functions will become commonplace, there is concern that erroneous judgments of reckless driving will hinder smooth traffic flow.

An object of the present disclosure is to solve the above-described problems and to provide an image processing device capable of smoothing traffic.

An image processing device according to an embodiment of the present disclosure includes a processor and an output unit. The processor detects a state of another moving body from a peripheral image of the surroundings of the moving body, and performs behavior determination processing of determining a behavior of the moving body based on the detected state of the other moving body. The output unit outputs to the mobile object information indicating the behavior of the mobile object determined by the processor. In the behavior determination processing, the processor determines a first behavior of the moving object based on a first state of another moving object detected from the peripheral image, and instructs the determined first behavior. Information is output to the moving object via the output unit. Then, the processor determines that the other moving body is driving dangerously based on the second state of the other moving body detected from the surrounding image after the first behavior of the moving body. If so, determine a second behavior of the moving object.

An imaging device according to an embodiment of the present disclosure includes the image processing device described above and an input unit that acquires the peripheral video.

A mobile object according to an embodiment of the present disclosure is equipped with the imaging device described above.

An image processing method according to an embodiment of the present disclosure is an image processing method executed by an image processing device, which detects the state of another moving body detected from a surrounding image captured around the moving body, and detects the state of the detected moving body. determining a first behavior of the moving body based on a first state of another moving body; outputting information indicating the determined first behavior to the moving body; When it is determined that the other moving body is driving dangerously based on the second state of the other moving body detected from the surrounding image after the first behavior by and determining behavior.

According to the present disclosure, smooth traffic can be achieved.

1 is a diagram illustrating an example configuration of an image processing apparatus according to an embodiment of the present disclosure; FIG. FIG. 10 is a diagram showing an example of the behavior of the host vehicle when the other vehicle is performing the first leaning operation on the host vehicle; FIG. 10 is a diagram showing an example of the behavior of the host vehicle when the other vehicle is performing the first leaning operation on the host vehicle; FIG. 10 is a diagram showing an example of the behavior of the host vehicle when the other vehicle is performing the first leaning operation on the host vehicle; FIG. 10 is a diagram showing an example of the behavior of the own vehicle when the other vehicle is performing a second leaning operation with respect to the own vehicle; FIG. 10 is a diagram showing an example of the behavior of the own vehicle when the other vehicle is performing a second leaning operation with respect to the own vehicle; FIG. 10 is a diagram showing an example of the behavior of the own vehicle when the other vehicle is performing a second leaning operation with respect to the own vehicle; FIG. 10 is a diagram showing an example of the behavior of the host vehicle when the other vehicle is performing a third ramping operation with respect to the host vehicle; FIG. 10 is a diagram showing an example of the behavior of the host vehicle when the other vehicle is performing a third ramping operation with respect to the host vehicle; FIG. 10 is a diagram showing an example of the behavior of the host vehicle when the other vehicle is performing a third ramping operation with respect to the host vehicle; FIG. 10 is a diagram showing an example of the behavior of the own vehicle when the other vehicle is performing a fourth leaning operation with respect to the own vehicle. FIG. 10 is a diagram showing an example of the behavior of the own vehicle when the other vehicle is performing a fourth leaning operation with respect to the own vehicle. FIG. 10 is a diagram showing an example of the behavior of the own vehicle when the other vehicle is performing a fourth leaning operation with respect to the own vehicle. FIG. 12 is a diagram showing an example of the behavior of the own vehicle when the opponent vehicle is performing a fifth ramping operation with respect to the own vehicle. FIG. 12 is a diagram showing an example of the behavior of the own vehicle when the opponent vehicle is performing a fifth ramping operation with respect to the own vehicle. FIG. 12 is a diagram showing an example of the behavior of the own vehicle when the opponent vehicle is performing a fifth ramping operation with respect to the own vehicle. FIG. 12 is a diagram showing an example of the behavior of the host vehicle when the opponent vehicle is performing a sixth ramping operation with respect to the host vehicle; FIG. 12 is a diagram showing an example of the behavior of the host vehicle when the opponent vehicle is performing a sixth ramping operation with respect to the host vehicle; FIG. 12 is a diagram showing an example of the behavior of the host vehicle when the opponent vehicle is performing a sixth ramping operation with respect to the host vehicle; 4 is a flow chart illustrating an example of an image processing method according to an embodiment of the present disclosure; It is a figure which shows an example of a state transition diagram. 4 is a flowchart showing an example of a first rushing driving determination process in an image processing method according to an embodiment of the present disclosure; 7 is a flowchart showing an example of a second rushing driving determination process in an image processing method according to an embodiment of the present disclosure; FIG. 11 is a flowchart showing an example of a third rushing driving determination process in an image processing method according to an embodiment of the present disclosure; FIG. FIG. 11 is a flowchart showing an example of a fourth rushing driving determination process in an image processing method according to an embodiment of the present disclosure; FIG. FIG. 11 is a flowchart showing an example of a fifth rushing driving determination process in an image processing method according to an embodiment of the present disclosure; FIG. FIG. 11 is a flowchart showing an example of a sixth rushing driving determination process in an image processing method according to an embodiment of the present disclosure; FIG.

Embodiments of the present disclosure will be exemplified below with reference to the drawings. In each figure, the same reference numerals indicate the same or equivalent components.

In this specification, six types of specific examples will be given and explained as "flickering driving". "Inciting driving" is an act in which the driver of the other vehicle impedes the driving of the own vehicle for some reason or purpose, and hinders the smoothness of traffic. The first lean driving is defined as "driving in which the other vehicle shortens the inter-vehicle distance with respect to the own vehicle". The second harassment driving is defined as "driving in which the other vehicle threatens the host vehicle with its horn". The third rushing driving is defined as "driving in which the other vehicle meanders behind the own vehicle". The fourth type of leaning driving is defined as "driving in which the other vehicle passes behind the own vehicle". The fifth type of leaning driving is defined as "driving in which the other vehicle sticks to or chases the own vehicle". The sixth leaning driving is defined as "driving in which the other vehicle moves closer to the host vehicle".

<Configuration of image processing apparatus>
A configuration of an image processing apparatus 10 according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an example of the configuration of an image processing apparatus 10 according to this embodiment.

As shown in FIG. 1, an image processing device 10 is mounted on a moving body 1 (eg, own vehicle), and detects the state of another moving body (eg, other vehicle) from a peripheral image of the surroundings of the moving body 1. To detect. The image processing device 10 determines the behavior to be executed by the moving body 1 based on the detected state of the other moving body. The image processing device 10 outputs to the moving body 1 information that instructs the behavior to be executed by the moving body 1 . The moving body 1 further includes, for example, a vehicle speed sensor, a ranging sensor, a microphone, a radar, a sonar, a lidar, etc., in addition to the imaging device 1A.

As the mobile body 1, for example, there is a vehicle having an automatic driving function. In the present embodiment, "automatic driving" includes automating part or all of user operations for driving a vehicle. For example, automated driving may include levels 1 to 5 defined in SAE (Society of Automotive Engineers). In the following description, the moving body 1 is assumed to have a fully automated driving function of level 4 or higher defined by SAE.

The image processing apparatus 10 includes a communication interface 12 as an output section, a processor 13 and a storage section 14 . An imaging device 1A including an image processing device 10 and an imaging unit 11 as an input unit is mounted on the moving object 1 .

The imaging unit 11 acquires a peripheral image of the surroundings of the moving body 1 and outputs the acquired peripheral image to the processor 13 . The imaging unit 11 is, for example, an in-vehicle camera mounted on the mobile object 1 . A plurality of imaging units 11 may be mounted on the moving body 1 . When four vehicle-mounted cameras are mounted on the mobile object 1, for example, the first vehicle-mounted camera can capture an image of the peripheral area in front of the mobile object 1 and at least part of the front side surface of the mobile object 1. placed in the correct position. For example, the second in-vehicle camera is arranged at a position capable of imaging the rear peripheral area of the mobile body 1 and at least part of the rear side surface of the mobile body 1 . For example, the third in-vehicle camera is arranged at a position capable of capturing an image of the peripheral area on the left side of the moving body 1 and at least part of the left side of the moving body 1 . For example, the fourth in-vehicle camera is arranged at a position capable of capturing an image of the surrounding area on the right side of the moving body 1 and at least part of the right side of the moving body 1 . By arranging the image capturing unit 11 in this manner, the image capturing unit 11 can capture images of the four surrounding areas of the moving object 1 .

The imaging unit 11 includes at least an imaging optical system and an imaging element.

The imaging optical system includes, for example, optical members such as one or more lenses and a diaphragm. A lens included in the imaging optical system is, for example, a lens with a wide angle of view such as a fisheye lens. The imaging optical system forms a subject image on the light receiving surface of the imaging element.

The imaging device includes, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. A plurality of pixels are arranged on the light receiving surface of the imaging device. The imaging device captures a subject image formed on the light receiving surface to generate a captured image. The imaging unit 11 outputs a captured image generated by the imaging element to the processor 13 via a cable or wirelessly. The imaging unit 11 may output the captured image to an external device such as an ECU (Electronic Control Unit) mounted on the moving body 1, a display, or a navigation device. The imaging unit 11 may also have a function of performing predetermined image processing such as white balance adjustment processing, exposure adjustment processing, and gamma correction processing on the captured image.

The communication interface 12 is an interface that communicates with various control systems of the moving body 1 via wire or wireless. The communication interface 12 communicates with a control system that controls running of the mobile body 1, a control system that controls other than running of the mobile body 1 (for example, turning on/off lights, blinking blinkers, sounding a horn, etc.), and the like. I do.

The storage unit 14 includes, for example, a primary storage device or a secondary storage device. The storage unit 14 stores various information and programs required for the operation of the image processing apparatus 10 .

The processor 13 includes, for example, a dedicated processor such as a DSP (Digital Signal Processor) or a general-purpose processor such as a CPU (Central Processing Unit). The processor 13 controls the operation of the image processing apparatus 10 as a whole. For example, the processor 13 detects the state of another mobile object from the peripheral image captured by the imaging unit 11, and performs behavior determination processing for determining the behavior of the mobile object 1 based on the detected state of the other mobile object. Other mobile object states detected by the processor 13 include, for example, a first state, a second state, and a third state.

The processor 13 selects, as the first state of the other moving bodies, for example, a state in which the vehicle-to-vehicle distance is reduced to the moving body 1, a state in which the moving body 1 is threatened with a horn, and a state in which the moving body 1 meanders behind. , a state of passing behind the moving body 1, a state of sticking or chasing the moving body 1, and a state of moving closer to the moving body 1 are detected. The processor 13 detects, for example, the continuation of the first state or repetition of the first state as the second state of the other moving body. The processor 13 detects, for example, continuation of the first state or repetition of the first state, as the third state of the other moving body.

The processor 13 accesses various control systems of the mobile body 1 via the communication interface 12 and determines the behavior of the mobile body 1 . Behaviors of the moving body 1 determined by the processor 13 include, for example, a first behavior, a second behavior, and a third behavior.

As the first behavior of the mobile object 1, the processor 13, for example, continues running at a legal speed, changes lanes, sounds a horn to call attention to other mobile objects, evacuates to the nearest convenience store, and moves to the nearest convenience store. Decide whether to evacuate to a nearby service area or a parking area. For example, the processor 13 accesses a control system that controls the running of the mobile body 1, continues running at legal speed, changes lanes, evacuates to the nearest convenience store, evacuates to the nearest service area or parking area. , etc., determine the first behavior in the moving body 1 . For example, the processor 13 accesses a control system that controls other than running of the mobile body 1 and determines the first behavior of the mobile body 1, such as sounding a horn to alert other mobile bodies.

As the second behavior of the moving body 1, the processor 13 can, for example, report to the police, stop, decelerate, evacuate to the nearest convenience store, evacuate to the nearest service area or parking area, drive to the drive recorder. Any one of recording, uploading peripheral images to a predetermined network, and displaying that another moving body is driving dangerously is determined. The second behavior is the behavior of the moving body 1 when the first state continues or is repeated by another moving body even after the moving body 1 performs the first behavior. and is also called “risk avoidance behavior”.

For example, the processor 13 accesses a control system that controls the traveling of the mobile body 1, and performs actions in the mobile body 1 such as stopping, deceleration, evacuation to the nearest convenience store, evacuation to the nearest service area or parking area, and the like. Determine the second behavior. For example, the processor 13 accesses the control system that controls the movement of the moving body 1 other than the running, and records the second behavior of the moving body 1, such as recording to the drive recorder and displaying that another moving body is driving dangerously. to decide.

As the third behavior of the moving body 1, the processor 13, for example, reports to the police, stops, evacuates to the nearest convenience store, evacuates to the nearest service area or parking area, locks the door after stopping, to determine which of The third behavior is the behavior of the moving body 1 when the first state continues or is repeated by another moving body even after the moving body 1 performs the second behavior. It is also called “emergency evacuation behavior”.

For example, the processor 13 accesses a control system that controls the running of the mobile body 1, and performs third actions in the mobile body 1 such as stopping, evacuation to the nearest convenience store, evacuation to the nearest service area or parking area, and the like. determines the behavior of For example, the processor 13 accesses a control system that controls the movement of the moving body 1 other than traveling, and determines a third behavior of the moving body 1, such as door locking after stopping.

The processor 13 performs the following operations as behavior determination processing. The processor 13 detects the state of the other moving body from the peripheral image captured by the imaging unit 11, and determines the first behavior of the moving body 1 based on the detected first state of the other moving body. The processor 13 outputs information indicating the determined first behavior to the control system of the mobile object 1 via the communication interface 12 .

For example, the processor 13 determines the first behavior of the mobile object 1, and continues running at a legal speed, changes lanes, sounds a horn to alert other mobile objects, and evacuates to the nearest convenience store. , evacuation to the nearest service area or parking area, etc. is output to the mobile object 1 via the communication interface 12, and the mobile object 1 is made to perform these operations.

Further, after the first behavior of the moving object 1, the processor 13 detects the state of another moving object from the peripheral image captured by the imaging unit 11, and based on the detected second state of the other moving object, is determined to be driving dangerously, the second behavior of the moving body 1 is determined. The processor 13 outputs information indicating the determined second behavior to the control system of the mobile object 1 via the communication interface 12 .

For example, the processor 13 determines the second behavior of the mobile unit 1, such as call the police, stop, slow down, evacuate to the nearest convenience store, evacuate to the nearest service area or parking area, drive recorder , uploading surrounding images to a predetermined network, displaying that another mobile is driving dangerously, etc. is output to the mobile 1 via the communication interface 12, 1 to perform these operations.

In other words, when the processor 13 determines that the other mobile body is driving dangerously even with respect to the first behavior of the mobile body 1, the processor 13 estimates that the other mobile body is driving in a rush. . Then, the processor 13 determines the second behavior of the moving body 1 in order to more accurately detect whether or not another moving body is driving in a rush.

Further, after the second behavior of the moving object 1, the processor 13 detects the state of the other moving object from the peripheral image captured by the imaging unit 11, and based on the detected third state of the other moving object, When it is determined that the moving body is driving dangerously, a third behavior of the moving body 1 is determined. The processor 13 outputs information indicating the determined third behavior to the control system of the mobile object 1 via the communication interface 12 .

For example, the processor 13 determines the third behavior of the mobile body 1, and calls the police, stops, evacuates to the nearest convenience store, evacuates to the nearest service area or parking area, and moves to the door after stopping. Information instructing locking, etc., is output to the moving body 1 via the communication interface 12, and the moving body 1 is made to perform these operations.

That is, when the processor 13 determines that the other mobile body is driving dangerously even with respect to the second behavior of the mobile body 1, the processor 13 estimates that the other mobile body is driving in a rush. . Then, the processor 13 determines the third behavior of the moving body 1 in order to more accurately detect whether or not another moving body is driving.

As described above, after the first behavior of the moving body 1, the processor 13 determines whether the other moving body is driving dangerously based on the second state of the other moving body detected from the peripheral image captured by the imaging unit 11. If it is determined that the movement is performed, the second behavior of the moving body 1 is determined. Furthermore, after the second behavior of the moving body 1, the processor 13 determines that the other moving body is driving dangerously based on the third state of the other moving body detected from the peripheral image captured by the imaging unit 11. , the third behavior of the moving body 1 is determined.

According to the image processing apparatus 10 according to the present embodiment, the first state of another moving body is detected from the surrounding image of the surroundings of the moving body 1, and the first state of the moving body 1 is detected based on the first state. determines the behavior of After the first behavior of the moving body 1, a second state of the other moving body is detected from a surrounding image of the surroundings of the moving body 1, and based on the second state, the other moving body performs dangerous driving. is performed, the second behavior of the moving body 1 is determined. That is, even after the first behavior of the moving body 1, it is determined whether or not the other moving body is still performing dangerous driving, thereby determining whether the other moving body is driving with a jerk to the moving body. can be accurately determined. That is, compared with the conventional method in which it is determined whether or not the other mobile body is performing the fanning operation with respect to the mobile body simply by detecting the state of the other mobile body, the judgment of the fanning operation can be performed. Accuracy can be improved. As a result, the moving body 1 can take appropriate measures against other moving bodies that are actually driving while driving, so that traffic can be made smoother.

<Specific example of behavior of moving object 1>
Next, with reference to FIGS. 2 to 7, a specific example of the behavior of the moving body 1 when the other moving body 2 is performing a tilting operation with respect to the moving body 1 will be described.

[In the case of the first rush driving]
As shown in FIG. 2A , the image processing apparatus 10 captures an image of the peripheral area behind the moving object 1 by using the imaging unit 11 arranged at a position capable of imaging the peripheral area behind the moving object 1 . Also, the image processing device 10 calculates the inter-vehicle distance between the moving body 1 and another moving body 2 .

The image processing device 10 estimates that the other moving body 2 is performing the first lean driving based on the peripheral image behind the moving body 1 and the inter-vehicle distance between the moving body 1 and the other moving body 2. do. In this case, as shown in FIG. 2B , the image processing device 10 determines “lane change” as the first behavior of the moving body 1 and outputs information instructing the first behavior to the moving body 1. do.

After that, the image processing device 10 performs another movement even after the first behavior of the mobile body 1 based on the peripheral image behind the mobile body 1 and the inter-vehicle distance between the mobile body 1 and the other mobile body 2 . When it is determined that the first tilting operation by the body 2 continues, the second behavior of the moving body 1 is determined. In this case, as shown in FIG. 2C, the image processing apparatus 10 determines "retreat to the nearest convenience store" as the second behavior of the moving object 1, and transmits information instructing the second behavior. Output to moving body 1.

[In the case of the second rush driving]
As shown in FIG. 3A , the image processing apparatus 10 captures an image of the peripheral area behind the moving object 1 by using the imaging unit 11 arranged at a position where the peripheral area behind the moving object 1 can be imaged. Further, the image processing device 10 detects the rumble of the horn of another moving body 2 using a microphone mounted on the moving body 1 .

The image processing device 10 estimates that the other moving body 2 is performing the second swaying operation based on the peripheral image behind the moving body 1 and the sound of the horn of the other moving body 2 . In this case, as shown in FIG. 3B , the image processing apparatus 10 determines “lane change” as the first behavior of the moving body 1 and outputs information instructing the first behavior to the moving body 1. do.

After that, the image processing device 10 detects the second behavior by the other mobile body 2 even after the first behavior of the mobile body 1 based on the peripheral image behind the mobile body 1 and the sound of the horn of the other mobile body 2 . 1, the second behavior of the moving object 1 is determined. In this case, as shown in FIG. 3C, the image processing apparatus 10 determines "retreat to the nearest convenience store" as the second behavior of the mobile body 1, and transmits information instructing the second behavior. Output to moving body 1.

[In the case of the third rush driving]
As shown in FIG. 4A , the image processing apparatus 10 captures an image of the peripheral area behind the moving body 1 using the imaging unit 11 arranged at a position capable of imaging the peripheral area behind the moving body 1 . Also, the image processing device 10 calculates the inter-vehicle distance between the moving body 1 and another moving body 2 .

The image processing device 10 estimates that the other moving body 2 is performing the third lean driving based on the peripheral image behind the moving body 1 and the inter-vehicle distance between the moving body 1 and the other moving body 2. do. In this case, as shown in FIG. 4B , the image processing apparatus 10 determines “lane change” as the first behavior of the moving body 1 and outputs information instructing the first behavior to the moving body 1. do.

After that, the image processing device 10 performs another movement even after the first behavior of the mobile body 1 based on the peripheral image behind the mobile body 1 and the inter-vehicle distance between the mobile body 1 and the other mobile body 2 . When it is determined that the first tilting operation by the body 2 continues, the second behavior of the moving body 1 is determined. In this case, as shown in FIG. 4C, the image processing apparatus 10 determines "retreat to the nearest convenience store" as the second behavior of the mobile object 1, and transmits information instructing the second behavior. Output to moving body 1.

[In the case of fourth rush driving]
As shown in FIG. 5A , the image processing apparatus 10 captures an image of the peripheral area behind the moving object 1 by using the imaging unit 11 arranged at a position where the peripheral area behind the moving object 1 can be imaged. Also, the image processing device 10 calculates the inter-vehicle distance between the moving body 1 and another moving body 2 . Further, the image processing device 10 detects that the other moving body 2 is blinking the front light behind the moving body 1 or that the other moving body 2 is blinking based on the surrounding image captured by the imaging unit 11 It is detected that the front light is turned on in a high luminance state upward at the rear of the moving body 1.例文帳に追加

The image processing device 10 detects the other moving body 2 based on the peripheral image behind the moving body 1, the inter-vehicle distance between the moving body 1 and the other moving body 2, and the state of the front lights of the other moving body 2. It is presumed that the driver is driving in the fourth mode. In this case, as shown in FIG. 5B , the image processing apparatus 10 determines “lane change” as the first behavior of the mobile object 1 and outputs information instructing the first behavior to the mobile object 1 . do.

After that, the image processing device 10 calculates the image of the moving body 1 based on the peripheral image behind the moving body 1, the inter-vehicle distance between the moving body 1 and another moving body 2, and the state of the front lights of the other moving body 2. Even after the first behavior, when it is determined that the first swaying operation by the other moving body 2 continues, the second behavior of the moving body 1 is determined. In this case, as shown in FIG. 5C, the image processing apparatus 10 determines "retreat to the nearest convenience store" as the second behavior of the mobile body 1, and transmits information instructing the second behavior. Output to moving body 1.

[In the case of the fifth rush driving]
As shown in FIG. 6A , the image processing apparatus 10 captures an image of the peripheral area behind the moving object 1 by using the imaging unit 11 arranged at a position where the peripheral area behind the moving object 1 can be imaged. Also, the image processing device 10 calculates the inter-vehicle distance between the moving body 1 and another moving body 2 .

The image processing device 10 estimates that the other moving body 2 is performing the fifth lean driving based on the peripheral image behind the moving body 1 and the inter-vehicle distance between the moving body 1 and the other moving body 2. do. In this case, as shown in FIG. 6B , the image processing device 10 determines “lane change” as the first behavior of the moving object 1 and outputs information instructing the first behavior to the moving object 1. do.

After that, the image processing device 10 performs another movement even after the first behavior of the mobile body 1 based on the peripheral image behind the mobile body 1 and the inter-vehicle distance between the mobile body 1 and the other mobile body 2 . When it is determined that the fifth fanning operation by the body 2 continues, the second behavior of the moving body 1 is determined. In this case, as shown in FIG. 6C, the image processing apparatus 10 determines "retreat to the nearest convenience store" as the second behavior of the mobile object 1, and transmits information instructing the second behavior. Output to moving body 1.

After that, the image processing device 10, based on the surrounding image of the moving body 1 and the inter-vehicle distance between the moving body 1 and the other moving body 2, continues to move the other moving body 2 even after the second behavior of the moving body 1. When it is determined that the fifth fanning operation by is continuing, the third behavior of the moving body 1 is determined. In this case, as shown in FIG. 6C, the image processing apparatus 10 determines "lock the door and call the police" as the third behavior of the moving object 1, and instructs the third behavior. Information is output to the mobile unit 1 .

[In the case of the sixth fanning operation]
As shown in FIG. 7A , the image processing apparatus 10 performs the peripheral area behind the moving body 1 , at least a part of the rear side surface of the moving body 1 , the right peripheral area of the moving body 1 , and the right side of the moving body 1 . The peripheral areas of the rear, rear side, right side, and right side of the moving body 1 are captured by the imaging unit 11 arranged at a position capable of capturing an image of at least a partial area of the plane. Also, the image processing device 10 calculates the inter-vehicle distance between the moving body 1 and another moving body 2 .

The image processing device 10 determines that the other moving body 2 is the second moving body 2 based on the surrounding image behind the moving body 1, the surrounding image on the right side of the moving body 1, and the inter-vehicle distance between the moving body 1 and the other moving body 2. It is presumed that the driver is driving with 6 rushes. In this case, as shown in FIG. 7B, the image processing apparatus 10 determines "ringing of the horn for calling attention to other moving bodies" as the first behavior of the moving body 1, and the first behavior to the moving body 1.

After that, the image processing device 10 calculates the first image of the moving body 1 based on the surrounding image behind the moving body 1, the surrounding image on the right side of the moving body 1, and the inter-vehicle distance between the moving body 1 and another moving body 2. If it is determined that the sixth fanning operation by the other moving body 2 continues even after the first behavior, the second behavior of the moving body 1 is determined. In this case, as shown in FIG. 7C , the image processing apparatus 10 determines “lane change, deceleration” as the second behavior of the moving body 1 and transmits information instructing the second behavior to the moving body 1. and output.

After that, the image processing device 10, based on the peripheral image behind the moving body 1 and the inter-vehicle distance between the moving body 1 and the other moving body 2, determines whether the moving body 1 still moves after the second behavior. When it is determined that the sixth fanning operation by the body 2 continues, the third behavior of the moving body 1 is determined. In this case, the image processing device 10 determines “stopping to avoid collision” as the third behavior, and outputs information instructing this behavior to the moving object 1 .

Table 1 shows other movements detected by the processor 13 based on the first behavior of the mobile body 1 and the second behavior of the mobile body 1 determined by the processor 13, peripheral images, etc., when traveling on a general road. 2 is a table summarizing a first state of a body 2 and a second state of another moving body 2;

For example, when the first state of the other moving body 2 is "the other moving body 2 shortens the inter-vehicle distance to the moving body 1", the processor 13 determines that the first behavior of the moving body 1 is " Obey the legal speed limit and continue driving" or "Check the safety of the lane ahead and change lanes".

Furthermore, when the second state of the other moving body 2 is "the first state of the other moving body 2 continues" or "the other moving body 2 sticks to the moving body 1 and chases after it" , the processor 13 determines "drop in at the nearest convenience store and give way" as the second behavior of the mobile object 1 .

For example, if the first state of the other moving body 2 is "the other moving body 2 honks the horn to the moving body 1", the processor 13 determines that the first behavior of the moving body 1 is " Obey the legal speed limit and continue driving" or "Check the safety of the lane ahead and change lanes".

Furthermore, when the second state of the other moving body 2 is "the first state of the other moving body 2 continues" or "the other moving body 2 sticks to the moving body 1 and chases after it" , the processor 13 determines "drop in at the nearest convenience store and give way" as the second behavior of the mobile object 1 .

For example, if the first state of the other moving body 2 is "the other moving body 2 is meandering behind the moving body 1", the processor 13 determines that the first behavior of the moving body 1 is " Obey the legal speed limit and continue driving" or "Check the safety of the lane ahead and change lanes".

Furthermore, when the second state of the other moving body 2 is "the first state of the other moving body 2 continues" or "the other moving body 2 sticks to the moving body 1 and chases after it" , the processor 13 determines "drop in at the nearest convenience store and give way" as the second behavior of the mobile object 1 .

For example, if the first state of the other moving body 2 is "the other moving body 2 is passing against the moving body 1", the processor 13 determines that the first behavior of the moving body 1 is "statutory It decides whether to observe the speed and continue driving, or to confirm the safety of the lane ahead of the course change and change lanes.

Furthermore, when the second state of the other moving body 2 is "the first state of the other moving body 2 continues" or "the other moving body 2 sticks to the moving body 1 and chases after it" , the processor 13 determines "drop in at the nearest convenience store and give way" as the second behavior of the mobile object 1 .

For example, when the first state of the other moving body 2 is "the other moving body 2 is sticking to the moving body 1 and chasing it", the processor 13 determines that the first behavior of the moving body 1 is " stop at a convenience store, etc., and give way” or “check the safety of the lane to change the course and change the lane”.

Furthermore, when the second state of the other moving body 2 is "the first state of the other moving body 2 continues", the processor 13 determines that the second behavior of the moving body 1 is " Stop at the nearest convenience store, etc., and report to the police."

For example, if the first state of the other moving body 2 is "the other moving body 2 is moving towards the moving body 1", the processor 13, as the first behavior of the moving body 1, "Sound horn to alert other moving body 2" is determined.

Furthermore, if the second state of the other moving body 2 is "the other moving body 2 stops the side-hulling action" or "the other moving body 2 continues the side-hulling action", the processor 13 As the second behavior of the mobile body 1, "continue running" or "decelerate to avoid collision" is determined.

Table 2 shows other movements detected by the processor 13 based on the first behavior of the mobile body 1 and the second behavior of the mobile body 1 determined by the processor 13, peripheral images, etc., while driving on a highway. 2 is a table summarizing a first state of a body 2 and a second state of another moving body 2;

For example, when the first state of the other moving body 2 is "the other moving body 2 shortens the inter-vehicle distance to the moving body 1", the processor 13 determines that the first behavior of the moving body 1 is " Obey the legal speed limit and continue driving" or "Check the safety of the lane ahead and change lanes".

Furthermore, when the second state of the other moving body 2 is "the first state of the other moving body 2 continues" or "the other moving body 2 sticks to the moving body 1 and chases after it" , the processor 13 determines "temporarily evacuate to a safe place such as a service area or a parking area" as the second behavior of the mobile unit 1 .

For example, if the first state of the other moving body 2 is "the other moving body 2 honks the horn to the moving body 1", the processor 13 determines that the first behavior of the moving body 1 is " Obey the legal speed limit and continue driving" or "Check the safety of the lane ahead and change lanes".

Furthermore, when the second state of the other moving body 2 is "the first state of the other moving body 2 continues" or "the other moving body 2 sticks to the moving body 1 and chases after it" , the processor 13 determines "temporarily evacuate to a safe place such as a service area or a parking area" as the second behavior of the mobile unit 1 .

For example, if the first state of the other moving body 2 is "the other moving body 2 is meandering behind the moving body 1", the processor 13 determines that the first behavior of the moving body 1 is " Obey the legal speed limit and continue driving" or "Check the safety of the lane ahead and change lanes".

Furthermore, when the second state of the other moving body 2 is "the first state of the other moving body 2 continues" or "the other moving body 2 sticks to the moving body 1 and chases after it" , the processor 13 determines "temporarily evacuate to a safe place such as a service area or a parking area" as the second behavior of the mobile unit 1 .

For example, if the first state of the other moving body 2 is "the other moving body 2 is passing against the moving body 1", the processor 13 determines that the first behavior of the moving body 1 is "statutory It decides whether to observe the speed and continue driving, or to confirm the safety of the lane ahead of the course change and change lanes.

Furthermore, when the second state of the other moving body 2 is "the first state of the other moving body 2 continues" or "the other moving body 2 sticks to the moving body 1 and chases after it" , the processor 13 determines "temporarily evacuate to a safe place such as a service area or a parking area" as the second behavior of the mobile unit 1 .

For example, when the first state of the other moving body 2 is "the other moving body 2 is sticking to the moving body 1 and chasing it", the processor 13 determines that the first behavior of the moving body 1 is " enter the service area or parking area” or “check the safety of the lane to change the course and change the lane”.

Furthermore, when the second state of the other moving body 2 is "the first state of the other moving body 2 continues", the processor 13 determines that the second behavior of the moving body 1 is " Call the police" or "Enter the nearest service area or parking area and call the police".

For example, if the first state of the other moving body 2 is "the other moving body 2 is moving towards the moving body 1", the processor 13, as the first behavior of the moving body 1, "Sound horn to alert other moving body 2" is determined.

Furthermore, if the second state of the other moving body 2 is "the other moving body 2 stops the side-hulling action" or "the other moving body 2 continues the side-hulling action", the processor 13 As the second behavior of the mobile body 1, "continue running" or "decelerate to avoid collision" is determined.

As described above, the image processing device 10 detects the first state of the other moving body 22 from the surrounding image of the surroundings of the moving body 1, and detects the first state of the moving body 1 based on the first state. determine behavior. Then, after the first behavior of the moving object 1, the image processing device 10 detects the second state of the other moving object 22 from the surrounding image of the surroundings of the moving object 1, and based on the second state, When it is determined that the other moving body 22 is driving dangerously, the second behavior of the moving body 1 is determined. That is, the image processing device 10 determines whether or not the other moving body 22 is still driving dangerously after the first behavior of the moving body 1, thereby allowing the other moving body 2 to Therefore, it is possible to accurately determine whether or not the driver is driving in a rush. As a result, the moving body 1 can take appropriate measures against the other moving body 22 that is actually driving while driving, so traffic can be made smoother.

<Operation of image processing apparatus>
Next, an image processing method in the image processing apparatus 10 according to this embodiment will be described with reference to FIG. FIG. 8 is a flow chart showing an example of an image processing method in the image processing apparatus 10 according to this embodiment.

Step S0: The processor 13 performs initialization processing.

Step S1: The processor 13 performs a first rush driving determination process. The information used by the processor 13 in the first rushing driving determination process includes, for example, the vehicle speed of the moving object 1 and the inter-vehicle distance between the moving object 1 and another moving object 2 according to the vehicle speed of the moving object 1 (Table 3 ), a residence time during which another mobile body 2 stays within a predetermined range of the mobile body 1, and the like.

For example, the processor 13 determines that the other moving body 2 is performing the first tilting operation when the other moving body 2 stays within the predetermined range of the moving body 1 for 3 seconds or longer. For example, the processor 13 determines that the other moving body 2 is not performing the first tilting operation when the other moving body 2 stays within the predetermined range of the moving body 1 for a time shorter than 3 seconds.

Step S2: The processor 13 performs a second rushing driving determination process. The information used by the processor 13 for the second rushing driving determination processing includes, for example, the vehicle speed of the moving object 1 and the inter-vehicle distance between the moving object 1 and another moving object 2 according to the vehicle speed of the moving object 1 (Table 3 ), the number of times the horn of another moving body 2 sounds and the sound volume of the horn, and the like.

For example, the processor 13 determines that the other moving body 2 is performing the second swaying operation when the other moving body 2 sounds the horn five times or more in one minute. For example, the processor 13 determines that the other moving body 2 is not performing the second swaying operation when the other moving body 2 honks its horn less than five times per minute.

Step S3: The processor 13 performs a third rushing driving determination process. The information used by the processor 13 in the third rushing driving determination process includes, for example, the vehicle speed of the moving object 1 and the inter-vehicle distance between the moving object 1 and another moving object 2 according to the vehicle speed of the moving object 1 (Table 3 ), residence time during which the other moving body 2 stays within a predetermined range of the moving body 1, amplitude of meandering operation of the other moving body 2, period of meandering operation of the other moving body 2, and the like.

For example, when the amplitude of the meandering operation of the other moving body 2 is 900 mm or more and the cycle of the meandering operation of the other moving body 2 is 1 second or more and 5 seconds or less, the processor 13 determines that the other moving body 2 It is determined that the driver is driving in the rush of 3. For example, if the amplitude of the meandering operation of the other moving body 2 is smaller than 900 mm, or if the period of the meandering operation of the other moving body 2 is longer than 5 seconds, the processor 13 determines that the other moving body 2 is the third It is determined that the driver is not driving while inciting. The width of 900 mm is approximately half of the standard vehicle width of 1800 mm.

Step S4: The processor 13 performs a fourth rush driving determination process. The information used by the processor 13 in the fourth rushing driving determination process includes, for example, the vehicle speed of the moving object 1 and the inter-vehicle distance between the moving object 1 and another moving object 2 according to the vehicle speed of the moving object 1 (Table 3 ), residence time for the other moving body 2 to stay within a predetermined range of the moving body 1, illuminance of the front light in the other moving body 2, frequency of passing in the other moving body 2, passing in the other moving body 2 light-dark cycle, and the like.

For example, the processor 13 determines that the other moving body 2 is performing the fourth lean driving when the illuminance of the front light of the other moving body 2 is 10,000 lx or more. For example, the processor 13 determines that the other mobile body 2 is not performing the fourth lean driving when the illuminance of the front light of the other mobile body 2 is less than 10,000 lx.

For example, the processor 13 determines that the other moving body 2 is performing the fourth tilting operation when the light-dark cycle of passing in the other moving body 2 is 500 milliseconds or more and 2 seconds or less. For example, the processor 13 determines that the other moving body 2 is not performing the fourth tilting operation when the light-dark cycle of passing in the other moving body 2 is longer than 2 seconds.

Step S5: The processor 13 performs a fifth rushing driving determination process. The information used by the processor 13 in the fifth rushing driving determination process includes, for example, the vehicle speed of the moving object 1 and the inter-vehicle distance between the moving object 1 and another moving object 2 according to the vehicle speed of the moving object 1 (Table 3 ), the residence time during which the other moving body 2 stays within the predetermined range of the moving body 1, the number of times the other moving body 2 sticks and follows.

For example, if another mobile body 2 repeats sticking and chasing two or more times even after the mobile body 1 changes lanes, the processor 13 determines that the other mobile body 2 performs the fifth rush driving. determine that there is For example, the processor 13 determines that the other mobile body 2 is not performing the fifth lean driving when the other mobile body 2 does not stick or follow after the mobile body 1 changes lanes.

Step S6: The processor 13 performs a sixth fanning driving determination process. The information used by the processor 13 in the sixth rushing driving determination process includes, for example, the vehicle speed of the moving object 1 and the distance between the moving object 1 and another moving object 2 according to the vehicle speed of the moving object 1 (Table 3 ), the distance between the moving body 1 and the other moving body 2 on the side surface, the number of times the other moving body 2 moves to the side, and the like.

For example, when the distance between the moving body 1 and the other moving body 2 in the lateral direction is shorter than 2 m, the processor 13 determines that the other moving body 2 is performing the sixth lean driving. For example, when the distance between the moving body 1 and the other moving body 2 in the lateral direction is 2 m or more, the processor 13 determines that the other moving body 2 is not performing the sixth tilting operation.

For example, the processor 13 determines that the other moving body 2 is performing the sixth lean driving when the other moving body 2 repeats the side approaching two or more times even after the moving body 1 changes lanes. judge. For example, the processor 13 determines that the other moving body 2 is not performing the sixth leaning operation when the other moving body 2 does not move to the side after the moving body 1 changes lanes.

The processor 13 concurrently executes the fanning driving determination processing in steps S1 to S6 described above. The processor 13 may execute the fanning driving determination process with only one step, or may perform the fanning driving determination process by combining a plurality of steps. By combining a plurality of steps and executing the rushing driving determination process, it becomes possible to more accurately judge whether or not the other moving body 2 is conducting rushing driving.

Next, with reference to FIG. 9, the tilt determination state flag will be briefly described. FIG. 9 is a diagram showing an example of a state transition diagram for the tilt determination state flag.

As shown in FIG. 9, the tilt determination state flag has three states, for example, a "none" state, a "provisionally confirmed" state, and a "confirmed" state. The processor 13 manages the fanning determination state flag and executes fanning driving determination processing such as the above-described steps S1 to S6.

State A is the "none" state, that is, the initial state.

State B is a state of "temporary confirmation". In the state B, the processor 13 outputs an action instruction to the mobile body 1 when estimating that the other mobile body 2 is driving while driving based on the first state of the other mobile body 2 . In state B, if the processor 13 estimates that the other moving body 2 is not in a rushing operation based on the first state, it does not output an action instruction to the moving body 1 (state B→state A).

State C is a state of "fixed". In the state C, the processor 13 outputs a danger avoidance action instruction to the mobile body 1 when estimating that the other mobile body 2 is driving in a rush based on the second state. In state C, if the processor 13 estimates that the other mobile body 2 is not driving in a rush based on the second state, it does not output the danger avoidance action instruction to the mobile body 1 (state C→state A ).

According to the image processing method according to the present embodiment, the tilt determination state flag has three states. In the "fixed" state, the processor 13 outputs a danger avoidance action instruction to the moving body 1. FIG. That is, according to the image processing method according to the present embodiment, it is determined whether or not the other moving body 2 is driving with a tilt relative to the moving body 1 based only on the first state of the other moving body 2. Instead, the second state of the other mobile body 2 after the first behavior of the mobile body 1 is taken into account to determine whether or not the other mobile body 2 is driving the mobile body 1. determine whether As a result, it is possible to suppress erroneous judgments of rushing driving and reduce the probability of a useless fight occurring between the moving body 1 and another moving body 2, so that traffic can be made smoother. .

<Specific example of rush driving determination processing>
Next, with reference to FIGS. 10 to 15, a specific example of the leaning driving determination process according to the present embodiment will be described.

[First Deflection Driving Judgment Process]
FIG. 10 is a flowchart showing an example of the first leaning driving determination process. The first leaning driving determination process is executed by the processor 13 mounted on the image processing device 10 .

Step S<b>11 : The processor 13 acquires the vehicle speed of the mobile object 1 from the vehicle speed sensor mounted on the mobile object 1 .

Step S12: The processor 13 calculates an appropriate inter-vehicle distance between the moving body 1 and another moving body 2 based on Table 3. That is, the processor 13 determines a safe inter-vehicle distance and a dangerous inter-vehicle distance according to the vehicle speed of the mobile object 1 .

Step S<b>13 : The processor 13 determines whether or not another mobile object 2 exists within a predetermined range of the mobile object 1 based on the image of the surroundings of the mobile object 1 . When the processor 13 determines that another moving body 2 exists within the predetermined range of the moving body 1 (step S13: Yes), the process proceeds to step S14. When the processor 13 determines that there is no other moving body 2 within the predetermined range of the moving body 1 (step S13: No), the process proceeds to step S20.

Step S14: The processor 13 determines whether or not the time during which the other moving body 2 stays within the predetermined range of the moving body 1 is 3 seconds or longer. When the processor 13 determines that the vehicle residence time is 3 seconds or longer (step S14: Yes), the process proceeds to step S15. When the processor 13 determines that the vehicle residence time is shorter than 3 seconds (step S14: No), the process proceeds to step S20.

Step S15: The processor 13 determines whether the first tilt determination flag is in the state of "no", the state of "temporary determination", or the state of "determined". When the processor 13 determines that the first fanning determination flag is in the "confirmed" state (step S15: "confirmed"), the process proceeds to step S17. When the processor 13 determines that the first tilting determination flag is in the determination state of "temporary confirmation" (step S15: "temporary confirmation"), the process proceeds to step S16. When the processor 13 determines that the first tilting determination flag is in the "absence" determination state (step S15: "absence"), the process proceeds to step S18.

Step S16: The processor 13 switches the first tilt determination flag from the state of "provisional confirmation" to the state of "confirmation".

Step S<b>17 : The processor 13 outputs information instructing the behavior (second behavior) of the mobile body 1 to the mobile body 1 via the communication interface 12 and outputs a danger avoidance action instruction to the mobile body 1 .

Step S18: The processor 13 switches the first tilting determination flag from the state of "absent" to the state of "provisional determination".

Step S<b>19 : The processor 13 outputs information instructing the behavior (first behavior) of the mobile object 1 to the mobile object 1 via the communication interface 12 and outputs an action instruction to the mobile object 1 .

Step S20: The processor 13 sets the first fanning determination flag to the "absent" state. That is, the processor 13 presumes that the driver of the other moving body 2 does not have the intention of driving in a rush, and continues the fully automatic driving of the moving body 1 .

[Second Deflection Driving Determination Process]
FIG. 11 is a flow chart showing an example of the first leaning driving determination process. The first leaning driving determination process is executed by the processor 13 mounted on the image processing device 10 .

Step S<b>21 : The processor 13 acquires the vehicle speed of the mobile object 1 from the vehicle speed sensor mounted on the mobile object 1 .

Step S22: The processor 13 calculates an appropriate inter-vehicle distance between the moving body 1 and another moving body 2 based on Table 3. That is, the processor 13 determines a safe inter-vehicle distance and a dangerous inter-vehicle distance according to the vehicle speed of the mobile object 1 .

Step S<b>23 : The processor 13 determines whether or not another moving body 2 exists within a predetermined range of the moving body 1 based on the image of the surroundings of the moving body 1 . When the processor 13 determines that another moving body 2 exists within the predetermined range of the moving body 1 (step S23: Yes), the process proceeds to step S14. When the processor 13 determines that there is no other moving body 2 within the predetermined range of the moving body 1 (step S23: No), the process proceeds to step S30.

Step S<b>24 : The processor 13 determines whether or not the sound of the horn is continuously detected from the microphone mounted on the moving body 1 . When the processor 13 determines that the sound of the horn is continuously detected (step S24: Yes), the process proceeds to step S25. When the processor 13 determines that the horn sound is not continuously detected (step S24: No), the process proceeds to step S30.

Step S25: The processor 13 determines whether the second tilt determination flag is in the state of "no", the state of "temporary determination", or the state of "determined". When the processor 13 determines that the second fanning determination flag is in the "confirmed" state (step S25: "confirmed"), the process proceeds to step S27. When the processor 13 determines that the second tilting determination flag is in the determination state of "temporary confirmation" (step S25: "temporary confirmation"), the process proceeds to step S26. When the processor 13 determines that the second tilting determination flag is in the state of "absence" in the determination state (step S25: "absence"), the process proceeds to step S28.

Step S26: The processor 13 switches the second tilt determination flag from the state of "temporary confirmation" to the state of "confirmation".

Step S<b>27 : The processor 13 outputs information instructing the behavior (second behavior) of the mobile body 1 to the mobile body 1 via the communication interface 12 and outputs a danger avoidance action instruction to the mobile body 1 .

Step S28: The processor 13 switches the second tilt determination flag from the state of "none" to the state of "provisional determination".

Step S<b>29 : The processor 13 outputs information instructing the behavior of the mobile object 1 (second behavior) to the mobile object 1 via the communication interface 12 and outputs an action instruction to the mobile object 1 .

Step S30: The processor 13 sets the second fanning determination flag to "absent". That is, the processor 13 presumes that the driver of the other moving body 2 does not have the intention of driving in a rush, and continues the fully automatic driving of the moving body 1 .

[Third Deflection Driving Determination Process]
FIG. 12 is a flowchart showing an example of the third leaning driving determination process. The third rushing driving determination process is executed by the processor 13 mounted on the image processing device 10 .

Step S<b>31 : The processor 13 acquires the vehicle speed of the mobile object 1 from the vehicle speed sensor mounted on the mobile object 1 .

Step S32: The processor 13 calculates an appropriate inter-vehicle distance between the moving body 1 and another moving body 2 based on Table 3. That is, the processor 13 determines a safe inter-vehicle distance and a dangerous inter-vehicle distance according to the vehicle speed of the mobile object 1 .

Step S<b>33 : The processor 13 determines whether or not another moving body 2 exists within a predetermined range of the moving body 1 based on the image of the surroundings of the moving body 1 . When the processor 13 determines that another moving body 2 exists within the predetermined range of the moving body 1 (step S33: Yes), the process proceeds to step S34. When the processor 13 determines that another moving body 2 does not exist within the predetermined range of the moving body 1 (step S33: No), the process proceeds to step S40.

Step S34: The processor 13 determines whether or not the time for which the other moving body 2 stays within the predetermined range of the moving body 1 is 3 seconds or longer, and whether or not the other moving body 2 is meandering. do. When the processor 13 determines that the vehicle residence time is 3 seconds or more and that the other moving body 2 is meandering (step S34: Yes), the process proceeds to step S35. When the vehicle residence time is 3 seconds or more and the processor 13 does not determine that the other moving body 2 is meandering (step S34: No), the process proceeds to step S40. The processor 13 determines, for example, that the meandering amplitude of the other moving body 2 is equal to or greater than a predetermined amplitude, that the meandering period of the other moving body 2 is included in the predetermined period, and the like. It is determined whether or not the body 2 is meandering.

Step S35: The processor 13 determines whether the third tilt determination flag is in the state of "absent", the state of "temporary determination", or the state of "determined". When the processor 13 determines that the third fanning determination flag is in the "confirmed" state (step S35: "confirmed"), the process proceeds to step S37. When the processor 13 determines that the third tilt determination flag is in the determination state of "temporary confirmation" (step S35: "temporary confirmation"), the process proceeds to step S36. When the processor 13 determines that the third tilting determination flag is in the "absence" determination state (step S35: "absence"), the process proceeds to step S38.

Step S36: The processor 13 switches the third tilt determination flag from the state of "provisional confirmation" to the state of "confirmation".

Step S<b>37 : The processor 13 outputs information instructing the behavior (second behavior) of the mobile body 1 to the mobile body 1 via the communication interface 12 and outputs a danger avoidance action instruction to the mobile body 1 .

Step S38: The processor 13 switches the third tilting determination flag from the state of "none" to the state of "provisional determination".

Step S<b>39 : The processor 13 outputs information instructing the behavior of the mobile object 1 (third behavior) to the mobile object 1 via the communication interface 12 and outputs an action instruction to the mobile object 1 .

Step S40: The processor 13 sets the third fanning determination flag to the "absent" state. That is, the processor 13 presumes that the driver of the other moving body 2 does not have the intention of driving in a rush, and continues the fully automatic driving of the moving body 1 .

[Fourth Deflection Driving Determination Process]
FIG. 13 is a flowchart showing an example of the fourth leaning driving determination process. The fourth rushing driving determination process is executed by the processor 13 mounted on the image processing device 10 .

Step S<b>41 : The processor 13 acquires the vehicle speed of the mobile object 1 from the vehicle speed sensor mounted on the mobile object 1 .

Step S42: The processor 13 calculates an appropriate inter-vehicle distance between the moving body 1 and another moving body 2 based on Table 3. That is, the processor 13 determines a safe inter-vehicle distance and a dangerous inter-vehicle distance according to the vehicle speed of the mobile object 1 .

Step S<b>43 : The processor 13 determines whether or not another moving body 2 exists within a predetermined range of the moving body 1 based on the image of the surroundings of the moving body 1 . When the processor 13 determines that another moving body 2 exists within the predetermined range of the moving body 1 (step S43: Yes), the process proceeds to step S44. When the processor 13 determines that there is no other moving body 2 within the predetermined range of the moving body 1 (step S43: No), the process proceeds to step S50.

Step S44: The processor 13 determines whether or not the time for which the other moving body 2 stays within the predetermined range of the moving body 1 is 3 seconds or longer, and whether or not the other moving body 2 is performing a passing action. do. When the processor 13 determines that the vehicle residence time is 3 seconds or longer and that the other moving body 2 is performing a passing action (step S44: Yes), the process proceeds to step S45. When the processor 13 does not determine that the vehicle residence time is 3 seconds or longer and the other moving body 2 is performing a passing action (step S44: No), the process proceeds to step S50. The processor 13 detects, for example, that the other moving body 2 is blinking the front lights behind the moving body 1, or that the other moving body 2 is turning on the front lights behind the moving body 1 in a high brightness state. It is determined whether or not the other moving body 2 is meandering based on the fact that the other mobile body 2 is moving.

Step S45: The processor 13 determines whether the fourth tilt determination flag is in the state of "none", the state of "temporary determination", or the state of "determined". When the processor 13 determines that the fourth fanning determination flag is in the "confirmed" state (step S45: "confirmed"), the process proceeds to step S47. When the processor 13 determines that the fourth tilt determination flag is in the determination state of "provisional determination" (step S45: "provisional determination"), the process proceeds to step S46. When the processor 13 determines that the fourth tilting determination flag is in the state of "absence" (step S45: "absence"), the process proceeds to step S48.

Step S46: The processor 13 switches the fourth tilt determination flag from the "provisionally confirmed" state to the "confirmed" state.

Step S<b>47 : The processor 13 outputs information instructing the behavior (second behavior) of the mobile body 1 to the mobile body 1 via the communication interface 12 and outputs a danger avoidance action instruction to the mobile body 1 .

Step S48: The processor 13 switches the fourth tilting determination flag from the state of "none" to the state of "provisional determination".

Step S<b>49 : The processor 13 outputs information instructing the behavior of the mobile object 1 (fourth behavior) to the mobile object 1 via the communication interface 12 and outputs an action instruction to the mobile object 1 .

Step S50: The processor 13 sets the fourth fanning determination flag to the "absent" state. That is, the processor 13 presumes that the driver of the other moving body 2 does not have the intention of driving in a rush, and continues the fully automatic driving of the moving body 1 .

[Fifth rush driving determination process]
FIG. 14 is a flow chart showing an example of the fifth lean driving determination process. The fifth rushing driving determination process is executed by the processor 13 mounted on the image processing device 10 .

Step S<b>51 : The processor 13 acquires the vehicle speed of the mobile object 1 from the vehicle speed sensor mounted on the mobile object 1 .

Step S52: The processor 13 calculates an appropriate inter-vehicle distance between the moving body 1 and another moving body 2 based on Table 3. That is, the processor 13 determines a safe inter-vehicle distance and a dangerous inter-vehicle distance according to the vehicle speed of the mobile object 1 .

Step S<b>53 : The processor 13 determines whether or not another moving body 2 exists within a predetermined range of the moving body 1 based on the image of the periphery of the moving body 1 . When the processor 13 determines that another moving body 2 exists within the predetermined range of the moving body 1 (step S53: Yes), the process proceeds to step S54. When the processor 13 determines that there is no other moving body 2 within the predetermined range of the moving body 1 (step S53: No), the process proceeds to step S60.

Step S54: The processor 13 determines that the time for which the other moving body 2 stays within the predetermined range of the moving body 1 is 3 seconds or longer, and that the other moving body 2 stays within the predetermined range even after the moving body 1 changes lanes. It is determined whether or not sticking and chasing are continued. The processor 13 determines that the time for which the other mobile body 2 stays within the predetermined range of the mobile body 1 is 3 seconds or more, and that the other mobile body 2 is stuck and chased after the mobile body 1 changes lanes. If it is determined that the rotation is continued (step S54: Yes), the process proceeds to step S55. The processor 13 determines that the time for which the other mobile body 2 stays within the predetermined range of the mobile body 1 is 3 seconds or more, and that the other mobile body 2 is stuck and chased after the mobile body 1 changes lanes. If it is not determined that the rotation is continued (step S54: No), the process proceeds to step S60.

Step S55: The processor 13 determines whether the fifth tilt determination flag is in the state of "none", the state of "temporary determination", or the state of "determined". When the processor 13 determines that the fifth fanning determination flag is in the "confirmed" state (step S55: "confirmed"), the process proceeds to step S57. When the processor 13 determines that the fifth tilting determination flag is in the determination state of "temporary confirmation" (step S55: "temporary confirmation"), the process proceeds to step S56. When the processor 13 determines that the fifth tilting determination flag is in the "absence" determination state (step S55: "absence"), the process proceeds to step S18.

Step S56: The processor 13 switches the fifth tilt determination flag from the state of "temporary confirmation" to the state of "confirmation".

Step S<b>57 : The processor 13 outputs information instructing the behavior (second behavior) of the mobile body 1 to the mobile body 1 via the communication interface 12 and outputs a danger avoidance action instruction to the mobile body 1 .

Step S58: The processor 13 switches the fifth fanning determination flag from the state of "none" to the state of "temporary determination".

Step S<b>59 : The processor 13 outputs information instructing the behavior of the mobile object 1 (fifth behavior) to the mobile object 1 via the communication interface 12 and outputs an action instruction to the mobile object 1 .

Step S60: The processor 13 sets the fifth fanning determination flag to the "absent" state. That is, the processor 13 presumes that the driver of the other moving body 2 does not have the intention of driving in a rush, and continues the fully automatic driving of the moving body 1 .

[Sixth rush driving determination process]
FIG. 15 is a flowchart showing an example of the sixth leaning driving determination process. The sixth rushing driving determination process is executed by the processor 13 mounted on the image processing device 10 .

Step S<b>61 : The processor 13 acquires the vehicle speed of the mobile object 1 from the vehicle speed sensor mounted on the mobile object 1 .

Step S62: The processor 13 calculates an appropriate inter-vehicle distance between the moving body 1 and another moving body 2 based on Table 3. That is, the processor 13 determines a safe inter-vehicle distance and a dangerous inter-vehicle distance according to the vehicle speed of the mobile object 1 .

Step S<b>63 : The processor 13 determines whether or not another moving body 2 exists within a predetermined range of the moving body 1 based on the image of the surroundings of the moving body 1 . When the processor 13 determines that another moving body 2 exists within the predetermined range of the moving body 1 (step S63: Yes), the process proceeds to step S64. When the processor 13 determines that there is no other moving body 2 within the predetermined range of the moving body 1 (step S63: No), the process proceeds to step S70.

Step S64: The processor 13 determines whether or not the time during which the other moving body 2 stays within the predetermined range of the moving body 1 is 3 seconds or longer. When the processor 13 determines that the vehicle residence time is 3 seconds or longer (step S64: Yes), the process proceeds to step S65. When the processor 13 determines that the vehicle residence time is shorter than 3 seconds (step S64: No), the process proceeds to step S70.

Step S65: The processor 13 determines whether the sixth fanning determination flag is in the state of "absent," "tentatively determined," or "determined." When the processor 13 determines that the sixth fanning determination flag is in the "confirmed" state (step S65: "confirmed"), the process proceeds to step S67. When the processor 13 determines that the sixth fanning determination flag is in the determination state of "temporary confirmation" (step S65: "temporary confirmation"), the process proceeds to step S66. When the processor 13 determines that the sixth fanning determination flag is in the "absence" determination state (step S65: "absence"), the process proceeds to step S68.

Step S66: The processor 13 switches the sixth tilt determination flag from the state of "provisional confirmation" to the state of "confirmation".

Step S<b>67 : The processor 13 outputs information instructing the behavior (second behavior) of the mobile body 1 to the mobile body 1 via the communication interface 12 and outputs a danger avoidance action instruction to the mobile body 1 .

Step S68: The processor 13 switches the sixth tilt determination flag from the state of "absent" to the state of "provisional determination".

Step S<b>69 : The processor 13 outputs information instructing the behavior of the mobile object 1 (sixth behavior) to the mobile object 1 via the communication interface 12 and outputs an action instruction to the mobile object 1 .

Step S70: The processor 13 sets the sixth fanning determination flag to the "absent" state. That is, the processor 13 presumes that the driver of the other moving body 2 does not have the intention of driving in a rush, and continues the fully automatic driving of the moving body 1 .

As described above, according to the image processing method according to the present embodiment, only the first state of the other moving body 2 determines whether or not the other moving body 2 is driving with a tilt relative to the moving body 1. In consideration of the second state of the other mobile body 2 after the first behavior of the mobile body 1, the other mobile body 2 performs the fanning operation with respect to the mobile body 1. Determine whether or not As a result, it is possible to suppress erroneous judgments of rushing driving and reduce the probability of a useless fight occurring between the moving body 1 and another moving body 2, so that traffic can be made smoother. .

<Modification>
In the present embodiment, the first behavior of the moving body 1 includes continuing running at a legal speed, changing lanes, sounding a horn to call attention to other moving bodies, retreating to the nearest convenience store, and moving to the nearest convenience store. Although evacuation to a service area or a parking area has been described as an example, it is not limited to this. The first behavior of the moving body 1 may be a behavior obtained by appropriately combining these actions. Moreover, when the moving body 1 has a voice output function, the first behavior of the moving body 1 may include voice output. Further, when the moving body 1 has a function of projecting onto the ground or the like, the first behavior of the moving body 1 may include projection of a predetermined image such as a message onto the sidewalk or the like.

Further, in this embodiment, the second behavior of the moving object 1 includes notifying the police, stopping, decelerating, evacuating to the nearest convenience store, evacuating to the nearest service area or parking area, and going to the drive recorder. recording, uploading peripheral images to a predetermined network, and displaying that other moving objects are driving dangerously, etc., but the present invention is not limited to these examples. In addition, when the moving object 1 has an audio output function, the second behavior of the moving object 1 may include audio output. Further, when the moving body 1 has a function of projecting onto the ground or the like, the second behavior of the moving body 1 may include projection of a predetermined image such as a message onto the sidewalk or the like.

Further, in this embodiment, the third behavior of the moving body 1 is to report to the police, stop, evacuate to the nearest convenience store, evacuate to the nearest service area or parking area, and lock the door after stopping. , etc. have been described as an example, but the present invention is not limited to this. In addition, when the moving object 1 has an audio output function, the third behavior of the moving object 1 may include audio output. Further, when the moving body 1 has a function of projecting onto the ground or the like, the third behavior of the moving body 1 may include projection of a predetermined image such as a message onto the sidewalk or the like.

For example, the processor 13 sequentially performs actions to avoid the state of the other moving body 2, such as decelerating at the end of the traveling lane and continuing traveling, gradually deviating from the traveling lane, and stopping on the shoulder of the road. , it may be determined whether or not the other moving body 2 is driving dangerously for each operation. Further, for example, if the processor 13 repeats the first behavior, the second behavior, and the third behavior a predetermined number of times and the other moving body 2 does not stop dangerous driving, may be released.

Further, in the present embodiment, the image processing device 10 mounted on the moving body 1 detects the first state, the second state, and the third state of the other moving body 2 from the surrounding image, and detects other states. determining the first behavior based on the first state of the moving object 2; determining the second behavior based on the second state of the other moving object 2; and determining the third behavior based on the third state of the other moving object 2 Although the description has been given using an example of determining the third behavior, etc., the present invention is not limited to this. For example, detection of the first state, the second state, and the third state of the other moving object 2 from the surrounding image, which is mainly provided by the processor 13, and the detection based on the first state of the other moving object 2 Functions such as determining the first behavior, determining the second behavior based on the second state of the other moving object 2, and determining the third behavior based on the third state of the other moving object 2 may be stored in a server or the like on the network. In this case, the mobile unit 1 can communicate with the server via the network, and the mobile unit 1 transmits peripheral images to the server. The server detects the first state, the second state, and the third state of the other mobile body 2 from the peripheral image acquired from the mobile body 1, or detects the first state of the other mobile body 2. determination of the first behavior based on the second state of the other moving object 2, determination of the third behavior based on the third state of the other moving object 2, and the like. and transmits information instructing the determined behavior to the mobile object 1 via the network.

In addition, in the present embodiment, an example of a case in which the tilt determination state flag has three states of "none", "temporarily confirmed", and "confirmed" has been described. The status flag is not limited to this. Further, in the present embodiment, the case where the fanning determination state flag is used to perform the fanning driving determination process has been described as an example, but the fanning driving determination process can also be performed using a known algorithm.

Further, in the present embodiment, the case where the processor detects the state of the other moving object 2 based on the peripheral image captured by the imaging unit 11 has been described as an example, but the present invention is not limited to this example. The processor may detect the state of another moving body 2 based on various sensors mounted on the moving body 1 . For example, the processor may detect the inter-vehicle distance between the mobile object 1 and another mobile object 2 using a range sensor mounted on the mobile object 1 . For example, the processor may detect blinking of front lights by other moving bodies 2 using an illuminance sensor mounted on the moving body 1 .

Also, in the present embodiment, descriptions such as “first”, “second”, “third”, “fourth”, “fifth”, and “sixth” are identifiers for distinguishing the configurations. be. Configurations that are differentiated in descriptions such as "first" and "second" in this disclosure may interchange the numbers in that configuration. For example, a first tilting operation can exchange identifiers "first" and "second" with a second tilting operation. The second tilting operation can exchange the identifiers "second" and "third" with the third tilting operation. The third tilting operation can exchange the identifiers "third" and "fourth" with the fourth tilting operation. The fourth inflection operation can exchange the identifiers "fourth" and "fifth" with the fifth inflection operation. The fifth inflection operation can exchange the identifiers "fifth" and "sixth" with the sixth inflection operation. The exchange of identifiers is done simultaneously. The configurations are still distinct after the exchange of identifiers. Identifiers may be deleted. Configurations from which identifiers have been deleted are distinguished by codes. Interpretation of the order of such configurations based solely on the description of identifiers such as "first", "second", "third", "fourth", "fifth", and "sixth" in this disclosure; It shall not be used as a justification for the existence of lower numbered identifiers.

Although one embodiment of the present disclosure has been described with reference to figures and examples, it should be noted that various variations and modifications can be easily made by those skilled in the art based on the present disclosure. Therefore, it should be noted that these variations and modifications are included within the scope of this disclosure.

1 moving body 1A imaging device 2 other moving body 10 image processing device 11 imaging unit (input unit)
12 Communication interface (output part)
13 processor 14 storage unit

Claims (13)

a processor that detects the state of another moving body from a peripheral image of the surroundings of the moving body and determines the behavior of the moving body based on the detected state of the other moving body;
an output unit that outputs to the mobile body information indicating the behavior of the mobile body determined by the processor;
In the behavior determination processing, the processor determines a first behavior of the moving object based on a first state of another moving object detected from the peripheral image, and instructs the determined first behavior. information is output to the moving object via the output unit, and the other moving object is detected based on the second state of the other moving object detected from the surrounding image after the first behavior of the moving object; An image processing device that determines a second behavior of the moving object when it is determined that dangerous driving is being performed. The first state of the other moving body includes a state in which the other moving body shortens the inter-vehicle distance with respect to the moving body, a state in which the other moving body meanders behind the moving body, and a state in which the other moving body meanders behind the moving body. a state in which the other moving body passes behind the moving body; a state in which the other moving body sticks to or follows the moving body; and a state in which the other moving body approaches the moving body. including any one of
The image processing apparatus according to claim 1. The moving body further comprises a microphone,
In the behavior determination processing, the processor determines a first behavior of the moving body based on a state in which the other moving body threatens the moving body with a horn detected from the surrounding image and the microphone. and outputting information indicating the determined first behavior to the moving object via the output unit, and for the moving object detected from the surrounding image after the first behavior by the moving object and the microphone. If it is determined that the other moving body is driving dangerously based on the continuation or repetition of the state in which the other moving body threatens with the horn, determining a second behavior of the moving body,
The image processing apparatus according to claim 1. the second state of the other moving body includes any one of continuation of the first state and repetition of the first state;
The image processing apparatus according to claim 1 or 2. The first behavior of the moving object includes any one of continuing running at a legal speed, changing lanes, sounding a horn, and evacuating to a place of evacuation.
The image processing apparatus according to any one of claims 1 to 4. The second behavior of the moving body includes reporting to the police, stopping, decelerating, evacuating to an evacuation site, recording to a drive recorder, uploading the surrounding image to a predetermined network, and dangerous driving by the other moving body. an indication that you are doing so, including any one of
The image processing apparatus according to any one of claims 1 to 5. The processor
When it is determined that the other moving body is driving dangerously based on the third state of the other moving body detected from the surrounding image after the second behavior of the moving body, the moving body determining a third behavior;
The image processing apparatus according to any one of claims 1 to 6. the third state of the other moving object includes any one of continuation of the first state and repetition of the first state;
The image processing apparatus according to claim 7. The third behavior of the moving object includes any one of reporting to the police, stopping, evacuating to an evacuation location, and locking the door after stopping,
The image processing apparatus according to claim 7 or 8. The mobile object further comprises any one of a vehicle speed sensor, a ranging sensor, a microphone, a radar, a sonar, and a lidar,
The image processing device according to any one of claims 1 to 9. an image processing device according to any one of claims 1 to 10;
and an input unit that acquires the peripheral image. A moving body equipped with the imaging device according to claim 11 . An image processing method executed by an image processing device,
a step of detecting a state of another moving body detected from a peripheral image of the surroundings of the moving body and determining a first behavior of the moving body based on the detected first state of the other moving body; ,
a step of outputting information indicating the determined first behavior to the moving object;
When it is determined that the other moving body is driving dangerously based on the second state of the other moving body detected from the surrounding image after the first behavior of the moving body, the moving body determining a second behavior.

Images