JP2023161248A - Moving object control method - Google Patents

Abstract

To restrain power consumption when controlling automatic operation of a moving object.SOLUTION: An image captured by a camera mounted on a moving object is acquired and automatic operation of the moving object is controlled based on a result of image processing on the image. A risk level representing a degree of risk for the moving object during automatic operation is obtained based on at least one of the number of objects in a risk determination range around the moving object and a distance between the moving object and each objects in the risk determination range. A frequency of image processing when the risk level is low is set lower than a frequency of image processing when the risk level is high.SELECTED DRAWING: Figure 2

Description

The present disclosure relates to a technique for controlling a moving object including a camera.

Patent Document 1 discloses an information processing device used as a peripheral recognition device installed in an autonomous vehicle. The information processing device acquires far-field information regarding a far-field area and performs far-field information processing using the far-field information. Further, the information processing device acquires neighborhood information regarding a neighborhood area, and performs neighborhood information processing using the neighborhood information. Further, the information processing device changes the size of resources allocated to far information processing and nearby information processing, depending on the priorities of far information processing and near information processing.

Japanese Patent Application Publication No. 2017-151541

In the automatic operation of mobile objects such as vehicles and robots, cameras mounted on the mobile objects are used. More specifically, image processing is performed on the image captured by the camera, and automatic operation of the mobile object is controlled based on the result of the image processing. However, if image processing is executed more than necessary, power consumption in automatic driving control increases unnecessarily.

One objective of the present disclosure is to provide a technique that can suppress power consumption when controlling automatic operation of a mobile body.

The first aspect relates to a mobile body control method for controlling a mobile body equipped with a camera.
The mobile object control method is
Obtaining an image captured by a camera and controlling automatic operation of a mobile object based on the results of image processing on the image;
Represents the degree of risk to a moving object during automated driving based on at least one of the number of objects within the risk judgment range around the moving object and the distance between the moving object and the objects within the risk judgment range. Obtaining the risk level and
This includes setting the frequency of image processing when the risk level is low to be lower than the frequency of image processing when the risk level is high.

The second aspect relates to a mobile object control system that controls a mobile object equipped with a camera.
A mobile control system includes one or more processors.
The one or more processors are:
Obtain an image captured by a camera, control the automatic operation of a mobile object based on the results of image processing on the image,
Represents the degree of risk to a moving object during automated driving based on at least one of the number of objects within the risk judgment range around the moving object and the distance between the moving object and the objects within the risk judgment range. Get the risk level,
The frequency of image processing when the risk level is low is set lower than the frequency of image processing when the risk level is high.

According to the present disclosure, the frequency of image processing in automatic driving control is not set uniformly but dynamically. More specifically, a risk level representing the degree of risk to the moving object during automatic driving is acquired. The frequency of image processing when the risk level is low is set to be lower than the frequency of image processing when the risk level is high. In environments with a high risk level, image processing is performed frequently, ensuring the safety of automated driving. On the other hand, in an environment with a low risk level, image processing is suppressed, which reduces power consumption. In other words, since image processing is not executed more than necessary, power consumption due to image processing is suppressed. As described above, according to the present disclosure, it is possible to achieve both safety of automatic driving and suppression of power consumption.

FIG. 1 is a conceptual diagram for explaining an overview of a vehicle control system according to an embodiment. FIG. 3 is a diagram illustrating an example of the relationship between the risk level and the frequency of execution of image processing according to the embodiment. FIG. 7 is a diagram showing another example of the relationship between the risk level and the frequency of execution of image processing according to the embodiment. It is a conceptual diagram for explaining an example of the risk level concerning an embodiment. FIG. 7 is a conceptual diagram for explaining another example of the risk level according to the embodiment. 1 is a block diagram showing a configuration example of a vehicle control system according to an embodiment. It is a block diagram showing an example of driving environment information concerning an embodiment. 7 is a flowchart illustrating processing related to dynamic setting of image processing frequency according to the embodiment.

Embodiments of the present disclosure will be described with reference to the accompanying drawings.

1. Overview of vehicle control system Let's consider the control of moving objects. Examples of the moving object include a vehicle, a robot, and the like. Examples of robots include logistics robots, delivery robots, work robots, and the like. As an example, in the following description, a case will be considered in which the moving object is a vehicle. For generalization, "vehicle" in the following description shall be replaced with "mobile object".

FIG. 1 is a conceptual diagram for explaining an overview of a vehicle control system 10 according to the present embodiment. Vehicle control system 10 controls vehicle 1 . Typically, the vehicle control system 10 is mounted on the vehicle 1. Alternatively, at least a portion of the vehicle control system 10 may be placed in an external device outside the vehicle 1 to remotely control the vehicle 1.

In particular, the vehicle control system 10 controls automatic driving of the vehicle 1. Here, automatic operation means that at least one of steering, acceleration, and deceleration of the vehicle 1 is automatically performed independently of the driver's operation. For example, we assume that automatic driving is based on the premise that the driver does not necessarily have to concentrate 100% on driving (so-called automatic driving at level 3 or higher).

The vehicle 1 is equipped with one or more cameras C (hereinafter simply referred to as "cameras C") that capture images of the surroundings of the vehicle 1. The vehicle control system 10 acquires an image captured by a camera C mounted on the vehicle 1, and performs image processing on the acquired image. For example, the vehicle control system 10 analyzes images using image recognition AI (Artificial Intelligence) obtained through machine learning, and recognizes and identifies objects OBJ around the vehicle 1. Examples of objects OBJ include pedestrians, bicycles, other vehicles, roadside structures, and obstacles. The vehicle control system 10 then controls automatic driving of the vehicle 1 based on the result of the image processing. For example, the vehicle control system 10 generates a driving plan and a target trajectory based on the positional relationship with the object OBJ recognized through image processing.

However, if image processing is executed more than necessary, power consumption in automatic driving control increases unnecessarily. For example, even in situations where the risk to the vehicle 1 is extremely low, constantly executing image processing is excessive and leads to an unnecessary increase in power consumption. Therefore, the present embodiment proposes a technique that can suppress power consumption when controlling automatic driving of the vehicle 1.

According to the present embodiment, the vehicle control system 10 acquires a "risk level RL" representing the degree of risk to the vehicle 1 during automatic driving. The degree of risk to the vehicle 1 is considered to depend on the number and relative distances of objects OBJ around the vehicle 1. Therefore, the vehicle control system 10 uses a recognition sensor mounted on the vehicle 1 to recognize the object OBJ within the risk determination range RNG around the vehicle 1. The recognition sensor includes at least one of a camera C, LIDAR (Laser Imaging Detection and Ranging), and radar. The risk determination range RNG may be a fixed range or may vary in size depending on the situation. For example, the risk determination range RNG may expand as the speed of the vehicle 1 increases. The vehicle control system 10 acquires the risk level RL based on at least one of the number of objects OBJ within the risk determination range RNG and the distance between the vehicle 1 and the objects OBJ within the risk determination range RNG. Note that a specific example of the method for setting the risk level RL will be described later.

Then, the vehicle control system 10 dynamically changes the execution frequency FR of image processing in automatic driving control according to the risk level RL. More specifically, the vehicle control system 10 sets the image processing execution frequency FR when the risk level RL is relatively low than the image processing execution frequency FR when the risk level RL is relatively high. For example, the execution frequency FR of image processing is set to once every 0.1 seconds, once every second, etc.

FIGS. 2 and 3 are diagrams showing examples of the relationship between the risk level RL and the image processing execution frequency FR. In the example shown in FIG. 2, as the risk level RL becomes higher, the image processing execution frequency FR increases monotonically. In the example shown in FIG. 3, the risk level RL and the image processing execution frequency FR change in stages. In either case, the image processing execution frequency FR when the risk level RL is relatively low is set to be lower than the image processing execution frequency FR when the risk level RL is relatively high.

<Effect>
As explained above, according to the present embodiment, the execution frequency FR of image processing in automatic driving control is not uniformly set but dynamically set. More specifically, a risk level RL representing the degree of risk to the vehicle 1 during automatic driving is acquired. The image processing execution frequency FR when the risk level RL is low is set to be lower than the image processing execution frequency FR when the risk level RL is high. In an environment where the risk level RL is high, image processing is performed frequently, so the safety of automatic driving is appropriately ensured. On the other hand, in an environment where the risk level RL is low, image processing is suppressed, so power consumption is suppressed. In other words, since image processing is not executed more than necessary, power consumption due to image processing is suppressed. In this way, according to the present embodiment, it is possible to achieve both safety of automatic driving and suppression of power consumption.

2. Setting Examples of Risk Level Various setting examples of the risk level RL will be explained below.

2-1. First Example In the first example, the risk level RL is set based on the number of objects OBJ within the risk determination range RNG around the vehicle 1. More specifically, the risk level RL is set to become higher as the number of objects OBJ in the risk determination range RNG increases.

2-2. Second Example In the second example, the risk level RL is set based on the relative distance between the object OBJ and the vehicle 1 within the risk determination range RNG. More specifically, the risk level RL is set to become higher as the relative distance becomes smaller. For example, the risk level RL regarding the object OBJ closest to the vehicle 1 is used as the representative value. As another example, the sum of risk levels RL calculated for each object OBJ may be used.

2-3. Third Example FIGS. 4 and 5 are conceptual diagrams for explaining the third example. In the third example, the risk determination range RNG is divided into a first risk determination range RNG-A closer to the vehicle 1 and a second risk determination range RNG-B surrounding the first risk determination range RNG-A. If object OBJ exists in the first risk determination range RNG-A, the risk level RL is set to "high". If the object OBJ does not exist in the first risk determination range RNG-A, but the object OBJ exists in the second risk determination range RNG-B, the risk level RL is set to "medium". If the object OBJ does not exist in either the first risk determination range RNG-A or the second risk determination range RNG-B, the risk level RL is set to "medium". The same applies when the number of divisions in the risk determination range RNG is three or more.

3. Example of vehicle control system 3-1. Configuration Example FIG. 6 is a block diagram showing a configuration example of the vehicle control system 10 according to the present embodiment. The vehicle control system 10 includes a recognition sensor 20, a position sensor 30, a vehicle condition sensor 40, a traveling device 50, and a control device 100.

The recognition sensor 20 recognizes (detects) the situation around the vehicle 1 . The recognition sensor 20 includes a camera C that captures images of the surroundings of the vehicle 1. The recognition sensor 20 may include LIDAR (Laser Imaging Detection and Ranging), radar, or the like.

The position sensor 30 detects the position and orientation of the vehicle 1. As the position sensor 30, a GPS (Global Positioning System) sensor is exemplified.

Vehicle condition sensor 40 detects the condition of vehicle 1. For example, the vehicle condition sensor 40 includes a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, and the like.

Travel device 50 includes a steering device, a drive device, and a braking device. The steering device steers the wheels. For example, the steering device includes a power steering (EPS: Electric Power Steering) device. The drive device is a power source that generates driving force. Examples of the drive device include an engine, an electric motor, an in-wheel motor, and the like. The braking device generates braking force.

Control device 100 controls vehicle 1 . The control device 100 includes one or more processors 110 (hereinafter simply referred to as processors 110) and one or more storage devices 120 (hereinafter simply referred to as storage devices 120). Processor 110 executes various processes. For example, processor 110 includes a CPU (Central Processing Unit). The storage device 120 stores various information. Examples of the storage device 120 include volatile memory, nonvolatile memory, HDD (Hard Disk Drive), SSD (Solid State Drive), and the like. Control device 100 may include one or more ECUs (Electronic Control Units). A part of the control device 100 may be an information processing device outside the vehicle 1. In that case, part of the control device 100 communicates with the vehicle 1 and remotely controls the vehicle 1.

The vehicle control program PROG is a computer program for controlling the vehicle 1. Various processes by the control device 100 are realized by the processor 110 executing the vehicle control program PROG. Vehicle control program PROG is stored in storage device 120. Alternatively, the vehicle control program PROG may be recorded on a computer-readable recording medium.

3-2. Driving Environment Information The control device 100 acquires driving environment information 200 indicating the driving environment of the vehicle 1. Driving environment information 200 is stored in storage device 120.

FIG. 7 is a block diagram illustrating an example of driving environment information 200. The driving environment information 200 includes map information 210, surrounding situation information 220, position information 230, and vehicle status information 240.

Map information 210 includes a general navigation map. The map information 210 may indicate lane arrangement and road shape. The map information 210 may include location information of structures, landmarks, and the like. The control device 100 acquires map information 210 of the required area from the map database.

The surrounding situation information 220 is information indicating the surrounding situation of the vehicle 1. Control device 100 recognizes the surrounding situation of vehicle 1 using recognition sensor 20 and acquires surrounding situation information 220. For example, the surrounding situation information 220 includes an image 221 captured by the camera C. As another example, the surrounding situation information 220 includes point cloud information obtained by LIDAR.

The surrounding situation information 220 further includes object information 222 regarding objects OBJ around the vehicle 1. Examples of objects OBJ include pedestrians, bicycles, other vehicles (preceding vehicles, following vehicles, parallel vehicles, parked vehicles, etc.), roadside structures, obstacles, landmarks, and the like. Object information 222 indicates the relative position and relative speed of object OBJ with respect to vehicle 1. For example, by analyzing the image 221 obtained by the camera C, it is possible to recognize and identify the object OBJ and calculate the relative position of the object OBJ. For example, the control device 100 recognizes and identifies the object OBJ in the image 221 using image recognition AI obtained through machine learning. Furthermore, it is also possible to recognize the object OBJ and obtain the relative position and relative velocity of the object OBJ based on the point cloud information obtained by LIDAR.

The position information 230 is information indicating the current position of the vehicle 1. The control device 100 acquires position information 230 from the detection result by the position sensor 30. Further, the control device 100 may acquire highly accurate position information 230 through well-known self-position estimation processing (localization) using the object information 222 and the map information 210.

The vehicle status information 240 is information indicating the status of the vehicle 1, and includes vehicle speed, acceleration, yaw rate, steering angle, and the like. Control device 100 acquires vehicle state information 240 from vehicle state sensor 40 .

3-3. Vehicle Travel Control The control device 100 executes “vehicle travel control” that controls the travel of the vehicle 1. Vehicle travel control includes steering control, acceleration control, and deceleration control. Control device 100 executes vehicle travel control by controlling travel device 50 . Specifically, the control device 100 executes steering control by controlling a steering device. Further, the control device 100 executes acceleration control by controlling the drive device. Furthermore, the control device 100 executes deceleration control by controlling a braking device.

3-4. Automatic Driving Control The control device 100 also performs automatic driving control based on the driving environment information 200. More specifically, control device 100 generates a driving plan for vehicle 1 based on driving environment information 200. Examples of the driving plan include maintaining the current driving lane, changing lanes, turning left and right, and avoiding obstacles. Further, the control device 100 generates a target trajectory necessary for the vehicle 1 to travel according to the travel plan based on the driving environment information 200. The target trajectory includes a target position and a target velocity. Then, the control device 100 performs vehicle travel control so that the vehicle 1 follows the target trajectory.

3-5. Dynamic Setting of Image Processing Frequency In automatic driving control, the results of image processing on the image 221 captured by camera C are used. For example, by analyzing the image 221, the object OBJ around the vehicle 1 is recognized, and the above-mentioned object information 222 is acquired. Then, based on the object information 222, a travel plan and a target trajectory for automatic driving of the vehicle 1 are generated. Control device 100 dynamically sets execution frequency FR of image processing in automatic driving control according to the surrounding situation of vehicle 1.

FIG. 8 is a flowchart showing processing related to dynamic setting of image processing frequency according to the present embodiment.

In step S110, the control device 100 acquires the above-mentioned object information 222 based on the recognition result by the recognition sensor 20. Then, the control device 100 obtains a risk level RL representing the degree of risk to the vehicle 1 based on the object information 222. An example of setting the risk level RL is as described in Section 2 above.

In step S120, the control device 100 dynamically sets the image processing execution frequency FR according to the risk level RL. More specifically, the control device 100 sets the image processing execution frequency FR when the risk level RL is relatively low to be lower than the image processing execution frequency FR when the risk level RL is relatively high (FIG. , see Figure 3). For example, the execution frequency FR of image processing is set to once every 0.1 seconds, once every second, etc.

1 Vehicle 10 Vehicle control system 20 Recognition sensor 100 Control device 110 Processor 120 Storage device 200 Driving environment information 220 Surrounding situation information 221 Image 222 Object information C Camera OBJ Object RL Risk level RNG Risk judgment range RNG-A First risk judgment range RNG -B Second risk judgment range

Claims (1)

A mobile body control method for controlling a mobile body equipped with a camera, the method comprising:
acquiring an image captured by the camera, and controlling automatic operation of the mobile body based on a result of image processing on the image;
The mobile body during automatic driving is determined based on at least one of the number of objects within the risk determination range around the mobile body and the distance between the mobile body and the objects within the risk determination range. obtaining a risk level representing the degree of risk to
A mobile object control method, comprising: setting the frequency of the image processing when the risk level is low to be lower than the frequency of the image processing when the risk level is high.

Images