WO2022059489A1

WO2022059489A1 - Information processing device, information processing method, and program

Info

Publication number: WO2022059489A1
Application number: PCT/JP2021/032172
Authority: WO
Inventors: 幹夫中井
Original assignee: ソニーグループ株式会社
Priority date: 2020-09-15
Filing date: 2021-09-01
Publication date: 2022-03-24
Also published as: JP2023159898A

Abstract

[Problem] To suppress a reduction in precision when measuring a self-position, and to improve precision. [Solution] An information processing device comprises a processor for controlling a moving body. The processor estimates the self-position on the basis of information acquired from a sensor, calculates the precision of the self-position, updates a parameter based on an algorithm that estimates the self-position when the precision is reduced, and creates a behavior plan on the basis of the updated parameter.

Description

Information processing equipment, information processing methods and programs

This disclosure relates to information processing devices, information processing methods and programs.

When trying to execute various processes using a moving object that moves autonomously, it is necessary to execute self-position estimation. For self-position estimation, a star reckoning method for estimating the absolute self-position and a dead reckoning method for estimating the relative self-position are used. As self-position estimation for autonomous movement, low-speed star reckoning is performed in a predetermined area, and based on this absolute position, relative self-position estimation is performed while moving by dead reckoning that can be executed at high speed. Is commonly done.

Since dead reckoning depends on past results, there is a problem that if an error occurs in the estimation, this error will be accumulated. In order to avoid this, after moving at a predetermined timing, for example, a predetermined distance, move to an area where star reckoning can be performed, perform star reckoning, and re-estimate the self-position at an absolute position. Resets the accumulation of errors due to dead reckoning.

However, the timing at which Star Reckoning can be used is often limited depending on the surrounding environment and the sensors and algorithms used. In such a case, it is necessary to move the moving body to a predetermined position at a predetermined timing, or to change the behavior such as interrupting the work being performed and moving for this movement. .. Further, even in the path for returning to a predetermined position, since the path length is not constant, there is a problem that the path must be returned with a considerable margin. In addition, the self-position estimation accuracy may decrease due to various causes regardless of dead reckoning. Even in such a case, it is necessary to change the behavior in order to solve the decrease in the self-position estimation accuracy as described above.

Japanese Unexamined Patent Publication No. 2008-59218

Therefore, the present disclosure provides an information processing device, an information processing method, and a program that suppresses a decrease in the accuracy of self-position estimation and improves the accuracy.

According to one embodiment, the information processing device includes a processor for controlling a mobile body. The processor calculates the accuracy of self-position estimation based on information obtained from one or more sensors, and if the accuracy is reduced or is likely to be reduced, at least one of the cases. , Update the parameters based on the algorithm that performs the self-position estimation, and create an action plan based on the updated parameters.

The processor may control the moving body based on the created action plan.

The processor may estimate the self-position using a self-position estimation algorithm set for each sensor.

The processor may estimate the self-position using one algorithm based on the outputs from the plurality of sensors.

The processor may update the parameters based on the sensor or algorithm used to estimate the self-position.

The processor may calculate the result of self-position estimation by the sensor or the algorithm, and calculate the accuracy.

The processor may calculate the accuracy using one algorithm based on the outputs from the plurality of sensors.

The processor may acquire the peripheral situation of the moving body based on the information acquired from the sensor, and may update the parameter based on the acquired peripheral situation.

A memory, which stores information regarding the update of the parameter, may be further provided. The processor may update the parameters based on the information stored in the memory.

The memory may store a table in which information relating to the sensor used for estimating the self-position, the type of algorithm, the parameter to be updated, the condition, and the update data is described. , The processor may update the parameters with reference to the table.

The processor may update the parameters for the algorithm that has or is likely to have reduced accuracy.

The processor may update the parameter so as to select a route that suppresses the decrease in accuracy or a route that improves the accuracy.

The parameter to be updated may be a cost map.

According to one embodiment, the information processing method is a method of controlling the information processing apparatus described above.

According to one embodiment, the program causes the processor to execute the process described above.

The block diagram of the information processing apparatus which concerns on one Embodiment. The flowchart which shows the processing of the information processing apparatus which concerns on one Embodiment. The flowchart which shows the processing of the information processing apparatus which concerns on one Embodiment. The figure which shows an example of the update table which concerns on one Embodiment. The figure which shows an example of the cost map which concerns on one Embodiment. The figure which shows an example of the cost map which concerns on one Embodiment. It is a block diagram which shows an example of the schematic structure of a vehicle control system. It is explanatory drawing which shows an example of the installation position of the vehicle exterior information detection unit and the image pickup unit.

Hereinafter, embodiments in the present disclosure will be described with reference to the drawings. The drawings are for illustration purposes only, and the shape, size, or size ratio of each part configuration to other configurations in an actual device need not be as shown in the figure. In addition, since the drawings are written in a simplified form, it is assumed that configurations necessary for mounting other than those shown in the drawings are appropriately prepared. It should be noted that the following and above expressions when comparing numerical values can be appropriately rewritten as less than and greater than, respectively.

FIG. 1 is a block diagram showing a configuration of an information processing device according to an embodiment. The information processing device 1 is a device that estimates the self-position and the self-position accuracy based on the information acquired from one or a plurality of sensors 2, and controls the movement of the moving body based on the estimation results. The information processing device 1 includes a storage unit 100, a self-position estimation unit 102, a self-position accuracy estimation unit 104, a recognizer 106, a parameter update unit 108, a change table holding unit 110, an action planning unit 112, and the like. It includes a control unit 114. The information processing device 1 may be mounted on a moving body together with the sensor 2, for example. Further, as another example, the information processing apparatus 1 may be provided in a place different from the mobile body on which the sensor 2 is mounted via a communication path such as a network.

The storage unit 100 stores data and the like required for the operation of the information processing device 1. In addition to input information and output information, data such as the progress of calculation may be temporarily stored. When the information processing apparatus 1 specifically executes information processing by software using hardware resources, a program or the like related to the software may be stored. The change table holding unit 110, which will be described later, may be a part of the storage unit 100, and in this case, the change table is stored in the storage unit 100.

A part or all the components of the information processing apparatus 1 are executed by a processor (processing circuit) such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit) that are operated by data stored in the storage unit 100. Alternatively, it may be implemented by a dedicated analog circuit or digital circuit, for example, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like.

The self-position estimation unit 102 executes self-position estimation based on the information acquired by the sensor 2. This self-position estimation may be, for example, star reckoning and dead reckoning, or a combination of these methods may be appropriately used according to the situation. For example, the self-position estimation unit 102 may execute a plurality of self-position estimation algorithms based on information acquired from a plurality of sensors and appropriately fuse them.

As star recording, GPS (Global Positioning System), image feature point map, NDT (Normal Distributions Transform) matching of LiDAR (Light Detection and Ringing), etc. may be used. Further, as the dead reckoning, vehicle odometry, visual SLAM (Simultaneous Localization and Mapping), depth SLAM, IMU (Inertial Measurement Unit) and the like may be used. It is necessary to have the corresponding sensor 2 used for each method, but any of these may be used, and the form in the present disclosure is not limited to the above example.

For example, the self-position estimation unit 102 may estimate the self-position by using an algorithm set for each sensor based on the information acquired from one or a plurality of sensors 2. In this case, the position estimation results based on the information acquired from each sensor 2 may be fused by an appropriate algorithm.

For example, the self-position estimation unit 102 may estimate the self-position by using one algorithm for the information acquired from the plurality of sensors 2.

The self-position accuracy estimation unit 104 calculates the self-position accuracy estimation unit 104 estimated by the self-position accuracy estimation unit 102. For the calculation of this accuracy, for example, a general method using a variance-covariance matrix or the like may be used. The accuracy of self-position estimation may differ depending on the method used to perform self-position estimation. In this case, the accuracy of self-position estimation suitable for the method may be calculated.

As another example, the self-position accuracy estimation unit 104 may detect a case where the accuracy of the self-position estimation is expected to decrease in the near future based on the result of the current self-position estimation. A situation in which the self-position accuracy is expected to decrease may be detected from the results of self-position estimation several times in the past, for example. Further, as another example, it may be detected based on statistical information, or it may be detected by inputting the result of past self-position estimation into a trained model by machine learning.

For example, when the self-position estimation method is LiDAR odometry, the accuracy can be calculated from the total mileage, matching rate, etc., or it can be detected that the accuracy is likely to decrease. Similarly, the part described as the calculation of accuracy may be rephrased as detecting the possibility that the accuracy will decrease. In the case of a wheel encoder, the accuracy can be calculated using the total mileage. In the case of an IMU, the accuracy may be calculated using the total absolute number of revolutions.

For example, when the self-position estimation method is star reckoning, the covariance by the above dead reckoning can be obtained as a numerical value, and the accuracy may be calculated using this covariance. In the case of scan matching by LiDAR, visual SLAM, etc., the accuracy may be calculated using the mileage and the mileage since the last matching, or the matching rate may be used. In the case of GPS, it is also possible to calculate the decrease in accuracy using DOP (Dilution of Precision).

In this way, the accuracy calculation method may change depending on the method adopted by the self-position estimation unit 102. The self-position accuracy estimation unit 104 calculates the self-position accuracy based on the method used by the self-position estimation unit 102.

The self-position accuracy estimation unit 104 may calculate the accuracy of self-position estimation for each sensor or for each algorithm.

Further, the self-position estimation unit 102 calculates the self-position estimation result for each sensor or algorithm, and the self-position accuracy estimation unit 104 calculates the accuracy of the result for each sensor or algorithm. May be calculated respectively.

When a plurality of self-position estimation algorithms are used, the self-position accuracy estimation unit 104 may calculate the final self-position accuracy fused. As another example, the self-position accuracy estimation unit 104 may calculate the accuracy of the self-position estimated by different algorithms. In this case, it may be determined whether or not to update the parameters for each algorithm, or whether or not to update the parameters as a result of the fusion may be determined. When the update is determined as a result of the fusion, the parameters for some of the algorithms may be selected and updated.

In the drawing, the self-position estimation unit 102 and the self-position accuracy estimation unit 104 are separate components, but these may be one component in some cases. For example, when a method of calculating self-position estimation and self-position accuracy together is used, these components execute self-position estimation and accuracy calculation as one component.

The recognizer 106 executes various recognition processes based on the output from the sensor 2. This recognition process executes recognition processes other than the recognition used for self-position estimation, such as detection of a person and detection of an obstacle. For this recognition, for example, any feature amount such as HoG (Histograms of Oriented Gradient), SIFT (Scaled Invariance Feature Transform), optical flow, etc. may be used. The recognition and detection are not limited to those using these features, and recognition may be performed using a recognition model trained in advance by machine learning. In this way, the recognizer 106 acquires information and a situation regarding the environment around the moving body based on the information acquired from the sensor.

The parameter update unit 108 performs autonomous movement using the table held in the change table holding unit 110 based on the self-position accuracy estimated by the self-position accuracy estimation unit 104 and the result recognized by the recognizer 106. Update the parameters to do.

The change table holding unit 110 stores, for example, an action plan parameter change table for each algorithm and sensor. The action plan parameter change table is a table that stores the parameters to be changed and the changed values based on the conditions by the algorithm and the sensor.

For example, when the action plan is determined by a method such as A * or Dijkstra, how to give the score for each grid, each route, etc. to the change table holding unit 110 according to the self-position accuracy and the condition of the recognition result. Whether to change is stored. By referring to this table, the parameter update unit 108 updates the score for, for example, the grid on the map, each area, and the like.

The action plan unit 112 generates an action plan based on the parameters updated by the parameter update unit 108. For example, the above-mentioned methods such as A * and Dijkstra's algorithm are used for determining this action plan, but the determination is not limited to this, and other methods may be used. For example, the action planning unit 112 updates the parameters for moving without changing the action performed by the moving body. By determining the action plan in this way, it becomes possible to appropriately move the moving body while performing the action carried out by the moving body.

The action planning unit 112 can, for example, create an action plan that moves along a route that does not reduce the accuracy of self-position estimation while performing the action currently being executed. In addition, as another example, the action planning unit 112 moves to a place suitable for star reckoning so as not to reduce the current accuracy in order to recover the accuracy of self-position estimation while performing the action currently being executed. You may decide on an action plan.

If the algorithm of the action plan is different from the above, of course, the parameters updated by the parameter update unit 108 and the contents of the table held in the change table holding unit 110 will be different, but the action plan is appropriately applied to the algorithm. Any configuration that can be determined is sufficient.

The control unit 114 controls the moving body based on the action plan generated by the action planning unit 112.

In the above, the sensor 2 is mounted on a moving body, but the sensor 2 is not limited to this. For example, it may be a fixed sensor that monitors an area where the moving body can move.

FIG. 2 is a flowchart showing the overall processing of the information processing apparatus 1 according to the embodiment.

First, the information processing apparatus 1 estimates the self-position by the self-position estimation unit 102 based on the information acquired from the sensor 2, and calculates the accuracy of this self-position by the self-position accuracy estimation unit 104 (S10). As described above, depending on the type of sensor and the algorithm used, it is possible to execute the self-position estimation and the accuracy calculation at the same time. In this case, the self-position estimation and the accuracy calculation may be executed at the same timing. ..

Next, the information processing apparatus 1 operates to suppress the error of the self-position accuracy by the parameter update unit 108, or the error of the self-position accuracy, in response to the action in which the accuracy of the self-position is lowered in the accuracy of the self-position. Update the parameters based on the behavior to improve (S20). Depending on the type of sensor and the algorithm used, there is an operation of suppressing the error of self-position estimation or an operation of reducing the error. The parameter update unit 108 may select and output this operation based on the accuracy of its own position.

FIG. 3 is a flowchart showing the update process of this action plan parameter more concretely.

In the action plan parameter update process, first, the parameter update unit 108 acquires the self-position accuracy calculated by the self-position accuracy estimation unit 104 (S200).

Next, the parameter update unit 108 acquires data related to the parameters of the action plan based on the sensors and algorithms that cause the accuracy to deteriorate (S202). For example, the parameter update unit 108 acquires data such as parameters to be updated, conditions to be updated, and values to be updated by referring to the parameter change table. This update may use, for example, the results of recognizer 106. As an example of this update, there may be the following.

For example, when using a wheel encoder, acquire a parameter that increases the cost for the total travel distance.

For example, when using an IMU, acquire a parameter that reduces the rotation of the moving object in the same direction as much as possible. This has the effect of suppressing the influence of the calibration deviation of Scale Factor.

For example, when using LiDAR matching, a parameter that increases the cost around a person is acquired from the result of human recognition. This is because moving objects affect recognition accuracy. You may also acquire parameters that reduce costs near walls and obstacles. This is because the accuracy of matching can be improved when it is near a wall or an obstacle that does not move.

For example, when using a GPU, the cost of roofs, tunnels, near buildings, etc. will be increased based on the map and recognition results. This is to make it easier to acquire radio waves from the satellite used for GPS.

The parameter update unit 108 acquires the data related to the parameter update by referring to the table in which the parameter update as described above is described.

FIG. 4 is a diagram showing an example of a change table held in the change table holding unit 110 according to the embodiment. In FIG. 4, as an example, a part of the parameter update related to LiDAR scan matching is extracted and shown.

The change table is, for example, a table including sensor algorithms, reflection parameters, conditions, and update data. If the sensor algorithm is LiDAR scan matching, the target to reflect the parameter update is the cost map. As a condition, for example, there may be a distance to a person or a distance to a wall. Data that reflects the parameter update is described in the update data. In the processing of S202, the update target parameters and update data that match the conditions are acquired from this conversion table.

As shown in Fig. 4, a table was prepared, but it is not limited to this. For example, a file storing different conversion data may be prepared for each algorithm or the like, and parameter update data may be acquired based on the contents of the file. As another example, it may be hard-coded in the program or coded in the circuit. Further, for example, it is described in a DB (Database) provided in the cloud, and may be based on this description. This DB may be updated appropriately by the administrator or automatically by a trained model or the like at an appropriate timing.

Returning to FIG. 3, the information processing apparatus 1 updates the parameters by the parameter update unit 108 (S204). For example, consider the case where the sensor algorithm used for self-position estimation is LiDAR scan matching, and the accuracy of this LiDAR scan matching is reduced.

It is assumed that there is a person in the recognition result of the recognizer 106. When using the table of FIG. 4, the parameter update unit 108 doubles the cost within 0.5 m around the person and doubles the cost in the range of 0.5 m to 1.0 m in the cost map. If the recognizer 106 determines that there is a wall, the parameter update unit 108 updates the cost map within 1 m from the wall to 1/2. In this way, the parameter update unit 108 updates the cost map.

When the decrease in self-position accuracy is recognized in this way, the parameter update unit 108 updates the parameters for executing the self-position estimation based on the information stored in the change table.

For example, the self-position accuracy estimation unit 104 may output the self-position accuracy as a numerical value, and the parameter update unit 108 may execute the above parameter update when this numerical value falls below a predetermined threshold value. Further, when the accuracy is higher as it is closer to 0, the parameter update may be executed when the predetermined threshold value is exceeded. In this case, as described above, the recognition result of the recognizer 106 may be used, or depending on the combination of the sensor and the algorithm, the update may be executed without using the recognition result of the recognizer 106.

When multiple sensors and algorithms are being executed in parallel, judgment based on self-position accuracy may be executed for each sensor and algorithm. In this case, the parameter update unit 108 may update the parameters by extracting only the data related to the sensor and the algorithm whose deterioration is recognized from the change table. By executing in this way, it is possible to realize a sensor having a deteriorated self-position accuracy, suppressing a decrease in self-position estimation by an algorithm, or improving the accuracy.

As another example, the parameter update unit 108 may update parameters related to other sensors and algorithms that are being executed in parallel when there is a sensor or algorithm that is found to be degraded. By executing in this way, parameter update may be performed for the sensor or algorithm whose self-position accuracy has deteriorated, and the self-position accuracy of other sensors or algorithms may be improved in order to interpolate this decrease. As a result, the deterioration of the self-positioning accuracy as a whole may be suppressed or reduced.

Returning to FIG. 2, the information processing apparatus 1 then creates an action plan by the action planning unit 112 based on the parameters updated by the parameter updating unit 108 (S30). When the parameter update unit 108 updates the cost of the cost map as in the above example, an action plan is created using an appropriate algorithm such as A * or Dijkstra using this updated cost map.

After this, the control unit 114 controls the moving body based on this action plan.

An example of updating parameters and creating an action plan when using a cost map will be explained.

Figure 5 is a diagram showing the action plan before updating the parameters. The route that the moving body moves while working in the passage sandwiched between the walls is shown. Costs are shown in shaded areas, with diagonal lines in the same direction and spacing mean the same cost. The wall may have a cost of ∞ or a value sufficiently larger than the other costs, for example, 100. .. For example, the center of the passage is set at cost 1, and the cost is set at 2, 4 as it approaches the wall. In such cases, the moving object's action plan goes straight and turns right toward the target point, as indicated by the arrow.

FIG. 6 is a diagram showing an action plan when there is a parameter update. For example, FIG. 6 shows a cost map that is updated when a decrease in the accuracy of self-position estimation is observed. In this case, the parameter update unit 108 updates the cost map from the state of FIG. 5 to the state of FIG. 6 based on the change table from the recognition result of the person on the passage and the recognition result of the wall. Based on the change table shown in Figure 4, the cost around the person is increased and the cost around the wall is updated low. As a result, it is possible to avoid a route in which the self-positioning accuracy is lowered by approaching the periphery of a person, or to select a route in which the self-positioning accuracy is easily restored by approaching the periphery of a wall. As a result, the information processing apparatus 1 can select a route that suppresses a decrease in self-position accuracy or a route that is expected to recover self-position accuracy by updating the parameters.

The action plan department 112 recreates the action plan based on this cost map. Then, the action planning unit 112 creates, for example, an action plan shown by a solid line with respect to the original action plan shown by the dotted line. For example, as shown in FIG. 6, when the target point is determined, the action planning unit 112 may create a new action plan as a route toward the target point. If you are moving freely, create an action plan to move appropriately based on the surrounding environment, rather than aiming for a specific target point.

As described above, according to this embodiment, it is possible to create an action plan in consideration of the accuracy of self-position estimation. In creating this action plan, it is possible to appropriately modify only the movement route without interrupting the work currently being performed by the moving body. For example, when a robot distributes something at the venue, it is possible to create an action plan appropriately without interrupting the distribution work and without degrading the accuracy of self-position estimation. In the case of automatic driving of a car, it is possible to appropriately modify an action plan such as a route while driving without changing the purpose of driving to the destination.

In this way, in an environment where the accuracy of self-position estimation can be reduced, it is possible to suppress the decrease in accuracy or restore the accuracy without changing the target behavior itself. As a result, it becomes possible to realize more Robustona self-position estimation. Furthermore, this eliminates the need for processing such as switching the robot's behavior with respect to the decrease in accuracy, so that it is possible to improve the robot's operating efficiency and the like, and also to improve the user interaction and the like.

In the above, the explanation was given in the self-position estimation for matching, but the present invention is not limited to this, and can be applied to updating the parameters for setting the action plan in various self-position estimations. In addition, although the form using the cost map has been described, this is also not limited to the cost map, and can be applied to the update of parameters for creating various forms of action plans.

The technology related to this disclosure can be applied to various products. For example, the technology according to the present disclosure is any kind of movement such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility, an airplane, a drone, a ship, a robot, a construction machine, and an agricultural machine (tractor). It may be realized as a device mounted on the body.

FIG. 7 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile control system to which the technique according to the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. In the example shown in FIG. 7, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside information detection unit 7400, an in-vehicle information detection unit 7500, and an integrated control unit 7600. .. The communication network 7010 connecting these multiple control units conforms to any standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network) or FlexRay (registered trademark). It may be an in-vehicle communication network.

Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a storage unit that stores programs executed by the microcomputer or parameters used for various arithmetic, and a drive circuit that drives various controlled devices. To prepare for. Each control unit is provided with a network I / F for communicating with other control units via the communication network 7010, and is connected to devices or sensors inside or outside the vehicle by wired communication or wireless communication. A communication I / F for performing communication is provided. In FIG. 7, as the functional configuration of the integrated control unit 7600, the microcomputer 7610, the general-purpose communication I / F7620, the dedicated communication I / F7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F7660, the audio image output unit 7670, The vehicle-mounted network I / F 7680 and the storage unit 7690 are illustrated. Other control units also include a microcomputer, a communication I / F, a storage unit, and the like.

The drive system control unit 7100 controls the operation of the device related to the drive system of the vehicle according to various programs. For example, the drive system control unit 7100 has a driving force generator for generating the driving force of the vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to the wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism for adjusting and a braking device for generating braking force of the vehicle. The drive system control unit 7100 may have a function as a control device such as ABS (Antilock Brake System) or ESC (Electronic Stability Control).

The vehicle state detection unit 7110 is connected to the drive system control unit 7100. The vehicle state detection unit 7110 may include, for example, a gyro sensor that detects the angular velocity of the axial rotation motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, an accelerator pedal operation amount, a brake pedal operation amount, or steering wheel steering. It includes at least one of sensors for detecting an angle, engine speed, wheel speed, and the like. The drive system control unit 7100 performs arithmetic processing using a signal input from the vehicle state detection unit 7110, and controls an internal combustion engine, a drive motor, an electric power steering device, a brake device, and the like.

The body system control unit 7200 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 7200 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as headlamps, back lamps, brake lamps, turn signals or fog lamps. In this case, a radio wave transmitted from a portable device that substitutes for a key or signals of various switches may be input to the body system control unit 7200. The body system control unit 7200 receives inputs of these radio waves or signals and controls a vehicle door lock device, a power window device, a lamp, and the like.

The battery control unit 7300 controls the secondary battery 7310, which is the power supply source of the drive motor, according to various programs. For example, information such as the battery temperature, the battery output voltage, or the remaining capacity of the battery is input to the battery control unit 7300 from the battery device including the secondary battery 7310. The battery control unit 7300 performs arithmetic processing using these signals, and controls the temperature control of the secondary battery 7310 or the cooling device provided in the battery device.

The vehicle outside information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000. For example, at least one of the image pickup unit 7410 and the vehicle exterior information detection unit 7420 is connected to the vehicle exterior information detection unit 7400. The image pickup unit 7410 includes at least one of a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The vehicle outside information detection unit 7420 is used, for example, to detect the current weather or an environment sensor for detecting the weather, or other vehicles, obstacles, pedestrians, etc. around the vehicle equipped with the vehicle control system 7000. At least one of the ambient information detection sensors is included.

The environment sensor may be, for example, at least one of a raindrop sensor that detects rainy weather, a fog sensor that detects fog, a sunshine sensor that detects the degree of sunshine, and a snow sensor that detects snowfall. The ambient information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device. The image pickup unit 7410 and the vehicle exterior information detection unit 7420 may be provided as independent sensors or devices, or may be provided as a device in which a plurality of sensors or devices are integrated.

Here, FIG. 8 shows an example of the installation position of the image pickup unit 7410 and the vehicle exterior information detection unit 7420. The

image pickup unit

7910, 7912, 7914, 7916, 7918 are provided, for example, at at least one of the front nose, side mirror, rear bumper, back door, and upper part of the windshield of the vehicle interior of the vehicle 7900. The image pickup unit 7910 provided in the front nose and the image pickup section 7918 provided in the upper part of the windshield in the vehicle interior mainly acquire an image in front of the vehicle 7900. The

image pickup units

7912 and 7914 provided in the side mirrors mainly acquire images of the side of the vehicle 7900. The image pickup unit 7916 provided in the rear bumper or the back door mainly acquires an image of the rear of the vehicle 7900. The image pickup unit 7918 provided on the upper part of the windshield in the vehicle interior is mainly used for detecting a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

Note that FIG. 8 shows an example of the shooting range of each of the

imaging units

7910, 7912, 7914, 7916. The imaging range a indicates the imaging range of the imaging unit 7910 provided on the front nose, the imaging ranges b and c indicate the imaging range of the

imaging units

7912 and 7914 provided on the side mirrors, respectively, and the imaging range d indicates the imaging range d. The imaging range of the imaging unit 7916 provided on the rear bumper or the back door is shown. For example, by superimposing the image data captured by the

image pickup units

7910, 7912, 7914, 7916, a bird's-eye view image of the vehicle 7900 can be obtained.

The vehicle exterior

information detection unit

7920, 7922, 7924, 7926, 7928, 7930 provided at the front, rear, side, corner and the upper part of the windshield of the vehicle interior of the vehicle 7900 may be, for example, an ultrasonic sensor or a radar device. The vehicle exterior

information detection units

7920, 7926, 7930 provided on the front nose, rear bumper, back door, and upper part of the windshield in the vehicle interior of the vehicle 7900 may be, for example, a lidar device. These out-of-vehicle information detection units 7920 to 7930 are mainly used for detecting a preceding vehicle, a pedestrian, an obstacle, or the like.

Return to Fig. 7 and continue the explanation. The vehicle outside information detection unit 7400 causes the image pickup unit 7410 to capture an image of the outside of the vehicle and receives the captured image data. Further, the vehicle exterior information detection unit 7400 receives detection information from the connected vehicle exterior information detection unit 7420. When the vehicle exterior information detection unit 7420 is an ultrasonic sensor, a radar device, or a lidar device, the vehicle exterior information detection unit 7400 transmits ultrasonic waves, electromagnetic waves, or the like, and receives received reflected wave information. The out-of-vehicle information detection unit 7400 may perform object detection processing or distance detection processing such as a person, a vehicle, an obstacle, a sign, or a character on a road surface based on the received information. The out-of-vehicle information detection unit 7400 may perform an environment recognition process for recognizing rainfall, fog, road surface conditions, etc. based on the received information. The out-of-vehicle information detection unit 7400 may calculate the distance to an object outside the vehicle based on the received information.

Further, the vehicle outside information detection unit 7400 may perform image recognition processing or distance detection processing for recognizing a person, a vehicle, an obstacle, a sign, a character on the road surface, or the like based on the received image data. The vehicle exterior information detection unit 7400 performs processing such as distortion correction or alignment on the received image data, and synthesizes image data captured by different image pickup units 7410 to generate a bird's-eye view image or a panoramic image. May be good. The vehicle exterior information detection unit 7400 may perform the viewpoint conversion process using the image data captured by different image pickup units 7410.

The in-vehicle information detection unit 7500 detects the in-vehicle information. For example, a driver state detection unit 7510 that detects the state of the driver is connected to the in-vehicle information detection unit 7500. The driver state detection unit 7510 may include a camera that captures the driver, a biosensor that detects the driver's biological information, a microphone that collects sound in the vehicle interior, and the like. The biosensor is provided on, for example, on the seat surface or the steering wheel, and detects the biometric information of the passenger sitting on the seat or the driver holding the steering wheel. The in-vehicle information detection unit 7500 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 7510, and may determine whether the driver is asleep. You may. The in-vehicle information detection unit 7500 may perform processing such as noise canceling processing on the collected audio signal.

The integrated control unit 7600 controls the overall operation in the vehicle control system 7000 according to various programs. An input unit 7800 is connected to the integrated control unit 7600. The input unit 7800 is realized by a device that can be input-operated by the occupant, such as a touch panel, a button, a microphone, a switch, or a lever. Data obtained by recognizing the voice input by the microphone may be input to the integrated control unit 7600. The input unit 7800 may be, for example, a remote control device using infrared rays or other radio waves, or an external connection device such as a mobile phone or a PDA (Personal Digital Assistant) corresponding to the operation of the vehicle control system 7000. You may. The input unit 7800 may be, for example, a camera, in which case the passenger can input information by gesture. Alternatively, data obtained by detecting the movement of the wearable device worn by the passenger may be input. Further, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on the information input by the passenger or the like using the above input unit 7800 and outputs the input signal to the integrated control unit 7600. By operating the input unit 7800, the passenger or the like inputs various data to the vehicle control system 7000 and instructs the processing operation.

The storage unit 7690 may include a ROM (Read Only Memory) for storing various programs executed by the microcomputer, and a RAM (Random Access Memory) for storing various parameters, calculation results, sensor values, and the like. Further, the storage unit 7690 may be realized by a magnetic storage device such as an HDD (Hard Disc Drive), a semiconductor storage device, an optical storage device, an optical magnetic storage device, or the like.

The general-purpose communication I / F 7620 is a general-purpose communication I / F that mediates communication with various devices existing in the external environment 7750. General-purpose communication I / F7620 is a cellular communication protocol such as GSM (registered trademark) (Global System of Mobile communications), WiMAX (registered trademark), LTE (registered trademark) (Long Term Evolution) or LTE-A (LTE-Advanced). , Or other wireless communication protocols such as wireless LAN (also referred to as Wi-Fi®), Bluetooth® may be implemented. The general-purpose communication I / F7620 connects to a device (for example, an application server or a control server) existing on an external network (for example, the Internet, a cloud network, or a business-specific network) via a base station or an access point, for example. You may. Further, the general-purpose communication I / F7620 uses, for example, P2P (Peer To Peer) technology, and is a terminal existing in the vicinity of the vehicle (for example, a driver, a pedestrian or a store terminal, or an MTC (Machine Type Communication) terminal). May be connected with.

The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol formulated for use in a vehicle. The dedicated communication I / F7630 uses a standard protocol such as WAVE (Wireless Access in Vehicle Environment), DSRC (Dedicated Short Range Communications), or cellular communication protocol, which is a combination of IEEE802.11p in the lower layer and IEEE1609 in the upper layer. May be implemented. Dedicated communication I / F7630 is typically vehicle-to-vehicle (Vehicle to Vehicle) communication, road-to-vehicle (Vehicle to Infrastructure) communication, vehicle-to-house (Vehicle to Home) communication, and pedestrian-to-vehicle (Vehicle to Pedestrian) communication. ) Carry out V2X communication, a concept that includes one or more of the communications.

The positioning unit 7640 receives, for example, a GNSS signal from a GNSS (Global Navigation Satellite System) satellite (for example, a GPS signal from a GPS (Global Positioning System) satellite), executes positioning, and executes positioning, and the latitude, longitude, and altitude of the vehicle. Generate location information including. The positioning unit 7640 may specify the current position by exchanging signals with the wireless access point, or may acquire position information from a terminal such as a mobile phone, PHS, or smartphone having a positioning function.

The beacon receiving unit 7650 receives, for example, a radio wave or an electromagnetic wave transmitted from a radio station or the like installed on a road, and acquires information such as a current position, a traffic jam, a road closure, or a required time. The function of the beacon receiving unit 7650 may be included in the above-mentioned dedicated communication I / F 7630.

The in-vehicle device I / F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various in-vehicle devices 7760 existing in the vehicle. The in-vehicle device I / F7660 may establish a wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication) or WUSB (Wireless USB). In addition, the in-vehicle device I / F7660 is via a connection terminal (and a cable if necessary) (not shown), USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface, or MHL (Mobile High)). -Definition Link) and other wired connections may be established. The in-vehicle device 7760 includes, for example, at least one of a passenger's mobile device or wearable device, or an information device carried in or attached to the vehicle. Further, the in-vehicle device 7760 may include a navigation device for searching a route to an arbitrary destination. The in-vehicle device I / F 7660 may be a control signal to and from these in-vehicle devices 7760. Or exchange the data signal.

The in-vehicle network I / F7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The vehicle-mounted network I / F7680 transmits / receives signals and the like according to a predetermined protocol supported by the communication network 7010.

The microcomputer 7610 of the integrated control unit 7600 is via at least one of general-purpose communication I / F7620, dedicated communication I / F7630, positioning unit 7640, beacon receiving unit 7650, in-vehicle device I / F7660, and in-vehicle network I / F7680. The vehicle control system 7000 is controlled according to various programs based on the information acquired. For example, the microcomputer 7610 calculates the control target value of the driving force generator, the steering mechanism, or the braking device based on the acquired information inside and outside the vehicle, and outputs a control command to the drive system control unit 7100. May be good. For example, the microcomputer 7610 realizes ADAS (Advanced Driver Assistance System) functions including vehicle collision avoidance or impact mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane deviation warning, and the like. Cooperative control may be performed for the purpose of. In addition, the microcomputer 7610 automatically travels autonomously without relying on the driver's operation by controlling the driving force generator, steering mechanism, braking device, etc. based on the acquired information on the surroundings of the vehicle. Coordinated control may be performed for the purpose of driving or the like.

The microcomputer 7610 has information acquired via at least one of a general-purpose communication I / F7620, a dedicated communication I / F7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F7660, and an in-vehicle network I / F7680. Based on the above, three-dimensional distance information between the vehicle and an object such as a surrounding structure or a person may be generated, and local map information including the peripheral information of the current position of the vehicle may be created. Further, the microcomputer 7610 may predict the danger of a vehicle collision, a pedestrian or the like approaching or entering a closed road, and generate a warning signal based on the acquired information. The warning signal may be, for example, a signal for generating a warning sound or lighting a warning lamp.

The audio image output unit 7670 transmits an output signal of at least one of audio and image to an output device capable of visually or audibly notifying information to the passenger or the outside of the vehicle. In the example of FIG. 7, an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are exemplified as output devices. The display unit 7720 may include, for example, at least one of an onboard display and a head-up display. The display unit 7720 may have an AR (Augmented Reality) display function. The output device may be other devices such as headphones, wearable devices such as eyeglass-type displays worn by passengers, projectors or lamps other than these devices. When the output device is a display device, the display device displays the results obtained by various processes performed by the microcomputer 7610 or the information received from other control units in various formats such as texts, images, tables, and graphs. Display visually. When the output device is an audio output device, the audio output device converts an audio signal composed of reproduced audio data, acoustic data, or the like into an analog signal and outputs the audio signal audibly.

In the example shown in FIG. 7, at least two control units connected via the communication network 7010 may be integrated as one control unit. Alternatively, each control unit may be composed of a plurality of control units. Further, the vehicle control system 7000 may include another control unit (not shown). Further, in the above description, the other control unit may have a part or all of the functions carried out by any of the control units. That is, as long as information is transmitted and received via the communication network 7010, predetermined arithmetic processing may be performed by any of the control units. Similarly, a sensor or device connected to any control unit may be connected to another control unit, and a plurality of control units may send and receive detection information to and from each other via the communication network 7010. ..

Note that a computer program for realizing each function of the information processing apparatus 1 according to the present embodiment described with reference to FIG. 1 can be implemented in any control unit or the like. It is also possible to provide a computer-readable recording medium in which such a computer program is stored. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed, for example, via a network without using a recording medium.

In the vehicle control system 7000 described above, the information processing device 1 according to the present embodiment described with reference to FIG. 1 can be applied to the integrated control unit 7600 of the application example shown in FIG. For example, each component of the information processing apparatus 1 corresponds to a microcomputer 7610, a storage unit 7690, and an in-vehicle network I / F 7680 of the integrated control unit 7600. For example, the integrated control unit 7600 can assist automatic driving and autonomous movement by generating a map and estimating a self-position.

Further, at least a part of the components of the information processing apparatus 1 described with reference to FIG. 1 are in the module for the integrated control unit 7600 shown in FIG. 7 (for example, an integrated circuit module composed of one die). It may be realized. Alternatively, the information processing apparatus 1 described with reference to FIG. 1 may be realized by a plurality of control units of the vehicle control system 7000 shown in FIG. 7.

The above-mentioned embodiment may be in the following form.

(1)
Equipped with a processor to control the mobile
The processor
Calculate the accuracy of self-position estimation based on the information obtained from one or more sensors,
When the accuracy is reduced, or at least one of the cases where the accuracy is expected to be reduced, the parameters based on the algorithm for performing the self-position estimation are updated.
Create an action plan based on the updated parameters,
Information processing equipment.

(2)
The processor
Control the moving body based on the created action plan,
The information processing device according to (1).

(3)
The processor
The self-position is estimated using the self-position estimation algorithm set for each sensor.
The information processing device according to (1) or (2).

(4)
The processor
Estimating the self-position using one algorithm based on the outputs from the plurality of sensors.
The information processing device according to any one of (1) to (2).

(5)
The processor
Update the parameters based on the sensor or algorithm used to estimate the self-position.
The information processing device according to any one of (1) to (4).

(6)
The processor
The result of self-position estimation is calculated by the sensor or the algorithm, and the accuracy is calculated.
The information processing device according to any one of (1) to (4).

(7)
The processor
One algorithm is used to calculate the accuracy based on the outputs from the plurality of sensors.
The information processing device according to any one of (1) to (6).

(8)
The processor
Based on the information acquired from the sensor, the surrounding condition of the moving body is acquired, and the surrounding condition is acquired.
The parameter is updated based on the acquired peripheral situation.
The information processing device according to any one of (1) to (7).

(9)
Further provided with a memory for storing information regarding the update of the parameters.
The processor
Update the parameter based on the information stored in the memory.
The information processing device according to any one of (1) to (8).

(10)
The memory is
A table in which information relating to the sensor and algorithm type used for estimating the self-position, the parameter to be updated, the condition, and the update data is described is stored.
The processor
Update the parameters with reference to the table.
The information processing device according to (9).

(11)
The processor
Updating the parameters for the algorithm that is or is likely to be less accurate.
The information processing apparatus according to (9) or (10).

(12)
The processor
The parameter is updated so as to select a route that suppresses the decrease in accuracy or a route that improves the accuracy.
The information processing apparatus according to any one of (9) to (11).

(13)
The parameter to be updated is a cost map,
The information processing apparatus according to any one of (1) to (12).

(14)
With a processor to control the mobile
Perform the method according to any one of (1) to (14),
Information processing method.

(15)
To the processor for controlling the moving body,
The method described in any of (1) to (14),
The program to be executed.

The aspect of the present disclosure is not limited to the above-mentioned embodiment, but also includes various possible modifications, and the effect of the present disclosure is not limited to the above-mentioned contents. The components in each embodiment may be applied in appropriate combinations. That is, various additions, changes and partial deletions are possible without departing from the conceptual idea and purpose of the present disclosure derived from the contents specified in the claims and their equivalents.

1: Information processing device,
100: Memory,
102: Self-position estimation unit,
104: Self-position accuracy estimation unit,
106: recognizer,
108: Parameter update section,
110: Change table holder,
112: Action Planning Department,
114: Control unit,
2: Sensor

Claims

Equipped with a processor to control the mobile
The processor
Calculate the accuracy of self-position estimation based on the information obtained from one or more sensors,
When the accuracy is reduced, or at least one of the cases where the accuracy is expected to be reduced, the parameters based on the algorithm for performing the self-position estimation are updated.
Create an action plan based on the updated parameters,
Information processing equipment.
The processor
Control the moving body based on the created action plan,
The information processing apparatus according to claim 1.
The processor
The self-position is estimated using the self-position estimation algorithm set for each sensor.
The information processing apparatus according to claim 1.
The processor
Estimating the self-position using one algorithm based on the outputs from the plurality of sensors.
The information processing apparatus according to claim 1.
The processor
Update the parameters based on the sensor or algorithm used to estimate the self-position.
The information processing apparatus according to claim 1.
The processor
The result of self-position estimation is calculated by the sensor or the algorithm, and the accuracy is calculated.
The information processing apparatus according to claim 1.
The processor
One algorithm is used to calculate the accuracy based on the outputs from the plurality of sensors.
The information processing apparatus according to claim 1.
The processor
Based on the information acquired from the sensor, the surrounding condition of the moving body is acquired, and the surrounding condition is acquired.
The parameter is updated based on the acquired peripheral situation.
The information processing apparatus according to claim 1.
Further provided with a memory for storing information regarding the update of the parameters.
The processor
Update the parameter based on the information stored in the memory.
The information processing apparatus according to claim 1.
The memory is
A table in which information relating to the sensor and algorithm type used for estimating the self-position, the parameter to be updated, the condition, and the update data is described is stored.
The processor
Update the parameters with reference to the table.
The information processing apparatus according to claim 9.
The processor
Updating the parameters for the algorithm that is or is likely to be less accurate.
The information processing apparatus according to claim 9.
The processor
The parameter is updated so as to select a route that suppresses the decrease in accuracy or a route that improves the accuracy.
The information processing apparatus according to claim 9.
The parameter to be updated is a cost map,
The information processing apparatus according to claim 1.
With a processor to control the mobile
Calculate the accuracy of self-position estimation based on the information obtained from one or more sensors,
When the accuracy is reduced, or at least one of the cases where the accuracy is expected to be reduced, the parameters based on the algorithm for performing the self-position estimation are updated.
Create an action plan based on the updated parameters,
Information processing method.
To the processor for controlling the moving body,
Calculate the accuracy of self-position estimation based on the information obtained from one or more sensors,
When the accuracy is reduced, or at least one of the cases where the accuracy is expected to be reduced, the parameters based on the algorithm for performing the self-position estimation are updated.
Create an action plan based on the updated parameters,
A program that lets you do things.