WO2021260910A1 - Ai integration system, ai integration device, and ai integration program - Google Patents

Ai integration system, ai integration device, and ai integration program Download PDF

Info

Publication number
WO2021260910A1
WO2021260910A1 PCT/JP2020/025175 JP2020025175W WO2021260910A1 WO 2021260910 A1 WO2021260910 A1 WO 2021260910A1 JP 2020025175 W JP2020025175 W JP 2020025175W WO 2021260910 A1 WO2021260910 A1 WO 2021260910A1
Authority
WO
WIPO (PCT)
Prior art keywords
control
data
vehicle
learning
unit
Prior art date
Application number
PCT/JP2020/025175
Other languages
French (fr)
Japanese (ja)
Inventor
紀俊 川口
一真 千々和
匠 星
Original Assignee
三菱電機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱電機株式会社 filed Critical 三菱電機株式会社
Priority to JP2022532199A priority Critical patent/JP7414995B2/en
Priority to PCT/JP2020/025175 priority patent/WO2021260910A1/en
Publication of WO2021260910A1 publication Critical patent/WO2021260910A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning

Definitions

  • the present disclosure relates to an AI integrated system that integrates a plurality of AIs (Artificial Intelligence) that control based on various input information, and an AI integrated device and an AI integrated program used therein.
  • AIs Artificial Intelligence
  • AI is installed in various in-vehicle devices such as using AI for controlling an in-vehicle camera 8 for detecting a human body and controlling a steering device.
  • Patent Document 1 a technique for controlling a plurality of lower-level devices equipped with AI from higher-level devices is disclosed (for example, Patent Document 1).
  • a user can easily construct a neural network mounted on a plurality of devices via a GUI, or control a plurality of devices equipped with a neural network from a higher-level device. Make it possible.
  • the present disclosure is to solve the above-mentioned problems, and an object of the present invention is to enable AI that performs various controls to be appropriately controlled after being integrated into one system.
  • the AI integrated system receives detection information indicating the characteristics of the environment in which the controlled device operates as input via at least one of a sensor and an external network, and generates a plurality of control signals for controlling the controlled device.
  • the device to be controlled is controlled using the control unit that selects one of the trained models based on at least one of the detection information or the generated control signal, and the selected trained model. It is equipped with a sensor processing unit.
  • the AI integrated system receives detection information indicating the characteristics of the environment in which the controlled device operates as input via at least one of a sensor and a communication network, and generates a control signal for controlling the controlled device.
  • Learning to make the trained model additionally trained for each of the trained model, the plurality of sensor processing units that are controlled by using the trained model corresponding to each of the plurality of controlled devices, and the plurality of sensor processing units. It has a part.
  • the AI integrated device generates a plurality of control signals for controlling the device to be controlled by inputting detection information indicating the characteristics of the environment in which the device to be controlled is operated via at least one of a sensor and an external network. It is provided with a control unit that selects one from the trained models of the above, based on at least one of the detection information and the generated control signal.
  • the AI integrated device is controlled by inputting detection information indicating the characteristics of the environment in which the controlled device operates, which corresponds to each of the plurality of controlled devices, as input via at least one of the sensor and the communication network. It is provided with a control unit that preferentially performs additional learning for at least one of the trained models that generate a control signal for controlling the device.
  • the AI integrated program receives detection information indicating the characteristics of the environment in which the controlled device operates as an input via at least one of a sensor and an external network to generate a control signal for controlling the controlled device. From the trained models of the above, one suitable for the environment is selected based on at least one of the detection information and the generated control signal, and the device to be controlled is controlled.
  • the AI integrated program is controlled by inputting detection information indicating the characteristics of the environment in which the controlled device operates, which corresponds to each of the plurality of controlled devices, as input via at least one of the sensor and the communication network. Priority is given to additional training for at least one of the trained models that generate control signals for controlling the device.
  • the effect of being controlled more appropriately than before can be obtained.
  • FIG. 3 is a system configuration diagram showing various AIs integrated into an AI integrated system. It is a system block diagram for demonstrating another AI integrated system.
  • FIG. 3 is a system configuration diagram showing various AIs integrated into the AI integrated system shown in FIG. It is a system configuration diagram for demonstrating the control part of the AI integrated system shown in FIG. It is a system configuration diagram for demonstrating the configuration of the AI integrated system shown in FIG. It is a schematic diagram for demonstrating the structure of a sensor processing part. It is a figure for demonstrating the learning to perform AI, and the trained model acquired by learning.
  • Embodiment 1 AI (Artificial Intelligence) is referred to as a "learning model”, a “learned model”, or a “learner”.
  • the “learning device” refers to a control device composed of software or LSI that can determine output information with respect to input information by using a known learning method, and the “learning model” is used.
  • the “learned model” refers to the software or LSI in the previous stage where the correspondence between the input information and the output information constituting the above-mentioned “learner” is determined, and the “learned model” refers to the input information and the output constituting the above-mentioned “learner”. It refers to software or LSI at the stage where the correspondence with information is determined, but it may have a configuration other than the above-mentioned configuration as long as it acquires intelligence through a learning process.
  • FIG. 1 is a system configuration diagram for explaining the AI integrated system 1 in the first embodiment of the present disclosure.
  • the AI integrated system 1 as used herein refers to a system that performs processing or operation according to an application by integrating and operating a plurality of devices controlled by using AI.
  • FIG. 1 shows an example of the AI integrated system 1.
  • the AI integrated system 1 is a system in which the industrial robots M1 and M2 and the conveyor device M3 are integrated.
  • the industrial robot M1 includes an arm node Arm1, a capture unit Hand1, and an image pickup unit Camera1 to identify a plurality of parts Parts1 carried by a conveyor while imaging with the image pickup unit Camera1 and identify the arm node arm1 and the capture unit Hand1. Is driven to supplement the parts Parts1.
  • the industrial robot M2 includes an arm node Arm2, a capture unit Hand2, and an image pickup unit Camera2, and identifies a plurality of parts Parts2 carried by a conveyor while taking an image with the image pickup unit Camera2, and identifies the arm node arm2 and the capture unit.
  • the part Hand2 is driven to supplement the parts Parts2.
  • the conveyor device M3 includes a conveyor Conveyor and a switching unit Switch.
  • one passage is branched into two passages in the middle to carry a plurality of parts Parts, and the switching unit Switch is a branch of the conveyor Conveyor.
  • the plurality of parts to be carried are switched to either side of the industrial robot M1 or the industrial robot M2 under predetermined conditions and flown.
  • FIG. 2 is a system configuration diagram showing various AIs integrated into the AI integrated system 1.
  • the arm node Arm1 of the industrial robot M1 is controlled by the AI AI1a
  • the capture unit Hand1 is controlled by the AI AI1h
  • the imaging unit Camera1 is controlled by the AI AI1c.
  • the arm node Arm2 of the industrial robot M2 is controlled by AI2a, which is AI
  • the capture unit Hand2 is controlled by AI2h, which is AI
  • the imaging unit Camera2 is controlled by AI2c, which is AI.
  • the conveyor device M3 is controlled by AI3 which is AI.
  • AI1a, AI1h and AI1c of the industrial robot M1, AI2a, AI2h and AI2c of the industrial robot M2, and AI3 of the conveyor device M3 are controlled so that the industrial robots M1, M2 and the conveyor device M3 can be interlocked with each other. conduct.
  • AI1a, AI1h and AI1c of the industrial robot M1, AI2a, AI2h and AI2c of the industrial robot M2, and AI3 of the conveyor device M3 are the industrial robots M1, M2 and the conveyor device M3 on which the respective AIs are mounted. Is individually learned about control before being integrated, and is not learned about control in the environment where each device actually operates after being integrated as AI integrated system 1. Therefore, there is a concern that there is no guarantee that each device will operate properly as a system.
  • an AI that causes each AI to perform learning for performing control suitable for the system will be described.
  • FIG. 2B an AI that causes a plurality of such AIs to perform learning is shown as AIsv.
  • FIG. 3 is a system configuration diagram for explaining another AI integrated system 1.
  • FIG. 3 shows the configuration of the vehicle 2 as the AI integrated system 1.
  • the vehicle 2 is referred to as an automobile (more specifically, a completed vehicle) here.
  • the AI integrated system 1 is also simply referred to as a “system”.
  • the vehicle body 3 which is the body of the vehicle 2, is equipped with a tire 4, a door 5, a headlight 6, an electronic mirror 7, an in-vehicle camera 8, a radar 9, and a transmission 10.
  • the in-vehicle camera 8 and the radar 9 are devices for detecting an object around the vehicle 2.
  • the in-vehicle camera 8 is an image pickup device.
  • an electromagnetic wave such as a laser radar such as LiDAR (light detection and ranging) or a millimeter wave radar is used.
  • the drive device 11 is a device that generates a driving force for driving the vehicle 2, such as an engine or a motor.
  • the braking device 12 is a device or a deceleration mechanism that generates a braking force for decelerating or stopping the vehicle 2, such as a mechanical brake or a power regenerative brake.
  • the steering device 13 is a steering device for changing the traveling direction of the vehicle 2.
  • the shock absorber 14 is a suspension device that generates a damping force for relaxing and cushioning stresses such as vibration and inertial force generated in the vehicle 2.
  • the driving device 11 and the braking device 12 apply a driving force and a braking force to the tire 4. Further, the steering device 13 gives a drag force toward the traveling direction by changing the direction of the tire 4. Further, the shock absorber 14 applies a damping force between the tire 4 and the vehicle body 3 in order to relieve the stress generated between the vehicle 2 and the road surface.
  • the UI device 15 displays the surroundings of the vehicle 2 to the occupants, such as an instrument panel including a meter device, a car navigation system, or an information terminal device, notifies the situation around the vehicle 2, and informs the vehicle 2. It is a device for UI (User Interface, User Interface) that enables operation of various mounted devices.
  • the door 5, the headlight 6, and the electronic mirror 7 operate by receiving an operation from the passenger via the UI device 15.
  • the recognition device 16 detects and recognizes an object inside and outside the vehicle 2 and detects and recognizes an object inside and outside the vehicle 2 by controlling an in-vehicle camera 8, a radar 9, a GPS (Global Positioning System) device (not shown), and the like individually or in conjunction with each other. Get information.
  • the recognition device 16 may control the headlight 6 to improve the accuracy of detection and recognition of an object or the like by imaging with the vehicle-mounted camera 8.
  • the transmission device 17 controls a transmission 10 which is a path related to drive transmission such as a transmission and a differential device.
  • the above-mentioned drive device 11, braking device 12, steering device 13, shock absorber 14, UI device 15, recognition device 16, and transmission device 17 may be mounted as the same device.
  • other devices mounted on the vehicle 2 may be included in the AI integrated system 1.
  • FIG. 4 is a system configuration diagram showing various AIs integrated into the AI integrated system 1 shown in FIG.
  • the AI integrated system 1 integrates an AI for controlling each of the drive device 11, the braking device 12, the steering device 13, the shock absorber 14, the UI device 15, the recognition device 16, and the transmission device 17.
  • Each AI is a learner Lm using machine learning, and is, for example, a trained model trained using a neural network.
  • Each AI is composed of software or LSI.
  • the AI that controls the drive device 11 is the drive control unit 21, the AI that controls the braking device 12 is the braking control unit 22, the AI that controls the steering device 13 is the steering control unit 23, and the AI that controls the shock absorber 14 is the buffer control unit.
  • the AI that controls the UI device 15 is the UI control unit 25
  • the AI that controls the recognition device 16 is the recognition control unit 26
  • the AI that controls the transmission device 17 is the transmission control unit 27.
  • FIG. 5 is a system configuration diagram for explaining the control unit 30 of the AI integrated system 1 shown in FIG.
  • the AI integrated system 1 includes a control unit 30.
  • the control unit 30 is also treated as an AI integrated device.
  • the control unit 30 shall also handle the AI integrated program, the AI integrated circuit, and the AI integrated data.
  • the control unit 30 causes the drive control unit 21, the braking control unit 22, the steering control unit 23, the buffer control unit 24, the UI control unit 25, the recognition control unit 26, and the transmission control unit 27 to perform integrated learning. Further, the control unit 30 estimates the driving scene Ds, which is the environment in which the vehicle 2 travels, evaluates the control of each AI integrated in the AI integrated system 1, and switches the trained model used for controlling each AI. And so on. The details of the processing in the control unit 30 will be described later.
  • the driving scene Ds is treated as an example of the environment in which the vehicle 2 is placed, but the environment here is not limited to the state or situation related to the driving of the vehicle 2, for example, the passenger. This includes those related to the control of the in-vehicle device, such as the behavior, physical condition and safety status of the vehicle 2, and the state of aging deterioration of the vehicle 2.
  • FIG. 6 is a system configuration diagram for explaining the configuration of the AI integrated system 1 shown in FIG.
  • the AI integrated system 1 includes a control unit 30 and a plurality of sensor processing units 31A, 31B, ..., 31N.
  • the control unit 30 connects to a plurality of sensor processing units 31 to transmit and receive various signals.
  • the AI integrated system 1 includes various sensors 32A, 32B, ..., 32N mounted on the vehicle 2, a communication device 41 connected to the vehicle-mounted network 40, and various vehicle-mounted devices mounted on the vehicle 2. Vd is connected. These sensors 32, the communication device 41, and the in-vehicle device Vd are connected via the signal transmission path 42.
  • the in-vehicle network 40 is connected to vehicle-to-vehicle communication, ground-to-vehicle communication, and a wide-area communication network or an Internet network laid as social infrastructure.
  • the sensor processing unit 31 connected to the sensor 32A is referred to as the sensor processing unit 31A
  • the sensor processing unit 31 connected to the sensor 32B is referred to as the sensor processing unit 31B. It does not have to be one-to-one, and may be one-to-many, multiple-to-one, or multiple-to-many.
  • Various sensors 32A, 32B, ..., 32N are devices mounted on the vehicle 2, and various information related to the vehicle 2, such as running speed information, load information applied to the suspension, and temperature information, are used. Is detected and output.
  • any of the plurality of sensors 32 may include velocity and acceleration, angles and angular velocities including roll pitch yaw, vibration amplitude and frequency, and positive torque and at least one of the vehicle body 3 and individual wheels.
  • Dynamic information related to driving control such as negative torque (that is, driving force and braking force) and temperature, humidity, illuminance, weight, etc. in at least one of the inside of the vehicle body 3, the outside of the vehicle body 3, and the vehicle body 3 itself. It detects static information related to driving conditions and outputs the detected information to the corresponding sensor processing unit 31.
  • any of the other sensors 32 is information related to image pickup by the camera 8, object detection by the radar 9, and recognition of surrounding conditions such as position detection using GPS and data communication using a communication device. Is acquired, and the acquired information is output to the corresponding sensor processing unit 31.
  • the vehicle 2 as a system 1 that integrates various devices recognizes the state of the own vehicle 2 based on the information obtained from the various sensors 32 and communication described above, and associates the action Act to be taken in the recognized state St. By handling the data set (behavior Act, state St), it interacts with the environment in which the own vehicle 2 is placed.
  • the vehicle 2 in which all the various devices are integrated is referred to as a completed vehicle.
  • the state St of the own vehicle 2 is derived from various information obtained via the sensor 32 and communication described above, for example, position information, distance information to the target, speed information, and the like.
  • the information obtained from various sensors 32 and communication is collectively referred to as detection information Si.
  • the action Act refers to the control of various in-vehicle devices Vd mounted on the vehicle 2 and the notification or provision of various information to the passengers.
  • the sensor processing unit 31 is connected to the vehicle-mounted device Vd to be controlled so as to be capable of bidirectional signal transmission.
  • the sensor processing unit 31 is mounted on the vehicle-mounted device Vd to be controlled or another device connected to the signal transmission path 42.
  • the sensor processing unit 31 generates control signals Cs based on the various sensors 32 described above and the detection information Si input via communication, and transmits the generated control signals Cs to the vehicle-mounted device Vd to be controlled. do.
  • the sensor processing unit 31 sequentially outputs the input detection information Si and the control signal Cs generated for controlling the in-vehicle device Vd to the control unit 30.
  • the control unit 30 includes a storage unit 30m, and can hold various information such as input information, derived information, or set information in the storage unit 30m until it is temporarily or erased.
  • the control unit 30 holds the related information Ri associated with the detection information Si, the control signal Cs, and the sensor processing unit 31 of the output source.
  • Information indicating the sensor processing unit 31 of the output source is added to the detection information Si and the control signal Cs input to the control unit 30. This is given as output source information when the sensor processing unit 31 outputs the detection information Si and the control signal Cs, or the device on the signal transmission path 42 gives the information and detection information of the output source sensor processing unit 31. It is possible by adding it to associate Si and the control signal Cs.
  • the above-mentioned data set (behavior Act, state St) can be treated as a data set (control signal Cs, detection information Si) in which the detection information Si and the control signal Cs are associated with each other.
  • a neural network is treated as an example of AI implemented by the sensor processing unit 31.
  • the sensor processing unit 31 holds a plurality of neural network data, switches and sets the plurality of neural network data, and controls the vehicle-mounted device Vd to be controlled.
  • the control signal Cs corresponding to the detection information Si is generated.
  • the neural network data will be referred to as NN data Nd.
  • the sensor processing unit 31A holds the learner Lm, the NN data NdA1, NdA2, ..., NdAn, and the teacher data TdA. Further, the sensor processing unit 31B holds the learner Lm, the NN data NdB1, NdB2, ..., NdBn, and the teacher data TdB. After that, the same applies to the sensor processing unit 31N.
  • the plurality of NN data Nd held by the sensor processing unit 31 is a trained model.
  • This trained model trains a training model constructed by a neural network using a data set of input detection information Si and output information related to control (or control signal Cs), that is, teacher data Td. Obtained by letting them do it.
  • the teacher data Td is also called correct answer data.
  • the learning of the plurality of NN data Nd held by the sensor processing unit 31 is performed prior to the integration of the sensor processing unit 31 into the vehicle 2.
  • the learning model of AI may be constructed by a machine learning method other than the neural network, for example, reinforcement learning.
  • FIG. 7 is a schematic diagram for explaining the configuration of the sensor processing unit 31.
  • the sensor processing unit 31 has an AI (that is, a learner Lm), a storage device (not shown) for holding a plurality of NN data Nd, and a setting (not shown) for setting one of the plurality of NN data Nd in the AI.
  • a communication unit (not shown) for communicating with other devices inside and outside the vehicle is provided.
  • the plurality of NN data Nd included in the sensor processing unit 31 is a learned model in which learning with the learner Lm using the teacher data group has converged before the sensor processing unit 31 is integrated into the vehicle 2.
  • the AI of the sensor processing unit 31A is realized by a neural network and has a perceptron.
  • Each of the plurality of NN data NdA1, NdA2, ..., NdAn is parameter data for configuring the network of the perceptron
  • the setting unit is a plurality of NN data NdA1, NdA2, ..., NdAn.
  • One of them is set to the AI perceptron. That is, the configuration of the perceptron can be changed by a plurality of NN data NdA1, NdA2, ..., NdAn.
  • the AI of the sensor processing unit 31A can generate different output information (that is, control signals Cs) for certain input information (that is, detection information Si) depending on the set NN data Nd. It will be possible.
  • each sensor processing unit 31 has high adaptability based on the detection information Si input according to the environment by setting the NN data Nd that can be adapted to the environment in which the vehicle 2 is placed in AI.
  • the control signal Cs (in other words, interactive with the environment) can be generated to control the vehicle-mounted device Vd to be controlled.
  • the configuration of the perceptron that is, the control characteristic of AI is changed by changing the parameter data.
  • the sensor processing unit 31 has a different configuration of the perceptron.
  • the control characteristics of the AI may be changed, or the perceptron may be provided in the learner Lm. That is, the arrangement of the perceptron shall be arbitrarily designed.
  • FIG. 8 is a diagram for explaining the learning performed by AI and the trained model acquired by the learning.
  • FIG. 8 deals with, for example, a learning model for controlling the braking device 12.
  • the information B1 input in the learning process of the learning model includes the traveling speed of the vehicle 2, the traveling point (or the traveling position), the target point (or the target distance or the target position), and the target point. , Target speed, etc.
  • the input information B2 that becomes the teacher data Td in the learning process is the traveling speed of the vehicle 2, the traveling point (or the traveling position), the target point (or the target distance or the target position), the target speed, and the braking force (or). , Braking amount or braking time) and the like.
  • the learning model trained using the teacher data Td can estimate an appropriate braking force (or braking amount or braking time) for the input information B1 and output it as output information C.
  • the data acquired for learning may handle various information related to the braking control Bc obtained in the actual traveling vehicle 2. good.
  • Various information related to the braking control Bc includes the running state (for example, the load applied to each tire 4, the inclination of the vehicle body 3, and the inertial moment acting on the vehicle body 3) obtained or derived by using the sensor 32.
  • data derived by constructing a model of the vehicle 2 and the environment and performing simulation analysis on the constructed model may be handled.
  • unique information of the vehicle 2 for example, vehicle body 3 weight, center of gravity at rest, two-wheel drive or four-wheel drive, front-wheel drive or rear-wheel drive, etc.
  • unique information of the vehicle 2 for example, vehicle body 3 weight, center of gravity at rest, two-wheel drive or four-wheel drive, front-wheel drive or rear-wheel drive, etc.
  • the AI of the sensor processing unit 31 whose control target is the braking device 12 uses the above-mentioned input information B1 and B2 and the output information C.
  • the braking control Bc can be appropriately determined at the time of an operation such as stopping at the vehicle.
  • NN data Nd as a plurality of learned models corresponding to various driving scenes Ds.
  • the various driving scenes Ds are distinguished by information on factors that characterize the various environments in which the finished vehicle travels, such as traffic regulations, driving locations, road conditions, climate, and temperature.
  • the teacher data Td used in the learning process is prepared in advance to include information on the elements that characterize this environment.
  • the driving scene Ds will be described later.
  • the in-vehicle device Vd mounted on the vehicle 2 can adjust the operation of its own device to the environment by the sensor processing unit 31 switching and applying these NN data Nd.
  • the AI of the sensor processing unit 31 is, for example, braking in an automobile equipped with an automatic driving function or a driving support function applicable to various environments. It is used for braking control of the device 12. This makes it possible for the finished vehicle manufacturer to secure the desired reliability for the in-vehicle device Vd mounted on the finished vehicle.
  • the vehicle 2 as a completed vehicle travels, for example, an urban area or a mountainous area traveling at a relatively low speed, a highway traveling at a relatively high speed or a traffic network in a suburban area with loose traffic restrictions, and a grip of a tire 4 are effective.
  • Examples include easy paved roads and unpaved roads where the grip of the tire 4 is difficult to use.
  • the detection information Si in these environments for example, in urban areas, there are many obstacles such as traveling lanes, signs, traffic lights, other vehicles 2, pedestrians, bicycles, and buildings that obstruct the visibility of the surroundings or the surroundings. .. Further, in mountainous areas, there are many frequent and severe undulations, slopes and curves in the traveling lane. Further, on the expressway, there are many positional relationships with the surrounding traveling vehicle 2, light and darkness due to entering and exiting the tunnel, lane changes including getting on and off the road, and traffic restrictions or signs.
  • a heavy rain situation in which it is difficult to recognize information from an image captured by an in-vehicle camera 8
  • a strong wind situation in which it is difficult to drive according to steering control
  • a road surface such as a puddle, freezing, and deep snow.
  • Examples include the weather and weather conditions that worsen the condition.
  • the detection information Si in the environment in which the vehicle 2 travels is affected by various traffic regulations, surrounding or surrounding conditions and road surface conditions, various weather and climatic conditions, and these conditions and conditions in a complex manner.
  • the driving scenes Ds that have multiple influences are, for example, general roads in urban areas in fine weather, unpaved driving roads in the suburbs in rainy weather, and highways in heavy snowfall, which characterize the environment. It is composed by rearranging and combining the elements of.
  • the sensor processing unit 31 inputs these plurality of elements as detection information Si.
  • the plurality of elements that characterize the environment may be information derived by simulation, or may be information acquired when the actual vehicle 2 is driven in the actual environment.
  • the AI of the sensor processing unit 31 that controls various in-vehicle devices Vd according to the environment in which the vehicle 2 is placed performs learning before being integrated into the vehicle 2.
  • a plurality of teacher data Td (hereinafter referred to as teacher data group) are prepared in advance for each driving scene Ds expressing an environment including various features.
  • a trained model is acquired by training a training model using the teacher data group for each of these running scenes Ds.
  • the trained model corresponding to one running scene Ds is one NN data Nd, and the sensor processing unit 31 holds a plurality of NN data Nd.
  • the sensor processing unit 31 performs control using the NN data Nd for which the switching instruction Sw has been given from the control unit 30 based on the detection information Si to be input.
  • the switching instruction Sw of the control unit 30 will be described later.
  • the switching of the NN data Nd in the sensor processing unit 31 at this time is the timing at which the control of the sensor processing unit 31 or the operation of the in-vehicle device Vd is stable even if it is not immediately after the switching instruction Sw from the control unit 30 is received. You may do it at.
  • the degree of change in speed is relaxed in the urban area and the in-vehicle camera is used.
  • the detection target of 8 and the in-vehicle radar 9 is aimed at relatively small objects (for example, pedestrians, bicycles, signals, etc.), and the sensitivity is particularly increased.
  • the sensitivity of the in-vehicle camera 8 and the in-vehicle radar is particularly increased toward relatively large objects (for example, curved cliffs and slopes). It is possible to control the vehicle-mounted device Vd.
  • the sensor processing unit 31 has priority over the amount of control, the period (or time), the rate of change (or the speed of change), and other controls, for example, even in the control of making a right turn, depending on the traveling environment. It is possible to generate and output control signals Cs having different degrees.
  • the control unit 30 detects information for estimating various driving scenes Ds, information indicating the association between the plurality of NN data Nd held by each sensor processing unit 31 and the driving scene Ds, and the sensor processing unit 31.
  • the related information Ri associated with the information Si and the control signal Cs is stored in advance. This related information Ri is based on the information related to the learning performed for the AI before integration into the system 1, that is, the data set of the input information B1 and B2 and the output information C shown in FIG. 8A. Generated. Further, the control unit 30 holds the information of the NN data Nd set in each sensor processing unit 31.
  • the control unit 30 includes an evaluation unit 30e and a selection unit 30s.
  • the evaluation unit 30e of the control unit 30 will be described.
  • the evaluation unit 30e Based on the input detection information Si, the evaluation unit 30e extracts or derives an element that characterizes the environment in which the vehicle 2 is placed, and whether control by the NN data Nd set in the sensor processing unit 31 is appropriate. Please evaluate.
  • the evaluation unit 30e evaluates the data set of the detection information Si and the control signal Cs output by each sensor processing unit 31 after being integrated into the vehicle 2 by using the above-mentioned related information Ri. That is, the evaluation unit 30e uses the data set of the state St (that is, the input detection information Si) and the action Act (that is, the output control signal Cs) of the vehicle 2 when the vehicle 2 interacts with the environment. The control of the NN data Nd set in each AI is evaluated.
  • FIG. 9 is a schematic diagram for explaining a process in which the evaluation unit 30e of the control unit 30 evaluates the NN data Nd based on the data set of the detection information Si and the control signal Cs.
  • the NN data NdA1 set in the sensor processing unit 31A is a learned model acquired for controlling the drive device 11 (for example, an engine or a motor) in traveling in an urban area.
  • the NN data NdA2 set in the sensor processing unit 31A is a learned model acquired for controlling the drive device 11 (for example, an engine or a motor) in traveling on a highway.
  • the NN data NdB1 set in the sensor processing unit 31B is a learned model acquired for controlling the braking device 12 (for example, the braking device) in traveling in an urban area. Further, the NN data NdB2 set in the sensor processing unit 31B is a learned model acquired for controlling the braking device 12 (for example, the braking device) in traveling on a highway.
  • the NN data NdC1 set in the sensor processing unit 31C is a learned model acquired for controlling the steering device 13 (for example, the steering device) in traveling in an urban area. Further, the NN data NdC2 set in the sensor processing unit 31C is a learned model acquired for controlling the steering device 13 (for example, the steering device) in traveling on a highway.
  • the NN data NdD1 set in the sensor processing unit 31D is a learned model acquired for controlling the shock absorber 14 (for example, the suspension device) in traveling in an urban area.
  • the NN data NdD2 set in the sensor processing unit 31D is a learned model acquired for controlling the shock absorber 14 (for example, a suspension device) in traveling on a highway.
  • the NN data Nd set in the sensor processing unit 31 is the environment based on the data set of the detection information Si and the control signal Cs input from the sensor processing units 31A to D after integration into the vehicle 2. Evaluate whether it is suitable for.
  • FIG. 9A shows the traveling speed v of the vehicle 2 and the traveling speed v of the vehicle 2 when the sensor processing unit 31A controls the drive device 11 (for example, an engine or a motor) using the NN data NdA1 and the NN data NdA2. It shows the relationship with the degree of instability Is of running derived from vibration and inclination.
  • the running instability degree Is a data set of information related to control for the in-vehicle device Vd to be evaluated and information related to the sensor 32 may be used, for example, the frictional force of each tire 4. And the load (or impact force), and the kinetic energy and moment of inertia of the vehicle 2 may be used.
  • the control for the drive device 11 is referred to as a drive control Dc.
  • the NN data NdA1 suppresses the degree of instability Is of traveling lower than that of the NN data NdA2 when the traveling speed v is 40 km / h to 60 km / h. be able to. Further, according to the learning process, the NN data NdA2 can suppress the degree of instability Is of traveling lower than that of the NN data NdA1 when the traveling speed v is in a state of 80 km / h to 100 km / h.
  • the evaluation unit 30e is setting NN data NdA1 in the sensor processing unit 31A based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31A, and the detection information Si and the control signal.
  • the traveling speed v is in the state of 40 km / h to 60 km / h based on the Cs data set
  • the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs
  • it is evaluated that the control of the NN data NdA1 is appropriate.
  • NN data NdA2 is being set in the sensor processing unit 31A based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31A, and the detection information Si
  • the traveling speed v is in the state of 80 km / h to 100 km / h based on the data set of the control signal Cs
  • the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs
  • it is evaluated that the control of the NN data NdA2 is appropriate.
  • FIG. 9B shows the traveling speed v of the vehicle 2 and the degree of instability of traveling when the sensor processing unit 31B controls the braking device 12 (for example, the braking device) using the NN data NdB1 and the NN data NdB2. It shows the relationship with Is.
  • the degree of instability in running Is is the same as in FIG. 9A.
  • the control for the braking device 12 is referred to as braking control Bc.
  • the NN data NdB1 suppresses the degree of instability Is of traveling lower than the NN data NdB2 when the traveling speed v is in a state of 40 km / h to 60 km / h. be able to.
  • the NN data NdB2 can suppress the degree of instability Is of traveling lower than that of the NN data NdB1 when the traveling speed v is in a state of 80 km / h to 100 km / h.
  • the evaluation unit 30e is setting NN data NdB1 in the sensor processing unit 31B based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31B, and the detection information Si and the control signal.
  • the traveling speed v is in the state of 40 km / h to 60 km / h based on the Cs data set
  • the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs
  • it is evaluated that the control of the NN data NdB1 is appropriate.
  • NN data NdB2 is being set in the sensor processing unit 31B based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31B, and the detection information Si
  • the traveling speed v is in the state of 80 km / h to 100 km / h based on the data set of the control signal Cs
  • the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs
  • it is evaluated that the control of the NN data NdB2 is appropriate.
  • FIG. 9C shows the traveling speed v of the vehicle 2 and the degree of instability of traveling when the sensor processing unit 31C controls the steering device 13 (for example, the steering device) using the NN data NdC1 and the NN data NdC2. It shows the relationship with Is.
  • the degree of instability in running Is is the same as in FIG. 9A.
  • the control for the steering device 13 is defined as steering control Sc.
  • the NN data NdC1 suppresses the degree of instability Is of traveling lower than the NN data NdC2 when the traveling speed v is in a state of 40 km / h to 60 km / h. be able to.
  • the NN data NdC2 can suppress the degree of instability Is of traveling lower than that of the NN data NdC1 when the traveling speed v is in a state of 80 km / h to 100 km / h.
  • the evaluation unit 30e is setting NN data NdC1 in the sensor processing unit 31C based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31C, and the detection information Si and the control signal.
  • the traveling speed v is in the state of 40 km / h to 60 km / h based on the Cs data set
  • the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs
  • it is evaluated that the control of the NN data NdC1 is appropriate.
  • NN data NdC2 is being set in the sensor processing unit 31C based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31C, and the detection information Si
  • the traveling speed v is in the state of 80 km / h to 100 km / h based on the data set of the control signal Cs
  • the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs
  • it is evaluated that the control of the NN data NdC2 is appropriate.
  • FIG. 9D shows the traveling speed v of the vehicle 2 and the degree of instability of traveling when the sensor processing unit 31D controls the shock absorber 14 (for example, the suspension device) using the NN data NdD1 and the NN data NdD2. It shows the relationship with Is.
  • the degree of instability in running Is is the same as in FIG. 9A.
  • the control for the shock absorber 14 is referred to as buffer control Cc.
  • the NN data NdD1 suppresses the degree of instability Is of traveling lower than that of the NN data NdD2 when the traveling speed v is 40 km / h to 60 km / h. be able to.
  • the NN data NdD2 can suppress the degree of instability Is of traveling lower than that of the NN data NdD1 when the traveling speed v is in a state of 80 km / h to 100 km / h.
  • the evaluation unit 30e is setting NN data NdD1 in the sensor processing unit 31D based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31D, and the detection information Si and the control signal.
  • the traveling speed v is in the state of 40 km / h to 60 km / h based on the Cs data set
  • the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs
  • it is evaluated that the control of the NN data NdD1 is appropriate.
  • NN data NdD2 is being set in the sensor processing unit 31D based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31D, and the detection information Si
  • the traveling speed v is in the state of 80 km / h to 100 km / h based on the data set of the control signal Cs
  • the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs
  • it is evaluated that the control of the NN data NdD2 is appropriate.
  • the evaluation unit 30e can be used as a data set of detection information Si and control signal Cs when the in-vehicle device Vd to be controlled is controlled by using one of the NN data Nd in which each sensor processing unit 31 has a plurality of NN data Nd. Based on this, it is evaluated whether or not the NN data Nd set in the sensor processing unit 31 is suitable, and the stability of the operation and state of the in-vehicle device Vd under the control of the NN data Nd is evaluated.
  • the evaluation unit 30e may further determine and evaluate whether or not the control when the NN data Nd being set is combined with the plurality of sensor processing units 31 is functioning appropriately.
  • the selection unit 30s of the control unit 30 will be described.
  • the selection unit 30s determines the input detection information Si and the control. Based on the data set of the signal Cs, it is estimated which of the above-mentioned plurality of driving scenes Ds is close to or which driving scene Ds is suitable.
  • Information for the selection unit 30s to estimate the traveling scene Ds includes, for example, the traveling speed, vibration, inclination, kinetic energy and moment of inertia of the vehicle 2, the frictional force and load (or impact force) applied to the tire 4, and The degree of slope or curvature of the road (that is, the degree of R), temperature or climate, etc. may be mentioned.
  • the selection unit 30s may estimate the driving scene Ds using the information obtained from the vehicle-mounted network 40.
  • control unit 30 holds in advance the information of the elements that characterize the environment and the information of the driving scene Ds associated with the information of the elements.
  • the detection information Si obtained from the actual environment rarely matches the information of the elements constituting the traveling scene Ds held in advance. Therefore, among the elements that characterize the environment, those related to driving safety or reliability are given high priority in advance, and the selection unit 30s may estimate the driving scene Ds having high priority comprehensively. A plurality of candidates for similar driving scenes Ds may be estimated, and the driving scene Ds corresponding to the control having the highest driving safety may be estimated.
  • the control having the highest traveling safety is, for example, one in which the control range of the traveling speed is low, or one in which the traveling control is performed based on the detection of a surrounding object or the recognition of the surrounding situation.
  • the selection unit 30s selects the NN data Nd held by each sensor processing unit 31 that is suitable for the estimated driving scene Ds. Further, the selection unit 30s selects a data set obtained by learning using a teacher data group including features that are close to or corresponding to the actual environment based on the input detection information Si and the control signal Cs data set. , NN data Nd may be selected using the information obtained from the vehicle-mounted network 40.
  • the selection unit 30s issues a switching instruction Sw to each sensor processing unit 31 so as to set the selected NN data Nd.
  • the selection unit 30s is new from a plurality of driving scenes Ds based on the data set of the detection information Si and the control signal Cs input from a certain sensor processing unit 31 regardless of the evaluation content of the evaluation unit 30e.
  • Other NN data Nd suitable for the driving scene Ds is selected from the NN data Nd set in a certain sensor processing unit 31, and the other NN data Nd is selected for the sensor processing unit 31.
  • the switching instruction Sw for setting the NN data Nd may be performed.
  • FIG. 10 is a flowchart for explaining the processing in the control unit 30 in the AI integrated system 1.
  • FIG. 10A shows the processing in the evaluation unit 30e.
  • the evaluation unit 30e inputs the data set of the detection information Si and the control signal Cs from each sensor processing unit 31.
  • the evaluation unit 30e evaluates the control of the NN data Nd of each sensor processing unit 31 corresponding to the data set input in the processing Sp81a.
  • the evaluation unit 30e determines whether or not the control in each NN data Nd is appropriate based on the evaluation in the processing Sp82a.
  • the process proceeds to the process Sp81a.
  • the process proceeds to processing Sp84a.
  • the process of FIG. 10B is performed.
  • FIG. 10B shows the processing in the selection unit 30s.
  • the selection unit 30s inputs the detection information Si from each sensor processing unit 31.
  • the selection unit 30s estimates the traveling scene Ds in which the vehicle 2 travels based on the detection information Si input in the processing Sp81b.
  • the selection unit 30s selects the NN data Nd suitable for the traveling scene Ds estimated by the processing Sp82b for each sensor processing unit 31.
  • the selection unit 30s transmits a switching instruction for switching to the NN data Nd selected in the processing Sp83b to each sensor processing unit 31.
  • FIGS. 10A and 10B may be performed independently. Further, the individual processes of FIGS. 10 (a) and 10 (b) may be started to be executed regardless of the progress of the next process. For example, if a data set is transmitted from each sensor processing unit 31, sequential input is performed.
  • the AI is mounted in the AI integrated system 1 in which a plurality of devices that perform operations for various purposes and a plurality of AIs that control the plurality of devices are integrated.
  • Each of the sensor processing units 31 inputs the detection information Si, which is information on the environment in which the system 1 is placed, and generates and outputs control signals Cs for controlling the device to be controlled.
  • Each sensor processing unit 31 holds a plurality of NN data Nd, and controls by setting one of the NN data Nd to AI.
  • the plurality of NN data Nd is a trained model in which the learning model is trained according to the environment in which the system 1 is placed.
  • the control unit 30 evaluates the control of each sensor processing unit 31 to the device to be controlled by the NN data Nd set in the AI. Further, the control unit 30 (selection unit 30s) selects the NN data Nd corresponding to the estimated driving scene Ds to each sensor processing unit 31 and instructs each sensor processing unit 31 to switch to the selected NN data Nd. Let it be set. As a result, when a plurality of AIs that control each device that performs operations for various purposes are integrated into the AI integrated system 1, NN data Nd suitable for the environment in which the system 1 is placed is sent to each sensor processing unit 31. It can be set to appropriately control each device operating in various applications.
  • the AI integrated system 1 includes, for example, an industrial robot (that is, factory automation), a monitoring system (or monitoring device), an air conditioning system (or air conditioning device), and an air conditioning device. , Home electronics, etc. that integrate multiple AIs are targeted.
  • the control unit 30 evaluates the control of each AI and sets an appropriate trained model so that the system 1 does not fall into an unstable state due to the control of a plurality of integrated AIs. Therefore, even if the AIs that control the devices that perform operations for various purposes are individually trained, when a plurality of AIs that are controlled by the trained model are integrated into the system 1, the system 1 will be used. It is possible to operate stably.
  • the control unit 30 of the AI integrated system 1 may select and set one of a plurality of trained models for one sensor processing unit corresponding to one in-vehicle device Vd.
  • FIG. 11 is a schematic diagram for explaining a first modification of the control unit 30.
  • FIG. 12 is a schematic diagram for explaining a second modification of the control unit 30.
  • the control unit 30 of the AI integrated system 1 may include at least one of the sensor processing units 31.
  • the control unit 30 of the AI integrated system 1 is located on the external server of the vehicle 2, and information is exchanged with the sensor processing unit 31 of the vehicle 2 and the like via the communication of the in-vehicle network 40. You may do it.
  • Embodiment 2 In the first embodiment, prior to integrating various vehicle-mounted devices Vd into the vehicle 2, the AI of the sensor processing unit 31 that controls the vehicle-mounted device Vd is made to learn for each of a plurality of driving scenes Ds. A plurality of NN data Nd corresponding to the scene Ds were acquired. Then, after the in-vehicle device Vd was integrated into the vehicle 2, each sensor processing unit 31 was made to switch and set a plurality of NN data Nd acquired according to the environment in which the vehicle 2 travels. In the second embodiment, integrated learning is performed on the NN data Nd set by each sensor processing unit 31 according to the environment in which the vehicle 2 is placed.
  • the plurality of NN data Nd held by each sensor processing unit 31 is a trained model acquired by the learning in the previous stage integrated as the final product (that is, the finished vehicle), and is the final model. There is no guarantee that appropriate control can be performed for the in-vehicle device Vd to be controlled in the integrated state as a product. That is, even if each sensor processing unit 31 can hold a plurality of learned NN data Nd in advance and switch between them to control the in-vehicle device Vd to be controlled, the vehicle travels in an actual environment. When integrated in the system 1 of a completed vehicle, it may not be possible to confirm the reliability of whether or not the NN data Nd set by the switching instruction Sw can continue to stably control each in-vehicle device Vd.
  • the high robustness in control means the AI of the sensor processing unit 31 (that is, learning of the learning process) in the environment in which the vehicle 2 actually travels after each sensor processing unit 31 is integrated into the vehicle 2.
  • the detection information Si including the feature not included in the teacher data Td in the learning process is input or is included in the teacher data Td.
  • the detection information Si including a plurality of features that are not input as a combination is input, or the detection information Si including a disturbance that is not considered in the learning process is input, the in-vehicle device Vd to be controlled is not input. It refers to the property of being able to quickly transition to a stable operating state and continue control without leaving it in a stable operating state.
  • FIG. 13 is a schematic diagram for explaining a control region for expressing the detection information Si and the control signal Cs handled in the control by AI in the second embodiment of the present disclosure in two dimensions.
  • the control unit 30 has the detection information Si and the detection information Si that can theoretically stably control the in-vehicle device Vd based on the teacher data Td obtained by the analysis by the simulation simulating the completed vehicle or the actual measurement using the actual completed vehicle.
  • Information in a controllable region represented by a range of control signals Cs is held in association with each sensor processing unit 31.
  • the control area of the sensor processing unit 31A whose control target is the drive device 11 handles the speed and acceleration of the vehicle 2 as the detection information Si, and handles the driving force Df as the control signal Cs.
  • the control area of the sensor processing unit 31B whose control target is the braking device 12 handles the slope and frictional force of the road surface (that is, the degree of grip between each tire 4 and the road surface) as the detection information Si, and the control signal Cs. It is assumed that the braking force Bf is treated as.
  • the control area of the sensor processing unit 31C whose control target is the steering device 13 handles changes in the direction and position of the vehicle 2 (for example, the amount of change or the rate of change) as the detection information Si, and the steering reaction as the control signal Cs.
  • Sr for example, steering amount or steering speed
  • the control area of the sensor processing unit 31D whose control target is the shock absorber 14 handles the vibration and stress change (for example, change amount or rate of change) of the vehicle 2 as the detection information Si, and the buffer reaction as the control signal Cs. Cr (eg, buffer amount or buffer rate) shall be dealt with.
  • the ellipses A1 to A4, B1 to B4, C1 to C4 and D1 to D4 on the control region are NN data NdA1 to NdA4, NdB1 to NdB4, NdC1 to NdC4 and NdD1 held by the sensor processing units 31A, 31B, 31C and 31D.
  • ⁇ NdD4 is shown, and the range of the detection information Si and the control signal Cs handled by each NN data Nd is schematically shown.
  • the ellipse A2 when comparing the ellipse A1 corresponding to the driving scene Ds in the urban area and the ellipse A2 corresponding to the driving scene Ds on the highway, the ellipse A2 has a higher numerical value than the ellipse A1.
  • the driving force will be controlled so as to maintain the speed and acceleration of the vehicle 2 in the range.
  • Temporary low-speed driving such as entering and exiting the expressway or slow driving such as traffic congestion shall be included in the driving scene Ds and the teacher data Td corresponding to the urban area or the expressway.
  • the range in which the speed and acceleration of the vehicle 2 are maintained is the same, but the ellipse A1
  • the ellipse A3 controls the driving force in a range where the numerical value is higher. Comparing the ellipse A1 corresponding to the driving scene Ds in the urban area and the ellipse A4 corresponding to the driving scene Ds on the unpaved road in the suburbs, the ellipse A3 has a higher numerical value than the ellipse A1.
  • the driving force will be controlled in the range where the numerical value is high while maintaining the speed and acceleration.
  • this elliptical frame is defined as the controllable area Ia.
  • the range of the detection information Si to be input and the control signal Cs to be generated corresponds to the ellipse B1 which is the controllable area Ia corresponding to the driving scene Ds in the urban area and the driving scene Ds on the highway.
  • the ellipse B2 which is a controllable area Ia
  • an ellipse B3 which is a controllable area Ia corresponding to a driving scene Ds in a mountainous area
  • an ellipse B4 which is a controllable area Ia corresponding to a driving scene Ds on an unpaved road.
  • the range of the detection information Si to be input and the control signal Cs to be generated corresponds to the ellipse C1 which is the controllable area Ia corresponding to the driving scene Ds in the urban area and the driving scene Ds on the highway.
  • the ellipse C2 which is a controllable area Ia
  • an ellipse C3 which is a controllable area Ia corresponding to a driving scene Ds in a mountainous area
  • an ellipse C4 which is a controllable area Ia corresponding to a driving scene Ds on an unpaved road.
  • the range of the detection information Si to be input and the control signal Cs to be generated corresponds to the ellipse D1 which is the controllable area Ia corresponding to the driving scene Ds in the urban area and the driving scene Ds on the highway.
  • the ellipse D2 which is a controllable area Ia
  • an ellipse D3 which is a controllable area Ia corresponding to a driving scene Ds in a mountainous area
  • an ellipse D4 which is a controllable area Ia corresponding to a driving scene Ds on an unpaved road.
  • Each of the NN data NdA1, NdA2, ..., NdD4 puts the vehicle 2 in the environment in the controllable area Ia acquired by the learning before the integration when the in-vehicle device Vd and the sensor processing unit 31 are integrated into the vehicle 2. It is expected to adapt and control driving.
  • the control unit 30 holds in advance the controllable area Ia of each NN data Nd as information represented by the data set of the detection information Si and the control signal Cs, as shown in FIG.
  • the evaluation unit 30e uses the information in the controllable area Ia to evaluate the stability of the operating state of the in-vehicle device Vd and evaluate whether the control with the NN data Nd being set is functioning properly. Can be done. Further, the selection unit 30s can select the NN data Nd suitable for the current driving scene Ds by using the information of the controllable area Ia, and can perform the switching instruction Sw.
  • FIG. 14 is a schematic diagram for explaining the mutual influence in the control of each in-vehicle device Vd.
  • FIG. 14 shows that when the vehicle 2 travels in a mountainous area, the sensor processing units 31A, 31B, 31C and 31D control the NN data NdA3, NdB3, NdC3 and NdD3.
  • the solid line arrows Eab, Eac and Ead indicate that the control of the sensor processing unit 31A by the NN data NdA3 interacts with the control of the other sensor processing units 31 by the NN data NdB3, NdC3 and NdD3.
  • each of the NN data NdB3, NdC3 and NdD3 in the braking device 12 is the point Ub of the controllable area Ia.
  • Uc and Ud are used for control.
  • the evaluation unit 30e evaluates that the control of each NN data Nd is functioning appropriately.
  • each of the NN data NdB3, NdC3 and NdD3 in the braking device 12 is in the controllable area Ia. It is assumed that control is performed at points Fb, Fc and Fd. At this time, the points Fa and Fb are included in the controllable area Ia, but the points Fc and Fd are not included in the controllable area Ia. Evaluate not.
  • the controllable range Ia There is a possibility that the vehicle 2 can be maintained in a stable running state.
  • the control system including the trained model (in other words, the control system) is expected to have some robustness. That is, it is expected that the NN data Nd of the sensor processing unit 31 can control the in-vehicle device Vd to be controlled even in the region of the broken line elliptical frame beyond the controllable range Ia by having this robustness.
  • this broken line elliptical frame is defined as the robust control area Ra.
  • Such control in the robust control range Ra is for temporary or temporal changes in running characteristics such as the air pressure or brake pressure of the tire 4 of the vehicle 2 and the weight or wind pressure of the vehicle body 3 due to the load.
  • the robust control region Ra is an uncertain region, and it is difficult to provide the control system (in other words, the control system) with the robustness intended by the design. Therefore, there is a possibility that there is an out-of-applicable region Na that cannot be dealt with by the robustness of the NN data Nd held by the sensor processing unit 31.
  • each in-vehicle device Vd may be controlled in an uncertain robust control area Ra other than the controllable area Ia of the NN data NdA3, NdB3, NdC3 and NdD3.
  • an uncertain robust control area Ra it is not always properly linked on the system 1 of the vehicle 2.
  • the result of the control in the robust control area Ra in one NN data Nd is the other NN data. It may lead to control in the non-adaptive region Na with respect to Nd, and may fall into a situation where the other NN data Nd cannot be controlled.
  • Such a non-adaptive region Na can be predicted to some extent by evaluating the range in which each NN data Nd of the sensor processing unit 31 can be controlled even before the in-vehicle device Vd is integrated into the vehicle 2.
  • the integrated learning to narrow the non-adaptive region Na is the individual NN data Nd before being integrated into the vehicle 2. (That is, in the process of acquiring NN data Nd as a trained model).
  • FIG. 15 is a schematic diagram for explaining the robustness of the control by AI.
  • the robust control areas Ra of the NN data NdA3, NdB3, NdC3 and NdD3 that control each in-vehicle device Vd are set so as not to fall into the control in the non-adaptive region Na shown in FIG.
  • the possibility that the control in another NN data Nd is included in the robust control area Ra increases.
  • each NN data Nd rapidly shifts the control of the controlled object into the controllable area Ia, and as a result, the possibility of continuing stable control increases.
  • FIG. 15 shows that as a result of the control in the NN data NdC3 being performed in the robust control area Ra, the control in the NN data NdD3 is in the extended robust control area Ra. As a result, at the timing of the next control, the NN data NdC3 and NdD3 can quickly return to the control within the controllable area Ia.
  • the control unit 30 includes a learning unit 30a.
  • the control unit 30 previously holds information on the controllable area Ia of the data sets of the plurality of NN data NdA1, NdA2, ..., NdD4 in each sensor processing unit 31 as shown in FIG. Then, when the NN data Nd set in the sensor processing unit 31 controls outside the controllable area Ia and then promptly returns to the control within the controllable area Ia, the learning unit 30a controls at this time.
  • the data set corresponding to the control outside the possible area Ia is set as the robust control area Ra of the NN data Nd, and the information of the controllable area Ia held in advance is added and updated.
  • the operating state of the various in-vehicle devices Vd constituting the vehicle 2 is stable or stable. This can be done by whether the transition to the above-mentioned state has occurred or by the evaluation unit 30e evaluating the stability in a period predetermined by the design.
  • FIG. 16 is a schematic diagram for explaining a state in which robustness in control by AI is expanded. As shown in FIG. 16, if the robust control range Ra of the NN data Nd is expanded in other driving scenes Ds as well as the driving scene Ds in the mountainous area, the traveling of the vehicle 2 can be controlled more stably. It becomes possible.
  • FIG. 17 is a schematic diagram showing an overlapping portion of the AI control area corresponding to the traveling scene Ds.
  • the overlapping control area Da is in the area where the controllable area Ia and the robust control area Ra of a plurality of NN data Nd corresponding to the traveling scene Ds held by the sensor processing unit 31 are combined. May exist.
  • the handling of the data set included in such an overlapping control area Da for example, the NN data Nd corresponding to the driving scene Ds having consistency or affinity with the driving scene Ds of the NN data Nd being set may be used.
  • FIG. 18 is a schematic diagram for explaining the relationship between the vehicle 2 as the system 1, the AI as a subsystem that controls various in-vehicle devices Vd, and the control unit 30.
  • control unit 30 treats each pair of the sensor processing unit 31 and the in-vehicle device Vd to be controlled as a subsystem.
  • Each subsystem uses the NN data Nd acquired by the learning before the integration, and after the integration into the system 1 called the vehicle 2, inputs the detection information Si and generates the control signal Cs, and each subsystem Perform the intended use of.
  • the vehicle type for example, the weight of the vehicle body 3 weight, the center of gravity, the vehicle width, the wheel base, etc.
  • the vehicle type is different, or the vehicle type is the same.
  • the specifications eg, hybrid engine or motor engine, two-wheel drive or four-wheel drive, and exhaust volume, etc.
  • equipment eg, tire 4, wheels, headlight 6, etc.
  • the completed vehicle as the system 1 has different properties (that is, characteristics determined by various parameters such as vehicle type, specifications, and equipment).
  • each finished vehicle has multiple unique parameters. Therefore, in order to perform integrated learning by interlocking the NN data Nd that controls various in-vehicle devices Vd, it is necessary to consider the influence of a plurality of unique parameters of the vehicle 2 on the in-vehicle device Vd.
  • controllable area Ia of the NN data Nd acquired by the learning before integration is the same, if the properties of the vehicle 2 are different, the area where the data set transitions will also be different due to the control. do. Therefore, it is desirable that the controllable area Ia of the NN data Nd is appropriately formed for each vehicle 2 (that is, the completed vehicle), and as a result, the robust control area Ra can be expected to be appropriately expanded.
  • the integrated learning after integration is performed for the purpose of reforming the controllable region Ia of the NN data Nd and further expanding the robust control region Ra based on the reformed controllable region Ia.
  • the integrated learning of the NN data Nd after the integration is performed by driving the completed vehicle in an actual environment or in an environment simulating a certain driving scene Ds. It is assumed that these environments include elements equivalent to the characteristic elements included in the driving scene Ds in the learning before integration.
  • the selection unit 30s estimates the driving scene Ds based on the input detection information Si, and among the plurality of NN data Nd held by each sensor processing unit 31, the estimated driving scene Ds.
  • the NN data Nd corresponding to is selected and set for each sensor processing unit 31.
  • a person may perform the estimation of the traveling scene Ds performed by the selection unit 30s and the selection and setting of the NN data Nd.
  • the NN data Nd held by each sensor processing unit 31 has an additional learning non-compatible mode in which the trained model is not changed and an additional learning compatible mode in which the trained model can be changed by additional learning. , Can be switched and set.
  • the NN data Nd of each sensor processing unit 31 is set to the additional learning non-compliant mode.
  • the instruction of the additional learning compatible mode from the learning unit 30a is referred to as the additional learning compatible mode AL
  • the instruction of switching the additional learning non-compatible mode from the learning unit 30a is referred to as the additional learning non-compatible mode NL.
  • the learning unit 30a switches the NN data Nd being set for each sensor processing unit 31 to the additional learning compatible mode, and the NN data Nd of each sensor processing unit 31 independently and simultaneously performs integrated learning. An example to be performed will be described.
  • the expressway is treated as a driving scene Ds for performing integrated learning.
  • two vehicles 2A and 2B are handled as different properties for each vehicle 2.
  • the vehicle 2A is a vehicle 2 having a large displacement, a high horsepower, and a heavy weight of the vehicle body 3 (for example, a vehicle having a displacement of 4000 cc or more).
  • the vehicle 2B is a vehicle 2 having a small displacement, a low horsepower, and a light weight of the vehicle body 3 (for example, a vehicle having a displacement of 660 cc or less).
  • vehicles 2A and 2B accelerate to maintain driving speed or overtake other vehicles, lane change or decelerate for curves near junctions, and steer to change direction of travel. ..
  • the drive device 11, the braking device 12, the steering device 13, and the sensor processing unit 31A, the sensor processing unit 31B, and C that control these three in-vehicle devices Vd are handled. It shall be.
  • the NN data Nd corresponding to the expressway set in the sensor processing unit 31A, the sensor processing unit 31B and C is referred to as NN data NdA2, NdB2 and NdC2.
  • the NN data NdA2, NdB2, and NdC2 control the vehicle-mounted device Vd to be controlled, and independently perform integrated learning all at once.
  • the NN data NdB2a that controls the braking device 12 of the vehicle 2A slows down sufficiently before approaching the curve because the weight of the vehicle 2A is heavy.
  • the braking device 12 is controlled with a large braking force exceeding the possible range Ia.
  • the NN data NdC2a that controls the steering device 13 of the vehicle 2A determines that the traveling safety cannot be ensured by the control in the controllable range Ia because the traveling speed is too fast at the time of approaching the curve, and other NNs.
  • the steering device 13 Independent of the control of the data Nd, the steering device 13 is controlled with a large steering amount and steering speed exceeding the controllable range Ia. As a result, it is conceivable that the vehicle 2A will slip due to the hard grip of the tire 4 due to the strong braking and the sudden steering.
  • the NN data NdA2a that controls the drive device 11 of the vehicle 2A controls within a range of higher driving force than the NN data NdA2b that controls the drive device 11 of the vehicle 2B in order to accelerate the vehicle body 3 that is heavier than the vehicle 2B. Will be done.
  • the fluctuation of the controlled amount for controlling the driving force that is, the amount of change or the rate of change
  • the control of the NN data NdA2a has a stronger tendency to increase the driving force as compared with the NN data NdA2b as the learning is performed.
  • the NN data NdB2a that controls the braking device 12 of the vehicle 2A has a higher braking force range than the NN data NdB2b that controls the braking device 12 of the vehicle 2B in order to decelerate the vehicle body 3 that is heavier than the vehicle 2B. It will be controlled. As a result, the fluctuation of the controlled amount for controlling the braking force (that is, the amount of change or the rate of change) becomes steep. Therefore, it is considered that the control of the NN data NdB2a has a stronger tendency to increase the braking force as compared with the NN data NdB2b as the learning is performed.
  • the NN data NdC2a that controls the steering device 13 of the vehicle 2A has a steering angle as compared with the NN data NdC2b that controls the braking device 12 of the vehicle 2B in order to steer the vehicle body 3 that is heavier than the vehicle 2B and has a long wheelbase.
  • Control becomes complicated. This is because the heavier the vehicle body 3, the greater the inertia in the traveling direction, which makes it difficult to stabilize the feedback control toward the traveling direction, which is a new target.
  • the control of the NN data NdC2a has a stronger tendency to change the steering angle more frequently than the NN data NdC2b as the learning is performed.
  • the NN data NdB2b that controls the braking device 12 of the vehicle 2B travels sufficiently before approaching the curve because the weight of the vehicle 2B is light. You can slow down. Therefore, the NN data NdB2b can control the braking device 12 in the controllable area Ia.
  • the steering device 13 can be controlled in the controllable range Ia. As a result, the vehicle 2B has a good grip of the tire 4 due to appropriate braking and steering, and does not slip.
  • the vehicle 2B Since the vehicle 2B has a small displacement, the weight of the vehicle body 3 is light, and the wheelbase is short, the fluctuation of the control amount (that is, the change amount or the rate of change) is slower than that of the control by the vehicle 2A.
  • the NN data Nd which is a trained model before integration, is used as it is for controlling the vehicle 2A
  • the NN data Nd tries to control the vehicle 2A based on the prior learning process (or the teacher data Td)
  • the robust control region deviates from the controllable range Ia acquired by the prior teacher data Td.
  • the possibility of taking control by Ra is relatively large. This makes it difficult to quickly transition from the control in the robust control area Ra to the control in the controllable area Ia. In such a case, even if integrated learning is performed, it is unlikely that the controllable area Ia of the trained model before integration can be reformed into the controllable area Ia suitable for the vehicle 2A.
  • the NN data Nd which is a trained model before integration
  • the NN data Nd controls the vehicle 2B based on the prior learning process (or teacher data Td). Since the fluctuation of the operation of the vehicle 2B in the actual environment (that is, the fluctuation of the data set indicating the state of the vehicle 2 with respect to the control signal Cs) is gradual, it deviates from the controllable range Ia acquired by the prior teacher data Td. Therefore, it is considered that the possibility of taking control in the robust control range Ra is relatively small. This facilitates a rapid transition from control in the robust control area Ra to control in the controllable area Ia. In such a case, it is highly possible that the controllable area Ia of the trained model before integration can be reformed into the controllable area Ia suitable for the vehicle 2B by performing the integrated learning.
  • the learning unit 30a considers the control priority P, causes the sensor processing unit 31 having a high priority P to switch the NN data Nd being set to the additional learning compatible mode, and each sensor processing unit 31.
  • An example of causing the NN data Nd of the above to perform integrated learning in an orderly manner will be described.
  • FIG. 19 is a schematic diagram for explaining the degree of convergence of integrated learning in each NN data Nd when the control priority P is taken into consideration. It is assumed that the highway is treated as the traveling scene Ds of the vehicle 2.
  • FIGS. 19 (a1) to 19 (a3) integrated learning is performed in the order of drive control Dc, braking control Bc, and steering control Sc as control priority P.
  • FIG. 19A1 shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the drive device 11 when the sensor processing unit 31A performs the drive control Dc using the NN data NdA2. There is.
  • control stability evaluation Se can be determined by the evaluation unit 30e based on the evaluation value derived based on the information obtained from the detection information Si. Then, the evaluation unit 30e determines that the learning has converged because the fluctuation of the evaluation value is equal to or less than the convergence determination value Conv.
  • the stability evaluation Se of the control for the drive device 11 and the braking device 12 uses, for example, information such as the amount and rate of change in the traveling speed, the degree of achievement of the target traveling speed in the target mileage, and fuel efficiency. It can be determined by the evaluation unit 30e. Further, the stability evaluation Se of the control for the steering device 13 uses, for example, information such as the degree of arrival at the target traveling track at the target mileage and the posture of the vehicle 2 on the track, and the evaluation unit 30e. Can be determined by. Further, the stability evaluation Se of the control for the shock absorber 14 uses information such as the moment of inertia acting on the vehicle 2, the load applied to each tire 4, the inclination of the vehicle 2, and the vibration of the vehicle 2 to be used in the evaluation unit 30e. Can be determined by.
  • the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs exceeds 10M.
  • the number of transmissions of the control signal Cs at this time is set to Ta1.
  • FIG. 19A2 shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the braking device 12 when the sensor processing unit 31D performs the braking control Bc using the NN data NdD2.
  • the number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1).
  • the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs exceeds 10M.
  • the number of transmissions of the control signal Cs at this time is set to Ta2.
  • FIG. 19A3 shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the steering device 13 when the sensor processing unit 31B performs the steering control Sc using the NN data NdB2.
  • the number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1).
  • the fluctuation of the evaluation value converges to the convergence judgment value Conv or less before the number of transmissions of the control signal Cs reaches 10M.
  • the number of transmissions of the control signal Cs at this time is set to Ta3.
  • FIG. 20 is another schematic diagram for explaining the degree of convergence of integrated learning in each NN data Nd when the control priority P is taken into consideration. It is assumed that the highway is treated as the traveling scene Ds of the vehicle 2. In FIGS. 20 (b1) to 20 (b3), integrated learning is performed in the order of steering control Sc, drive control Dc, and braking control Bc as control priority P.
  • FIG. 20 (b1) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the steering device 13 when the sensor processing unit 31B performs the steering control Sc using the NN data NdB2.
  • 1M described in the number of transmissions of the control signal Cs indicates a predetermined number of times
  • 10M indicates 10 times of 1M
  • 100M indicates 100 times of 1M.
  • the number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1).
  • the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs exceeds 100M.
  • the number of transmissions of the control signal Cs at this time is Tb1.
  • FIG. 20 (b2) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the drive device 11 when the sensor processing unit 31A performs the drive control Dc using the NN data NdA2.
  • the number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1).
  • the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs exceeds 100M.
  • the number of transmissions of the control signal Cs at this time is Tb2.
  • FIG. 20 (b3) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the braking device 12 when the sensor processing unit 31D performs the braking control Bc using the NN data NdD2.
  • the number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1).
  • the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs exceeds 100M.
  • the number of transmissions of the control signal Cs at this time is Tb3.
  • the NN data NdA2, NdB2 and NdD2 have more control signals even though the fluctuation of the evaluation value converges as compared with the case of FIGS. 19 (a1) to 19 (a3). It requires transmission of Cs. That is, in FIGS. 20 (b1) to 20 (b3), by performing control with this priority P, the NN data NdA2, NdB2 and NdD2 can stably control the in-vehicle device Vd to be controlled.
  • it contains a large amount of data sets when the control is not performed properly, so that the acquired controllable area Ia and robust control area Ra are included. It may not be appropriate as compared with the cases of FIGS. 19 (a1) to 19 (a3).
  • FIG. 21 is yet another schematic diagram for explaining the degree of convergence of integrated learning in each NN data Nd when the control priority P is taken into consideration. It is assumed that the highway is treated as the traveling scene Ds of the vehicle 2. In FIGS. 21 (c1) to 21 (c4), integrated learning is performed in the order of buffer control Cc, drive control Dc, braking control Bc, and steering control Sc as control priority P.
  • FIG. 21 (c1) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the shock absorber 14 when the sensor processing unit 31C performs the buffer control Cc using the NN data NdC2.
  • the number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1).
  • the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs does not reach 10M.
  • Dc1 be the number of times the control signal Cs is transmitted at this time.
  • FIG. 21 (c2) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the drive device 11 when the sensor processing unit 31A performs the drive control Dc using the NN data NdA2.
  • the number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1).
  • the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs does not reach 10M.
  • the number of transmissions of the control signal Cs at this time is Dc2.
  • FIG. 21 (c3) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the braking device 12 when the sensor processing unit 31D performs the braking control Bc using the NN data NdD2.
  • the number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1).
  • the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs does not reach 10M.
  • Dc3 be the number of transmissions of the control signal Cs at this time.
  • FIG. 21 (c4) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the steering device 13 when the sensor processing unit 31B performs the steering control Sc using the NN data NdB2.
  • the number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1).
  • the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs does not reach 10M.
  • the number of transmissions of the control signal Cs at this time is Dc4.
  • the NN data NdA2, NdB2 and NdD2 have less control signals Cs even though the fluctuation of the evaluation value converges as compared with the case of FIGS. 19 (a1) to 19 (a3). It is enough to send. That is, in FIGS. 21 (c2) to 21 (c4), by performing control with this priority P, the NN data NdA2, NdB2 and NdD2 can stably control the vehicle-mounted device Vd to be controlled. Further, as compared with the cases of FIGS. 19 (a1) to 19 (a3), a large number of appropriately controlled data sets are included, and therefore, the acquired controllable area Ia and robust control area Ra are shown in FIG. 19 (a1). It can be expected that it is more appropriate than the cases of a1) to (a3).
  • the evaluation unit 30e determines whether or not the integrated learning in each NN data Nd converges, for example, the number of transmissions of the control signal Cs before the fluctuation of the evaluation value converges to the convergence judgment value Conv or less. This can be done by reaching a predetermined number of times or by elapse of a predetermined time.
  • the predetermined number of times and the predetermined time may be set in advance in the control unit 30 for each of the driving scene Ds and the sensor processing unit 31, or the control unit 30 sets the properties of the vehicle 2 and the driving scene Ds through integrated learning in the own vehicle 2. It may be set by deriving the vehicle based on the vehicle or acquiring the vehicle via the in-vehicle network 40.
  • the learning unit 30a may cause the selection unit 30s to perform the reselection instruction Rs and the switching instruction Sw of the NN data Nd.
  • the difference in the degree of convergence of the integrated learning in the drive control Dc, the braking control Bc, and the steering control Sc in FIGS. 19 (a1) to (a3) and 20 (b1) to (b3) drives the priority P of the integrated learning.
  • the control Dc, the braking control Bc, and the steering control Sc were performed in this order.
  • the priority P of the three control items Ci of the drive control Dc, the braking control Bc, and the steering control Sc is considered as follows. Even if the braking control Bc and the steering control Sc are performed in a state where the drive control Dc is not properly performed and the traveling speed is very high due to excessive acceleration, the frictional force between the tire 4 and the road surface (that is, the tire 4) is performed. The grip) may slip due to the inertial force of the vehicle body 3. Therefore, it is difficult to maintain the vehicle 2 in a stable state in a state where the drive control Dc is not properly performed. Therefore, among the three control items Ci, the one in which the drive control Dc is appropriately performed has the highest priority.
  • the steering control Sc is performed preferentially over the braking control Bc in a state where the drive control Dc is appropriately performed and the traveling speed is appropriate, the frictional force between the tire 4 and the road surface (that is, the tire) is performed.
  • the grip of 4) may slip due to the inertial force of the vehicle body 3. Therefore, it is difficult to maintain the vehicle 2 in a stable state even if the steering control Sc is performed in a state where the braking control Bc is not properly performed. Therefore, among the three control items Ci, the one in which the braking control Bc is appropriately performed has the next priority.
  • FIG. 21 (c1) This is the priority given to the integrated learning in the buffer control Cc.
  • the inclination (or posture) of the vehicle body 3 and the position of the center of gravity are controlled against the inertial force, and each tire 4 is equal to the road surface. It is desirable to ground with a moderate load.
  • the position of the center of gravity of the vehicle 2 is less likely to fluctuate, and it becomes easier for each tire 4 to come into contact with the road surface with an even load.
  • the frictional force between each tire 4 and the road surface is not easily impaired, and it becomes easy to maintain a state in which the grip is effective.
  • the shock absorber 14 changes the position of the center of gravity of the vehicle 2 by changing the degree of cushioning (that is, the amount of change or the speed of change) of each tire 4 (that is, the balance between the front / rear / left / right inclination and the weight of the vehicle body 3). Can be changed).
  • the degree of cushioning that is, the amount of change or the speed of change
  • the frictional force that is, grip
  • the tire 4 that transmits the force to the road surface against the inertial force acting in the direction different from the traveling direction of the vehicle 2 is applied. It will be possible to improve it further.
  • the shock absorber 14 increases the degree of cushioning of the rear wheels against this inertial force (that is, the rear wheels By operating (to prevent it from sinking), it is possible to prevent the inertial force from concentrating on the rear wheels and losing the frictional force of the rear wheels, and as a result, the forward rotational force from the tire 4 is easily transmitted to the road surface. be able to.
  • the vehicle 2 when the vehicle 2 curves while traveling, the vehicle 2 tilts in the direction opposite to the direction in which the vehicle 2 curves due to the inertial force.
  • the inertial force By acting to strengthen (that is, to prevent the wheels on the outside of the curve from sinking), it prevents the inertial force from concentrating on the wheels on the outside of the curve and losing the frictional force of the wheels, that is, left and right.
  • the inertial force can be appropriately distributed to the wheels of the tire 4, and as a result, the reaction force for changing the traveling direction from the tire 4 can be easily transmitted to the road surface.
  • the vehicle 2 When the vehicle 2 decelerates along with the curve, the vehicle 2 tilts forward, but the load applied to the front wheels increases the frictional force (that is, the reaction force for changing the traveling direction) at the curve. On the other hand, too much load is applied to the front wheels (in other words, the vehicle 2 tilts too much forward), which causes an action of weakening the frictional force of the rear wheels (that is, slipping of the rear wheels). It is desirable that the shock absorber 14 adjusts the degree of cushioning of the front and rear wheels on the outside of the curve so that the rear wheels do not slip due to such excessive action.
  • the buffer control Cc suitable for the vehicle 2 By performing the buffer control Cc suitable for the vehicle 2 by the integrated learning, it becomes possible to suppress the inclination of the vehicle body 3 and to make each tire 4 touch the road surface with an even load. As a result, it is possible to suppress a decrease in the frictional force between each tire 4 and the road surface to bring about an action of making the grip easier to use. Therefore, giving priority to the buffer control Cc over the drive control Dc, the braking control Bc, and the steering control Sc makes it easy to secure the stability of the vehicle 2 in running.
  • the priority P of the drive control Dc, the braking control Bc, and the steering control Sc of the vehicles 2A and B having different properties is dealt with.
  • the displacement, horsepower, and weight of the vehicle body 3 are larger in the vehicle 2A than in the vehicle 2B, the fluctuation of the control amount is also large by that amount, and the NN data Nd acquired by the learning before the integration is appropriately controlled.
  • the NN data Nd acquired by the learning before the integration is appropriately controlled.
  • the vehicle 2B has a smaller size and lighter weight of the vehicle body 3 and wheels than the vehicle 2A, so that stability is ensured when traveling on the uneven road surface. It's hard.
  • the cushioning control Cc here is, for example, the cushioning of the shock absorber 14 for each tire 4 according to the inclination of the vehicle body 3, the size of the unevenness on which the tire 4 rides, the size of the inertial force acting on the curve, and the like. It is conceivable to change the strength of the degree or change the response of the buffering operation.
  • each AI after integration may not be able to fulfill the purpose in the system 1. That is, even if integrated learning is performed, the controllable area Ia of the NN data Nd may not be properly reshaped, and the operation of the system 1 may remain unstable.
  • the learning unit 30a performs the stability evaluation Se shown in FIGS. 19 to 21 and determines that the control in the NN data Nd to be integrated learning is not functioning properly with respect to the environment. Feeds back to the process of deriving the control priority P, changes the control priority P to perform integrated learning, thereby suppressing excess or deficiency of control to the in-vehicle device Vd to be controlled. Is possible.
  • the result of performing such stability evaluation Se is accumulated as information indicating the characteristics of the operation of each in-vehicle device Vd with respect to the control, and the influence on the property of the vehicle 2 or the characteristics of the driving scene Ds is analyzed. It can be used for the teacher data Td in the learning before the integration, the evaluation index in the learning process, the estimation of the driving scene Ds in the integrated learning after the integration, and the derivation of the priority P of the control.
  • FIG. 22 is a schematic diagram for explaining an example of a mechanism in which a physical quantity related to the vehicle 2 affects the control item Ci.
  • control item Ci As the control item Ci, the drive control Dc, the braking control Bc, the steering control Sc, and the buffer control Cc are dealt with here. Further, as various parameters related to the vehicle 2, the vehicle body 3, the vehicle width, the wheelbase, the traveling speed, the weight (or mass), and the center of gravity are dealt with here.
  • the weight (or mass) is determined by the vehicle body 3 which is a completed vehicle.
  • the center of gravity is determined by the vehicle body 3, the vehicle width, the wheelbase, and the like. Passengers and loads may also be treated as parameters relating to weight (or mass) and center of gravity.
  • control results of the drive control Dc, the braking control Bc, the steering control Sc, and the buffer control Cc of the control item Ci influence each other.
  • the physical quantities constituting the control model Ca of each control item Ci are frictional force Fb, load (or impact force) Fp, kinetic energy E, and moment of inertia.
  • the control model Ca of the drive control Dc is the control model CaA
  • the control model Ca of the braking control Bc is the control model CaB
  • the control model Ca is determined for each control item Ci.
  • the load (or impact force) Fp and the kinetic energy E the quantity or volatility may be dealt with.
  • the control model Ca of the drive control Dc can be expressed including at least the frictional force Fb acting on the vehicle 2 and the kinetic energy E.
  • the control model Ca of the braking control Bc can be expressed including at least the frictional force Fb acting on the vehicle 2 and the kinetic energy E.
  • the control model Ca of the steering control Sc can be expressed including at least the frictional force Fb acting on the vehicle 2 and the moment of inertia I (or kinetic energy E).
  • the control model Ca of the buffer control Cc can be expressed including at least the load (or impact force) Fp acting on the vehicle 2.
  • the frictional force Fb and the load (or impact force) Fp can be regarded as acting on each tire 4 of the vehicle 2.
  • the drive control Dc and the braking control Bc are affected by two fluctuations of the frictional force Fb and the kinetic energy E. Then, the steering control Sc is affected by two fluctuations of the frictional force Fb and the moment of inertia I (or kinetic energy E). Then, the buffer control Cc is affected by one fluctuation of the load (or impact force) Fp.
  • the frictional force Fb can be expressed by, for example, the mathematical formula (1).
  • the load or impact force Fp can be expressed by, for example, the mathematical formula (2).
  • the kinetic energy E can be expressed by, for example, the mathematical formula (3).
  • the moment of inertia I can be expressed by, for example, the mathematical formula (4).
  • the control model Ca may be included in the learning model, or may be applied to a general control circuit or control program not limited to machine learning.
  • N included in the frictional force Fb for a very short time varies depending on the load applied to each tire 4 or the impact force Fp. Further, the load or impact force Fp applied to each tire 4 varies depending on the kinetic energy E of the traveling vehicle 2. Further, the kinetic energy E of the traveling vehicle 2 fluctuates depending on the traveling speed v of the vehicle 2 and the moment of inertia I. Further, the moment of inertia I fluctuates depending on the position d of the center of gravity of the vehicle 2.
  • the drive control Dc and the braking control Bc change the traveling speed v of the vehicle 2.
  • the buffer control Cc changes the position d of the center of gravity of the vehicle 2.
  • the steering control Sc can be treated as a small one that does not change the traveling speed and the center of gravity of the vehicle 2.
  • both the load (or impact force) Fp expressed by the kinetic energy E and the frictional force Fb expressed by the load (or impact force) Fp vary depending on the drive control Dc and the braking control Bc.
  • the steering control Sc that includes the moment of inertia I that directly receives the fluctuation of the position d of the center of gravity in the control model Ca is strongly influenced by the buffer control Cc that fluctuates the position of the center of gravity.
  • the drive control Dc, the braking control Bc, and the steering control Sc that include the kinetic energy E that directly receives the fluctuation of the traveling speed v in the control model Ca are affected by the driving control Dc and the braking control Bc that fluctuate the traveling speed v. strong.
  • the drive control Dc and the braking control Bc are compared, the drive control Dc increases the kinetic energy E and the braking control Bc decreases the kinetic energy E, so that the drive control Dc gives the other control item Ci. The influence is strong.
  • the steering control Sc is strongly influenced by the drive control Dc, the braking control Bc, and the buffer control Cc.
  • the braking control Bc is strongly influenced by the drive control Dc and the buffer control Cc.
  • the drive control Dc is strongly influenced by the braking control Bc and the buffer control Cc.
  • the buffer control Cc is affected by the drive control Dc and the braking control Bc.
  • the influence of the drive control Dc is stronger than the influence of the braking control Bc.
  • the drive control Dc, the braking control Bc, and the steering control Sc are affected by the moment of inertia I, and the moment of inertia I is affected by the steering control Sc. Therefore, the degree to which each control item Ci affects, that is, the priority P of control, is in the order of buffer control Cc> drive control Dc> braking control Bc> steering control Sc.
  • the control model Ca of the vehicle 2 is not limited to the above equations (1) to (4), and uses the functions of Lagrangian or Hamiltonian, such as momentum, potential, frictional force, and air resistance.
  • Various physical quantities including the external force expression of the above may be derived by being calculated by a computer capable of high-speed calculation (that is, a calculation device or a quantum computer specialized in the analysis of each control model Ca).
  • a computer capable of high-speed calculation that is, a calculation device or a quantum computer specialized in the analysis of each control model Ca.
  • the calculation amount may be suppressed by treating it as an analysis of the momentum.
  • a plurality of prior analysis results of each control model Ca are held in, for example, a control unit 30, a sensor processing unit 31, an in-vehicle device Vd, or a server on the in-vehicle network 40.
  • the efficiency or speed of the analysis calculation may be improved.
  • the learning unit 30a sets the NN data Nd of the sensor processing unit 31D whose control target is the shock absorber 14 to the additional learning compatible mode, and after integration.
  • the buffer control Cc suitable for the vehicle 2 of the above is integratedly learned.
  • the learning unit 30a sets the NN data Nd of the sensor processing unit 31A whose control target is the drive device 11 to the additional learning compatible mode, and causes integrated learning of the drive control Dc suitable for the vehicle 2 after integration.
  • the learning unit 30a sets the NN data Nd of the sensor processing unit 31B whose control target is the braking device 12 to the additional learning compatible mode, and causes integrated learning of the braking control Bc suitable for the vehicle 2 after integration.
  • the learning unit 30a sets the NN data Nd of the sensor processing unit 31C whose control target is the steering device 13 to the additional learning corresponding mode, and causes integrated learning of the steering control Sc suitable for the vehicle 2 after integration. In this way, by considering the control priority P, it is possible to switch the integrated learning of the NN data Nd related to each control item Ci in chronological order.
  • FIG. 23 is a schematic diagram for explaining a process of integrated learning of a plurality of AIs integrated in the AI integrated system 1.
  • the two axes shown in FIG. 23 have a control item Ci corresponding to AI on the vertical axis and a time axis Co as a time-series quantity on the horizontal axis.
  • the control items Ci on the vertical axis are arranged in the order of control priority P in the environment in which the system 1 operates (for example, the traveling scene Ds).
  • the horizontal axis may be, for example, the number of counts for calculation or counting as long as it can be handled in time series.
  • the buffer control Cc, the drive control Dc, the braking control Bc, the steering control Sc, the transmission control Tc, the recognition control Rc, the UI control Ui, and the battery control Ec are arranged in this order.
  • the learning unit 30a causes the integrated learning Ld1 to be performed on the NN data NdD of the sensor processing unit 31D corresponding to the buffer control Cc having the highest priority P at the time T1 at which the integrated learning is started.
  • the evaluation unit 30e evaluates the progress of the integrated learning Ld1 started from the time T1 based on the information such as the degree of convergence in the integrated learning Ld1 or the stability evaluation Se.
  • the learning unit 30a determines whether to switch to integrated learning of another AI based on the evaluation by the evaluation unit 30e.
  • the integrated learning Ld1 is made according to the characteristics of the devices integrated into the system 1 and the AI to be controlled. It is also possible to have the learning unit 30a process so as to make a determination to switch to the integrated learning of another AI according to the progress of learning obtained from the stability evaluation Se or the like even if the learning is not converged. Further, it is also possible to process the learning unit 30a so as to sequentially end the integrated learning of the AI and switch to the integrated learning of the next AI in descending order of the priority P of the control.
  • the learning unit 30a causes the NN data NdA of the sensor processing unit 31A corresponding to the drive control Dc having the second highest priority P to perform the integrated learning La1 at the time T2.
  • the evaluation unit 30e evaluates the progress of the integrated learning La1 started from the time T2 based on the information such as the degree of convergence in the integrated learning La1 or the stability evaluation Se.
  • the learning unit 30a determines whether to switch to integrated learning of another AI based on the evaluation by the evaluation unit 30e.
  • the learning unit 30a determines to switch to the integrated learning of another AI according to the progress of learning obtained from the stability evaluation Se or the like even if the integrated learning La1 does not converge. do.
  • the learning unit 30a causes the NN data NdD of the sensor processing unit 31D to perform the integrated learning Ld2 again at the time T3.
  • the learning unit 30a can make a determination to switch from the integrated learning Ld1 to the integrated learning La1 at the time T2 and also to make a determination to switch to the integrated learning Ld2 of the buffer control Cc again according to the progress in the integrated learning La1. Is. That is, based on the above-mentioned policy, the learning unit 30a causes the integrated learning y of another AI having a low priority P to proceed halfway before the end of the integrated learning x having a high priority P, and the priority P is again set. By advancing the high integrated learning x, it is possible to change the trained model in parallel for a plurality of AIs while considering the control priority P.
  • the learning unit 30a causes the integrated learning Lb1 to be performed on the NN data NdB of the sensor processing unit 31B corresponding to the braking control Bc having the third highest priority P at the time T4.
  • the evaluation unit 30e evaluates the progress of the integrated learning Lb1 started from the time T4 based on the information such as the degree of convergence in the integrated learning Lb1 or the stability evaluation Se.
  • the learning unit 30a determines whether to switch to integrated learning of another AI based on the evaluation by the evaluation unit 30e.
  • the learning unit 30a causes the NN data NdA of the sensor processing unit 31A corresponding to the drive control Dc to perform the integrated learning La2 again at the time T5.
  • the learning unit 30a has the NN data NdD of the sensor processing unit 31D corresponding to the buffer control Cc and the NN data NdA of the sensor processing unit 31A corresponding to the drive control Dc.
  • the NN data NdB of the sensor processing unit 31B corresponding to the braking control Bc and the NN data NdC of the sensor processing unit 31C corresponding to the steering control Sc are subjected to integrated learning.
  • integrated learning Ld1 integrated learning La1, integrated learning Ld2, integrated learning Lb1, integrated learning La2, integrated learning Lc1, integrated learning Ld3, integrated learning La3, integrated learning Lb2 and integrated learning Lc2
  • priority is given to control.
  • the learning of the NN data Nd can be sequentially completed while reflecting the degree P.
  • the learning unit 30a is connected to the NN data Nd of the sensor processing unit 31 corresponding to each of the transmission control Tc, the recognition control Rc, the UI control Ui, and the battery control Ec having a low control priority P from the time T11 to the time T17.
  • the learning unit 30a is connected to the NN data Nd of the sensor processing unit 31 corresponding to each of the transmission control Tc, the recognition control Rc, the UI control Ui, and the battery control Ec having a low control priority P from the time T11 to the time T17.
  • the integrated learning Lg1 for the NN data NdG of the sensor processing unit 31G corresponding to the UI control Ui in parallel during the integrated learning Lf1 such as starting the integrated learning Lf1 for the NN data NdF, and the integrated learning Lg1 for the NN data NdG, and
  • the integrated learning Lh1 for the NN data NdH of the sensor processing unit 31H corresponding to the battery control Ec may be started.
  • the learning unit 30a can be divided into a plurality of AIs to perform integrated learning. This allows multiple AIs to have not only one-sided additional learning (ie, integrated learning) that follows the strength of their influence, but also two-way additional learning (that is, integrated learning) that reflects the consequences of their mutual influence. Learning) makes it possible to change the learning model.
  • integrated learning one-sided additional learning
  • two-way additional learning that is, integrated learning
  • FIG. 24 is a flowchart for explaining the processing in the control unit 30 in the AI integrated system 1.
  • FIG. 24 shows a process in which the control unit 30 causes the NN data Nd of each sensor processing unit 31 to perform integrated learning.
  • the selection unit 30s estimates the traveling scene Ds in which the vehicle 2 travels based on the detection information Si input in the processing Sp81b of FIG. 10B.
  • the learning unit 30a derives the priority P of control in the NN data Nd of each sensor processing unit 31 based on the traveling scene Ds estimated by the processing Sp161.
  • the learning unit 30a causes the NN data Nd of each sensor processing unit 31 to perform integrated learning based on the control priority P derived in the processing Sp162.
  • the evaluation unit 30e evaluates the control of the NN data Nd based on the data set input by the integrated learning for the NN data Nd in the process Sp163.
  • the evaluation unit 30e determines whether or not the integrated learning in each NN data Nd has converged based on the evaluation in the process Sp164.
  • the process proceeds to Sp161
  • the control priority P is continuously derived and each of them is derived.
  • the learning unit 30a proceeds to the process Sp163 based on the evaluation in the process Sp164, and switches the execution of the integrated learning in each NN data Nd as shown in FIG. 23.
  • the process may proceed to Sp162, and the priority P of control in each NN data Nd may be derived again according to the characteristics of the traveling scene Ds.
  • the series of processes shown in FIG. 24 may be performed independently of the processes shown in FIGS. 10 (a) and 10 (b). Further, the individual processes of FIG. 24 may be started to be executed regardless of the progress of the next process. For example, while deriving the control priority P for each NN data Nd, the individual NN data Nd may be subjected to integrated learning based on the already derived priority P.
  • the learning unit 30a may always set each NN data Nd to the additional learning non-compatible mode or set the additional learning compatible mode after the integrated learning of each NN data Nd has converged. You may.
  • NN data Nd in which integrated learning is performed in one vehicle 2 is acquired by communication with another vehicle 2. It may be used.
  • each NN data Nd subjected to integrated learning is compared and analyzed to extract a difference in information, and the extracted difference information is fed back to each NN. It can be used to improve problems in control with data Nd, or reflected in the control model Ca when deriving the priority P of control, and the control with each NN data Nd is more stable, safe and reliable. It may be applied so as to have a high value. By doing so, in various environments in which the vehicle 2 travels, the vehicle 2 can individually integrate a plurality of AIs into the AI integrated system 1 without steadily performing integrated learning among the plurality of vehicles 2. It is possible to adapt a plurality of AIs to the system 1 of the vehicle 2 with higher accuracy and more quickly while complementing each other for an unsupported environment.
  • the server connected to the in-vehicle network 40 communicates with the control unit 30 mounted on the plurality of vehicles 2 and various information held by each control unit 30 and the sensor processing unit 31. And NN data Nd may be updated.
  • the control unit 30 causes a plurality of AIs to perform integrated learning in consideration of the control priority P.
  • the control unit 30 causes the NN data Nd held by each sensor processing unit 31 integrated in the system 1 to perform integrated learning.
  • the supervision unit 30 causes integrated learning to be performed on the NN data Nd corresponding to the running scene Ds estimated by the supervision unit 30 (selection unit 30s).
  • control unit 30 (learning unit 30a) is ordered from among the NN data Nd set in each sensor processing unit 31 corresponding to the driving scene Ds, based on the control priority P (that is,). Let them perform integrated learning (in order).
  • the control priority P here is, for example, the strength relationship of the degree of influence of each control item Ci on each other, and the physical quantity included when each control item Ci is expressed by the control model Ca (that is, system 1). Learning based on the causal relationship between the control items Ci via the parameter) and the stability evaluation Se of the operation performed by the device to be controlled by the control performed by the NN data Nd of each sensor processing unit 31.
  • the unit 30a may be derived, or may be set by a person or via a network.
  • the NN data Nd set in the plurality of AIs is ordered in the environment in which the system 1 operates. It is possible to stand up and make integrated learning.
  • the control item Ci (in other words, in other words, which is the basis for maintaining a stable operating state in the environment in which the system 1 operates) is used. Since the critical control item Ci) will function properly in advance, the other control item Ci will be able to function more appropriately, and as a result, it will be easier to adapt multiple AIs after integration to the system 1.
  • the AI after integration into the system 1 can adapt and change the trained model acquired by the learning before the integration by the integrated learning to the system 1. It can be said that the AI whose control is changed according to the system 1 has higher robustness in controlling than before the integration into the system 1.
  • the control unit 30 causes the integrated learning to be performed again.
  • the system 1 can continue a stable operating state.
  • the learning unit 30a processes the NN data Nd of the sensor processing unit 31 that handles the sensor 32 that has been added or changed so as to preferentially perform integrated learning, so that the system 1 operates stably again. It can be expected to make it. This leads to the improvement of the robustness in the control by the AI whose control is changed according to the system 1.
  • the system 1 can be stably operated by the integrated learning of the changed NN data Nd by the control unit 30. It will be possible. At this time, not only the changed NN data Nd but also other NN data Nd may be integratedly learned based on the control priority P.
  • the control priority P may be derived again by the control unit 30, or may already be derived.
  • the control unit 30 causes the in-vehicle device Vd or the sensor processing unit 31 in the abnormal state.
  • the possibility of continuously operating the system 1 can be obtained. This improves the robustness of control for the operation of the device in the system 1.
  • the learning unit 30a derives the priority P for the control performed by each NN data Nd and uses it for integrated learning, but regarding at least one of the operation of the in-vehicle device Vd itself and the in-vehicle device Vd.
  • the priority P may be derived and used for integrated learning.
  • the second embodiment can be applied. It is possible.
  • the control unit 30 derives the priority P for causing the single NN data Nd (that is, AI) used in each sensor processing unit 31 to perform integrated learning as described above, and derives the priority.
  • the learning of each NN data Nd is executed based on P.
  • AI that is, NN data Nd
  • AI which is a learned model for controlling the in-vehicle device Vd to be controlled by learning before being integrated into the system 1
  • the integrated learning can be performed in an orderly manner, and as a result, the effect of more appropriately controlling the AI of each sensor processing unit 31 can be obtained. This leads to improving the robustness of the control (in other words, autonomous control) acquired by AI through learning.
  • control priority P is derived using the control model Ca expressing each control item Ci.
  • control priority P is derived based on the learning process in each control item Ci.
  • FIG. 25 is a schematic diagram for explaining a process of determining a control priority P through integrated learning to NN data Nd in the third embodiment of the present disclosure.
  • the learning unit 30a can acquire information regarding learning before integration of the NN data Nd held by each sensor processing unit 31 into the vehicle 2.
  • the information regarding the learning before the integration of the NN data Nd into the vehicle 2 is the information regarding the learning process of the NN data Nd in various driving scenes Ds, for example, the control of each driving scene Ds in each NN data Nd.
  • Various evaluation items such as the degree of contribution, the degree of convergence of learning, or the degree of variation in the learning process when the seed used for learning is changed can be mentioned.
  • the degree of contribution of control for each driving scene Ds is an influence on the driving of the vehicle 2, for example, when the vehicle-mounted device Vd to be controlled by the NN data Nd operates appropriately (for example, control of the vehicle-mounted device Vd). And when the result is close to the teacher data Td corresponding to the driving scene Ds), or when it does not operate properly (for example, the control of the in-vehicle device Vd and the result are the teacher data corresponding to the driving scene Ds). It can be derived from (when it is not close to Td) and the stability evaluation of running after control.
  • the degree of convergence of learning can be derived from, for example, the number of transmissions of control signals Cs until it is determined that learning has converged, the number of teacher data (or the number of data sets) used for learning, and the like.
  • the degree of variation in the learning process when the seeds used for learning are changed can be derived from, for example, the relationship between the number of seeds and the variation in the learning process.
  • the evaluation unit 30e evaluates the NN data Nd of each sensor processing unit 31 before integration into the vehicle 2 with the above-mentioned evaluation items for each driving scene Ds.
  • the above-mentioned evaluation items are the degree of contribution of control, the degree of convergence of learning, and the degree of variation in the learning process.
  • the evaluation unit 30e derives the evaluation value Val for each driving scene Ds based on the evaluation result, and extracts the one having a high evaluation value Val. In the process of extracting the evaluation value Val, the weighting may be changed for each evaluation item.
  • priority P is set in descending order of the evaluation value Val, and integrated learning is performed. Further, for other driving scenes Ds, evaluation, extraction and determination of priority P are performed in the same manner, and integrated learning is performed on each NN data Nd.
  • the evaluation unit 30e provides information on, for example, the learning process before integration in the NN data Nd of each sensor processing unit 31.
  • the evaluation value Val is derived based on the degree of contribution of control in each NN data Nd
  • the control unit 30 determines each control item based on the derived evaluation value Val. It is possible to determine the priority P of the control of Ci and perform integrated learning. It is also possible to derive the control priority P in combination with the method of the second embodiment.
  • control item Ci extracted in the third embodiment is further combined to perform integrated learning.
  • FIG. 26 is a schematic diagram for explaining another process of determining the priority P of control through integrated learning to NN data Nd in the fourth embodiment of the present disclosure.
  • the learning unit 30a has four control items Ci, the buffer control Cc, the drive control Dc, the braking control Bc, and the steering control Sc, among the control items Ci carried by the various sensor processing units 31 integrated in the vehicle 2. Was extracted.
  • the learning unit 30a combines two or more of the four extracted control items Ci (that is, drive control Dc, braking control Bc, steering control Sc, and buffer control Cc) on a trial basis. Have students perform integrated learning.
  • the learning unit 30a sets the two control item Cis to be combined as the additional learning support mode for the six patterns in which two of the four extracted control item Cis are combined, and other control items.
  • Ci performs trial integrated learning as a mode that does not support additional learning.
  • the evaluation unit 30e evaluates the learning process when trial integrated learning is performed on the combined 6 patterns.
  • the evaluation of the learning process for example, the degree of variation in the output of each NN data Nd (that is, the control signal Cs), the stability evaluation Se of the control shown in FIGS. 19 to 21, and the like can be considered.
  • the learning unit 30a determines the NN data Nd to be preferentially combined to perform integrated learning, and the learning of each combined NN data Nd converges. Let them perform integrated learning.
  • the learning unit 30a supports additional learning by changing the order of the two control item Cis to be combined for 12 patterns considering the order of the combination of two of the four extracted control item Cis.
  • the mode is set, and the other control item Ci is set as a mode that does not support additional learning, and trial integrated learning is performed.
  • the evaluation unit 30e evaluates the learning process when trial integrated learning is performed on the combined 12 patterns. Based on the evaluation in such trial integrated learning, the learning unit 30a determines the NN data Nd to be preferentially combined and performed integrated learning in consideration of the order, and the combined NN data Nd. Let the integrated learning be performed until the learning converges in sequence.
  • 26 (a) and 26 (b) are examples, and the learning unit 30a causes trial integrated learning to be performed by combining a plurality of NN data Nd from various control items Ci, and determines the priority P. You may have them perform integrated learning.
  • the learning unit 30a is, for example, an NN in a plurality of patterns in which the NN data Nd of each sensor processing unit 31 is combined. Trial integrated learning is performed with the data Nd, and the evaluation unit 30e evaluates these trial integrated learning. Then, the learning unit 30a can determine the priority P of the combination of the NN data Nd based on the evaluation by the evaluation unit 30e, and make it possible to perform the integrated learning of the NN data Nd. It is also possible to derive the control priority P in combination with the methods of the second and third embodiments.
  • the neural network according to the above-described embodiments 1 to 4 is generally composed of an input layer composed of a plurality of neurons, an intermediate layer (hidden layer) composed of a plurality of neurons, and an output layer composed of a plurality of neurons.
  • the intermediate layer may be one layer or two or more layers.
  • FIG. 27 is a schematic diagram for explaining an example of a three-layer neural network. For example, in the case of a three-layer neural network as shown in FIG. 27, when a plurality of inputs are input to the input layer (X1-X3), the value is multiplied by the weight W1 (w11-w16) to form an intermediate layer (w11-w16).
  • FIG. 28 is a hardware configuration example for implementing the technique of the present disclosure shown in each of the above-described embodiments and modifications.
  • the hardware is composed of at least a CPU which is an arithmetic unit, an auxiliary storage device which is a storage device such as a memory, and a main storage device such as a hard disk or an optical disk.
  • the hardware configuration is not limited to the example of FIG. 28. Further, it may be provided with a communication device for connecting to an external network.
  • the AI integrated system receives detection information indicating the characteristics of the environment in which the controlled device operates as input via at least one of a sensor and an external network, and generates a plurality of control signals for controlling the controlled device. It includes a control unit that selects one of the trained models based on the input detection information, and a sensor processing unit that controls the device to be controlled by using the selected trained model. The control unit may select one of the plurality of trained models based on at least one of the input detection information and the control signal.
  • the AI integrated system receives detection information indicating the characteristics of the environment in which the controlled device operates as input via at least one of a sensor and a communication network, and generates a control signal for controlling the controlled device.
  • Learning to make the trained model additionally trained for each of the trained model, the plurality of sensor processing units that are controlled by using the trained model corresponding to each of the plurality of controlled devices, and the plurality of sensor processing units. It has a part.
  • the learning unit may have at least one of the plurality of sensor processing units perform additional learning.
  • the AI integrated device generates a plurality of control signals for controlling the device to be controlled by inputting detection information indicating the characteristics of the environment in which the device to be controlled is operated via at least one of a sensor and an external network. It is equipped with a control unit that selects and controls one of the trained models of the above based on the input detection information.
  • the control unit may select one of the plurality of trained models based on at least one of the input detection information and the control signal.
  • the AI integrated device is controlled by inputting detection information indicating the characteristics of the environment in which the controlled device operates, which corresponds to each of the plurality of controlled devices, as input via at least one of the sensor and the communication network. It is provided with a control unit that preferentially performs additional learning for at least one of the trained models that generate a control signal for controlling the device.
  • the AI integrated program generates a plurality of control signals for controlling the controlled device by inputting detection information indicating the characteristics of the environment in which the controlled device operates as an input via at least one of a sensor and an external network. From the trained models of, select one that is suitable for the environment based on the input detection information, and control the device to be controlled.
  • the AI integrated program may select one of a plurality of trained models based on at least one of the input detection information and the control signal.
  • the AI integrated program is controlled by inputting detection information indicating the characteristics of the environment in which the controlled device operates, which corresponds to each of the plurality of controlled devices, as input via at least one of the sensor and the communication network. Priority is given to additional training for at least one of the trained models that generate a control signal for controlling the device.
  • 1 AI integrated system 2 vehicles, 3 car bodies, 4 tires, 5 doors, 6 headlights, 7 electronic mirrors, 8 in-vehicle cameras, 9 radars, 10 transmissions, 11 drive devices, 12 braking devices, 13 steering devices, 14 shock absorbers. , 15 UI device, 16 recognition device, 17 transmission device, 21 drive control unit, 22 braking control unit, 23 steering control unit, 24 buffer control unit, 25 UI control unit, 26 recognition control unit, 27 transmission control unit, 30 supervision Unit, 30e evaluation unit, 30s selection unit, 30a learning unit, 30m storage unit, 31,31A, 31B, ..., 31N sensor processing unit, 32,32A, 32B, ..., 32N sensor, 40 in-vehicle network, 41 communication device, 42 signal transmission path, 101 subsystem, Nd, NdA, NdA1, NdA2, ..., NdB, NdB1, ..., NdNn NN data, Lm learner, Si detection information, Cs control signal, Sw.
  • Switching instruction Rs reselection instruction, AL additional learning compatible mode, NL additional learning non-compatible mode, Ds, DsA, DsB, ..., DsN driving scene, Ca, CaA, CaB, ..., CaN control model, Se Stability evaluation, P control priority, Ci control items, Dc drive control, Bc braking control, Sc steering control, Cc buffer control, Ui UI control, Rc recognition control, Tc transmission control, Ia controllable area, Ra robust control area , Na Non-adaptive area.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Traffic Control Systems (AREA)

Abstract

An AI integration system (1) is provided with an integration unit (30) and a sensor processing unit (31). Among a plurality of learned models (Nd) that receive input of a detection information (Si), which is input via a sensor and/or an external network and indicates a characteristic of an environment in which a device to be controlled operates, and that generate a control signal (Cs) for controlling the device to be controlled, the integration unit selects one leaned model on the basis of the detection information (Si) and/or the generated control signal (Cs). The sensor processing unit uses the selected learned model (Nd) to control the device to be controlled.

Description

AI統合システム、AI統合装置及びAI統合プログラムAI integrated system, AI integrated device and AI integrated program
本開示は、種々の入力情報をもとに制御を行う複数のAI(Artificial Intelligence)を統合したAI統合システムと、そこで用いられるAI統合装置及びAI統合プログラムに関する。 The present disclosure relates to an AI integrated system that integrates a plurality of AIs (Artificial Intelligence) that control based on various input information, and an AI integrated device and an AI integrated program used therein.
近年、自動車を始めとする車両の自動運転技術の発展に伴い、人体検知を行う車載カメラ8の制御やステアリング装置の制御にAIを用いるなど、様々な車載装置にAIが搭載されている。 In recent years, with the development of automatic driving technology for vehicles such as automobiles, AI is installed in various in-vehicle devices such as using AI for controlling an in-vehicle camera 8 for detecting a human body and controlling a steering device.
このような状況にあり、AIを搭載した複数の下位の装置に対して上位の装置から制御を行う技術が開示されている(例えば、特許文献1)。特許文献1の手法によれば、利用者がGUIを介して、複数の機器に搭載されたニューラルネットワークを簡易に構築したり、上位の装置からニューラルネットワークを搭載した複数の装置を制御したりすることを可能とする。 Under such circumstances, a technique for controlling a plurality of lower-level devices equipped with AI from higher-level devices is disclosed (for example, Patent Document 1). According to the method of Patent Document 1, a user can easily construct a neural network mounted on a plurality of devices via a GUI, or control a plurality of devices equipped with a neural network from a higher-level device. Make it possible.
特開2018-14060号JP-A-2018-14060
しかしながら、先行技術によれば、車両の種別、仕様、及び、購入者の要望によるオプションの選択など、完成車のバリエーションが多様であるため、1つの車両にAIを搭載した複数の装置が統合された完成車において、統合前に学習過程を経た複数のAIが適切に連動して制御を行えるかどうか、信頼性を確保できないおそれがある。 However, according to the prior art, there are various variations of the finished vehicle such as vehicle type, specifications, and option selection according to the purchaser's request, so multiple devices equipped with AI in one vehicle are integrated. In a completed vehicle, it may not be possible to ensure reliability as to whether or not a plurality of AIs that have undergone a learning process before integration can be appropriately interlocked and controlled.
本開示は、上述の問題を解決するためのものであって、種々の制御を行うAIが1つのシステムに統合された後に適切に制御できるようにすることを目的とする。 The present disclosure is to solve the above-mentioned problems, and an object of the present invention is to enable AI that performs various controls to be appropriately controlled after being integrated into one system.
AI統合システムは、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び外部ネットワークの少なくともいずれかを介して入力とし、制御対象の装置を制御するための制御信号を生成する複数の学習済みモデルのうちから、検知情報又は生成された制御信号の少なくともいずれかをもとに、1つを選択する統括部と、選択された学習済みモデルを用いて、制御対象の装置を制御するセンサ処理部とを備える。 The AI integrated system receives detection information indicating the characteristics of the environment in which the controlled device operates as input via at least one of a sensor and an external network, and generates a plurality of control signals for controlling the controlled device. The device to be controlled is controlled using the control unit that selects one of the trained models based on at least one of the detection information or the generated control signal, and the selected trained model. It is equipped with a sensor processing unit.
また、AI統合システムは、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び通信ネットワークの少なくともいずれかを介して入力とし、制御対象の装置を制御するための制御信号を生成する学習済みモデルと、複数の制御対象の装置の各々と対応して学習済みモデルを用いて制御する複数のセンサ処理部と、複数のセンサ処理部の各々に対し、学習済みモデルに追加学習させる学習部とを備える。 Further, the AI integrated system receives detection information indicating the characteristics of the environment in which the controlled device operates as input via at least one of a sensor and a communication network, and generates a control signal for controlling the controlled device. Learning to make the trained model additionally trained for each of the trained model, the plurality of sensor processing units that are controlled by using the trained model corresponding to each of the plurality of controlled devices, and the plurality of sensor processing units. It has a part.
また、AI統合装置は、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び外部ネットワークの少なくともいずれかを介して入力として制御対象の装置を制御するための制御信号を生成する複数の学習済みモデルのうちから、検知情報又は生成された制御信号の少なくともいずれかをもとに、1つを選択する統括部を備える。 Further, the AI integrated device generates a plurality of control signals for controlling the device to be controlled by inputting detection information indicating the characteristics of the environment in which the device to be controlled is operated via at least one of a sensor and an external network. It is provided with a control unit that selects one from the trained models of the above, based on at least one of the detection information and the generated control signal.
また、AI統合装置は、複数の制御対象の装置の各々と対応した、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び通信ネットワークの少なくともいずれかを介して入力として制御対象の装置を制御するための制御信号を生成する各々の学習済みモデルの少なくともいずれかに対し、優先的に追加学習させる統括部を備える。 Further, the AI integrated device is controlled by inputting detection information indicating the characteristics of the environment in which the controlled device operates, which corresponds to each of the plurality of controlled devices, as input via at least one of the sensor and the communication network. It is provided with a control unit that preferentially performs additional learning for at least one of the trained models that generate a control signal for controlling the device.
また、AI統合プログラムは、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び外部ネットワークの少なくともいずれかを介して入力として制御対象の装置を制御するための制御信号を生成する複数の学習済みモデルのうちから、検知情報又は生成された制御信号の少なくともいずれかをもとに、環境に適する1つを選択し、制御対象の装置を制御させる。 Further, the AI integrated program receives detection information indicating the characteristics of the environment in which the controlled device operates as an input via at least one of a sensor and an external network to generate a control signal for controlling the controlled device. From the trained models of the above, one suitable for the environment is selected based on at least one of the detection information and the generated control signal, and the device to be controlled is controlled.
また、AI統合プログラムは、複数の制御対象の装置の各々と対応した、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び通信ネットワークの少なくともいずれかを介して入力として制御対象の装置を制御するための制御信号を生成する各々の学習済みモデルの少なくともいずれかに対し、優先的に追加学習させる。 Further, the AI integrated program is controlled by inputting detection information indicating the characteristics of the environment in which the controlled device operates, which corresponds to each of the plurality of controlled devices, as input via at least one of the sensor and the communication network. Priority is given to additional training for at least one of the trained models that generate control signals for controlling the device.
本開示によれば、複数のAIが1つのシステムに統合された後、従来よりも適切に制御させられる効果が得られる。 According to the present disclosure, after a plurality of AIs are integrated into one system, the effect of being controlled more appropriately than before can be obtained.
本開示の実施の形態1における、AI統合システムを説明するためのシステム構成図である。It is a system block diagram for demonstrating the AI integrated system in Embodiment 1 of this disclosure. AI統合システムに統合される種々のAIを示すシステム構成図である。FIG. 3 is a system configuration diagram showing various AIs integrated into an AI integrated system. 別のAI統合システムを説明するためのシステム構成図である。It is a system block diagram for demonstrating another AI integrated system. 図3に示すAI統合システムに統合される種々のAIを示すシステム構成図である。FIG. 3 is a system configuration diagram showing various AIs integrated into the AI integrated system shown in FIG. 図3に示すAI統合システムの統括部を説明するためのシステム構成図である。It is a system configuration diagram for demonstrating the control part of the AI integrated system shown in FIG. 図3に示すAI統合システムの構成を説明するためのシステム構成図である。It is a system configuration diagram for demonstrating the configuration of the AI integrated system shown in FIG. センサ処理部の構成を説明するための模式図である。It is a schematic diagram for demonstrating the structure of a sensor processing part. AIに行わせる学習と、学習によって獲得される学習済みモデルとを説明するための図である。It is a figure for demonstrating the learning to perform AI, and the trained model acquired by learning. 統括部の評価部が、検知情報及び制御信号のデータセットをもとにNNデータを評価する過程を説明するための模式図である。It is a schematic diagram for explaining the process which the evaluation part of the control part evaluates NN data based on the data set of detection information and control signal. AI統合システムにおける統括部での処理を説明するためのフローチャート図である。It is a flowchart for demonstrating the processing in the control part in the AI integrated system. 統括部の第1の変形例を説明するための模式図である。It is a schematic diagram for demonstrating the 1st modification of a control part. 統括部の第2の変形例を説明するための模式図である。It is a schematic diagram for demonstrating the 2nd modification of a control part. 本開示の実施の形態2における、AIでの制御で扱う検知情報及び制御信号を2次元で表現する制御領域を説明するための模式図である。It is a schematic diagram for demonstrating the control area which expresses the detection information and the control signal which are handled by the control by AI in 2D in Embodiment 2 of this disclosure. 各々の車載装置での制御における相互の影響を説明するための模式図である。It is a schematic diagram for demonstrating the mutual influence in the control in each in-vehicle device. AIによる制御が有するロバスト性を説明するための模式図である。It is a schematic diagram for demonstrating the robustness which control by AI has. AIでの制御におけるロバスト性が拡張された状態を説明するための模式図である。It is a schematic diagram for demonstrating the state which the robustness in the control by AI is expanded. 走行シーンと対応したAIの制御領域の重複部分を示す模式図である。It is a schematic diagram which shows the overlap part of the control area of AI corresponding to a driving scene. システムとしての車両と、種々の車載装置を制御するサブシステムとしてのAIと、さらに統括部との関係を説明するための模式図である。It is a schematic diagram for demonstrating the relationship between the vehicle as a system, AI as a subsystem which controls various in-vehicle devices, and further, a control unit. 制御の優先度を考慮したときの各々のNNデータにおける統合学習の収束度合いを説明するための模式図である。It is a schematic diagram for demonstrating the degree of convergence of integrated learning in each NN data when the priority of control is considered. 制御の優先度を考慮したときの各々のNNデータにおける統合学習の収束度合いを説明するための別の模式図である。It is another schematic diagram for demonstrating the degree of convergence of integrated learning in each NN data when the priority of control is considered. 制御の優先度を考慮したときの各々のNNデータにおける統合学習の収束度合いを説明するためのさらに別の模式図である。It is still another schematic diagram for demonstrating the degree of convergence of integrated learning in each NN data when the priority of control is taken into consideration. 車両に関わる物理量が制御項目に影響を及ぼすメカニズムの一例を説明するための模式図である。It is a schematic diagram for demonstrating an example of the mechanism that a physical quantity related to a vehicle influences a control item. AI統合システムに統合された複数のAIを統合学習させる過程を説明するための模式図である。It is a schematic diagram for demonstrating the process of making a plurality of AI integrated into an AI integrated system integrated learning. AI統合システムにおける統括部での処理を説明するためのフローチャート図である。It is a flowchart for demonstrating the processing in the control part in the AI integrated system. 本開示の実施の形態3における、NNデータへの統合学習を通じて制御の優先度を決定する過程を説明するための模式図である。It is a schematic diagram for demonstrating the process of determining a control priority through integrated learning to NN data in Embodiment 3 of this disclosure. 本開示の実施の形態4における、NNデータへの統合学習を通じて制御の優先度を決定する別の過程を説明するための模式図である。It is a schematic diagram for demonstrating another process of determining a control priority through integrated learning to NN data in Embodiment 4 of this disclosure. 3層のニューラルネットワークの例を説明するための模式図である。It is a schematic diagram for demonstrating an example of a three-layer neural network. 本開示の技術を実施するためのハードウェア構成である。This is a hardware configuration for implementing the technology of the present disclosure.
実施の形態1.
本開示では、AI(Artificial Intelligence)を“学習モデル”、“学習済みモデル”又は“学習器”と記載する。また、本開示では、“学習器”とは、既知の学習手法を用いることにより入力情報に対して出力情報を決定できるソフトウェア又はLSIで構成される制御装置を指し、“学習モデル”とは、前述の“学習器”を構成する入力情報と出力情報との対応付けが決まる前段階のソフトウェア又はLSIを指し、“学習済みモデル”とは、前述の“学習器”を構成する入力情報と出力情報との対応付けが決まった段階のソフトウェア又はLSIを指すが、学習過程を経て知能を獲得するものであれば上述の構成以外のものであってもよい。
Embodiment 1.
In this disclosure, AI (Artificial Intelligence) is referred to as a "learning model", a "learned model", or a "learner". Further, in the present disclosure, the "learning device" refers to a control device composed of software or LSI that can determine output information with respect to input information by using a known learning method, and the "learning model" is used. The "learned model" refers to the software or LSI in the previous stage where the correspondence between the input information and the output information constituting the above-mentioned "learner" is determined, and the "learned model" refers to the input information and the output constituting the above-mentioned "learner". It refers to software or LSI at the stage where the correspondence with information is determined, but it may have a configuration other than the above-mentioned configuration as long as it acquires intelligence through a learning process.
図1は、本開示の実施の形態1における、AI統合システム1を説明するためのシステム構成図である。ここでいうAI統合システム1とは、AIを用いて制御する装置を複数統合させて動作させることにより、用途に応じた処理又は動作を行うシステムを指す。 FIG. 1 is a system configuration diagram for explaining the AI integrated system 1 in the first embodiment of the present disclosure. The AI integrated system 1 as used herein refers to a system that performs processing or operation according to an application by integrating and operating a plurality of devices controlled by using AI.
図1に、AI統合システム1の例を示す。AI統合システム1は、産業用ロボットM1,M2及びコンベアー装置M3を統合したシステムである。 FIG. 1 shows an example of the AI integrated system 1. The AI integrated system 1 is a system in which the industrial robots M1 and M2 and the conveyor device M3 are integrated.
産業用ロボットM1は腕節部Arm1、捕捉部Hand1及び撮像部Camera1を備えており、撮像部Camera1で撮像しながらコンベアーによって運ばれる複数の部品類Parts1を識別し、腕節部Arm1及び捕捉部Hand1を駆動させて部品類Parts1を補足する。また、産業用ロボットM2は腕節部Arm2、捕捉部Hand2及び撮像部Camera2を備えており、撮像部Camera2で撮像しながらコンベアーによって運ばれる複数の部品類Parts2を識別し、腕節部Arm2及び捕捉部Hand2を駆動させて部品類Parts2を補足する。また、コンベアー装置M3は、コンベアーConveyor及び切り替え部Switchを備えており、コンベアーConveyorは1つの通路が途中で2つの通路に分岐して複数の部品類Partsを運び、切り替え部SwitchはコンベアーConveyorの分岐点において、運ばれてくる複数の部品類Partsを所定条件のもと、産業用ロボットM1又は産業用ロボットM2のいずれかの側に切り替えて流す。 The industrial robot M1 includes an arm node Arm1, a capture unit Hand1, and an image pickup unit Camera1 to identify a plurality of parts Parts1 carried by a conveyor while imaging with the image pickup unit Camera1 and identify the arm node arm1 and the capture unit Hand1. Is driven to supplement the parts Parts1. Further, the industrial robot M2 includes an arm node Arm2, a capture unit Hand2, and an image pickup unit Camera2, and identifies a plurality of parts Parts2 carried by a conveyor while taking an image with the image pickup unit Camera2, and identifies the arm node arm2 and the capture unit. The part Hand2 is driven to supplement the parts Parts2. Further, the conveyor device M3 includes a conveyor Conveyor and a switching unit Switch. In the conveyor Conveyor, one passage is branched into two passages in the middle to carry a plurality of parts Parts, and the switching unit Switch is a branch of the conveyor Conveyor. At the point, the plurality of parts to be carried are switched to either side of the industrial robot M1 or the industrial robot M2 under predetermined conditions and flown.
図2は、AI統合システム1に統合される種々のAIを示すシステム構成図である。図2(a)に示すように、産業用ロボットM1の腕節部Arm1はAIであるAI1aによって制御され、捕捉部Hand1はAIであるAI1hによって制御され、撮像部Camera1はAIであるAI1cによって制御される。また、産業用ロボットM2の腕節部Arm2はAIであるAI2aによって制御され、捕捉部Hand2はAIであるAI2hによって制御され、撮像部Camera2はAIであるAI2cによって制御される。また、コンベアー装置M3はAIであるAI3によって制御される。産業用ロボットM1のAI1a、AI1h及びAI1c、産業用ロボットM2のAI2a,AI2h及びAI2c、並びに、コンベアー装置M3のAI3は、それぞれ、産業用ロボットM1,M2及びコンベアー装置M3が連動できるように制御を行う。 FIG. 2 is a system configuration diagram showing various AIs integrated into the AI integrated system 1. As shown in FIG. 2A, the arm node Arm1 of the industrial robot M1 is controlled by the AI AI1a, the capture unit Hand1 is controlled by the AI AI1h, and the imaging unit Camera1 is controlled by the AI AI1c. Will be done. Further, the arm node Arm2 of the industrial robot M2 is controlled by AI2a, which is AI, the capture unit Hand2 is controlled by AI2h, which is AI, and the imaging unit Camera2 is controlled by AI2c, which is AI. Further, the conveyor device M3 is controlled by AI3 which is AI. AI1a, AI1h and AI1c of the industrial robot M1, AI2a, AI2h and AI2c of the industrial robot M2, and AI3 of the conveyor device M3 are controlled so that the industrial robots M1, M2 and the conveyor device M3 can be interlocked with each other. conduct.
ところで、産業用ロボットM1のAI1a、AI1h及びAI1c、産業用ロボットM2のAI2a,AI2h及びAI2c、並びに、コンベアー装置M3のAI3は、各々のAIが搭載される産業用ロボットM1,M2及びコンベアー装置M3が統合される前に個別に制御に関する学習を行っており、AI統合システム1として統合された後に各々の装置が実際に稼働する環境においては制御に関する学習を行っていない。そのため、システムとして各々の装置が適切に動作する保証が持てないことが危惧される。 By the way, AI1a, AI1h and AI1c of the industrial robot M1, AI2a, AI2h and AI2c of the industrial robot M2, and AI3 of the conveyor device M3 are the industrial robots M1, M2 and the conveyor device M3 on which the respective AIs are mounted. Is individually learned about control before being integrated, and is not learned about control in the environment where each device actually operates after being integrated as AI integrated system 1. Therefore, there is a concern that there is no guarantee that each device will operate properly as a system.
本開示では、上述のような、複数のAIを統合するシステムにおいて、各々のAIがシステムに適した制御を行うための学習を行わせるAIについて説明する。図2(b)では、このような複数のAIに学習を行わせるAIをAIsvとして示している。 In the present disclosure, in a system that integrates a plurality of AIs as described above, an AI that causes each AI to perform learning for performing control suitable for the system will be described. In FIG. 2B, an AI that causes a plurality of such AIs to perform learning is shown as AIsv.
次に、AI統合システム1の別の例として、自動運転機能又は運転支援機能を実装した自動車について説明する。 Next, as another example of the AI integrated system 1, a vehicle equipped with an automatic driving function or a driving support function will be described.
図3は、別のAI統合システム1を説明するためのシステム構成図である。図3では、AI統合システム1として車両2の構成を示している。車両2は、ここでは自動車(より具体的には、完成車)とする。以下、AI統合システム1を単に“システム”とも記載する。 FIG. 3 is a system configuration diagram for explaining another AI integrated system 1. FIG. 3 shows the configuration of the vehicle 2 as the AI integrated system 1. The vehicle 2 is referred to as an automobile (more specifically, a completed vehicle) here. Hereinafter, the AI integrated system 1 is also simply referred to as a “system”.
車両2のボディである車体3には、タイヤ4、ドア5、ヘッドライト6、電子ミラー7、車載カメラ8、レーダ9及びトランスミッション10が備え付けられる。車載カメラ8及びレーダ9は、車両2の周辺の対象物を検出するための装置である。車載カメラ8は撮像装置である。レーダ9にはLiDAR(light detection and ranging)などのレーザーレーダ又はミリ波レーダなどの電磁波が用いられる。 The vehicle body 3, which is the body of the vehicle 2, is equipped with a tire 4, a door 5, a headlight 6, an electronic mirror 7, an in-vehicle camera 8, a radar 9, and a transmission 10. The in-vehicle camera 8 and the radar 9 are devices for detecting an object around the vehicle 2. The in-vehicle camera 8 is an image pickup device. For the radar 9, an electromagnetic wave such as a laser radar such as LiDAR (light detection and ranging) or a millimeter wave radar is used.
駆動装置11は、エンジン又はモーターなど、車両2を走行させるための駆動力を発生する装置である。制動装置12は、機械的ブレーキ又は電力回生ブレーキなど、車両2を減速又は停止させるための制動力を発生する装置又は減速機構である。操舵装置13は、車両2の進行方向を変えるためのステアリング装置である。緩衝装置14は、車両2に生じる振動及び慣性力などの応力を緩和及び緩衝するための減衰力を発生するサスペンション装置である。
駆動装置11及び制動装置12は、タイヤ4に駆動力及び制動力を与える。また、操舵装置13は、タイヤ4の向きを変えることで進行方向に向かう抗力を与える。また、緩衝装置14は、車両2と路面との間に生じる応力を緩和するためにタイヤ4と車体3との間に減衰力を与える。
The drive device 11 is a device that generates a driving force for driving the vehicle 2, such as an engine or a motor. The braking device 12 is a device or a deceleration mechanism that generates a braking force for decelerating or stopping the vehicle 2, such as a mechanical brake or a power regenerative brake. The steering device 13 is a steering device for changing the traveling direction of the vehicle 2. The shock absorber 14 is a suspension device that generates a damping force for relaxing and cushioning stresses such as vibration and inertial force generated in the vehicle 2.
The driving device 11 and the braking device 12 apply a driving force and a braking force to the tire 4. Further, the steering device 13 gives a drag force toward the traveling direction by changing the direction of the tire 4. Further, the shock absorber 14 applies a damping force between the tire 4 and the vehicle body 3 in order to relieve the stress generated between the vehicle 2 and the road surface.
UI装置15は、メーター機器を含めたインストルメントパネル、カーナビゲーション又は情報端末装置など、搭乗者に対して車両2の周囲を表示したり、車両2の周辺の状況を通知したり、車両2に搭載された種々の装置類の操作を可能としたりする、UI(ユーザーインターフェース、User Interface)のための装置である。
ドア5、ヘッドライト6及び電子ミラー7は、UI装置15を介して、搭乗者からの操作を受け付けて動作する。
The UI device 15 displays the surroundings of the vehicle 2 to the occupants, such as an instrument panel including a meter device, a car navigation system, or an information terminal device, notifies the situation around the vehicle 2, and informs the vehicle 2. It is a device for UI (User Interface, User Interface) that enables operation of various mounted devices.
The door 5, the headlight 6, and the electronic mirror 7 operate by receiving an operation from the passenger via the UI device 15.
認識装置16は、車載カメラ8、レーダ9及び図示しないGPS(Global Positioning System)装置などを個別又は連動させて制御することにより、車両2内外の対象物を検出及び認識したり、車両2の位置情報を取得したりする。なお、認識装置16がヘッドライト6を制御することにより、車載カメラ8での撮像による物体などの検出及び認識の精度を向上させるようにしてもよい。
伝達装置17は、変速機及び差動装置など駆動伝達に関わる経路であるトランスミッション10を制御する。
The recognition device 16 detects and recognizes an object inside and outside the vehicle 2 and detects and recognizes an object inside and outside the vehicle 2 by controlling an in-vehicle camera 8, a radar 9, a GPS (Global Positioning System) device (not shown), and the like individually or in conjunction with each other. Get information. The recognition device 16 may control the headlight 6 to improve the accuracy of detection and recognition of an object or the like by imaging with the vehicle-mounted camera 8.
The transmission device 17 controls a transmission 10 which is a path related to drive transmission such as a transmission and a differential device.
上述の駆動装置11、制動装置12、操舵装置13、緩衝装置14、UI装置15、認識装置16及び伝達装置17は同一の装置として搭載されてもよい。また、上述の装置以外にも、車両2に搭載される他の装置をAI統合システム1に含めてもよい。 The above-mentioned drive device 11, braking device 12, steering device 13, shock absorber 14, UI device 15, recognition device 16, and transmission device 17 may be mounted as the same device. In addition to the above-mentioned devices, other devices mounted on the vehicle 2 may be included in the AI integrated system 1.
図4は、図3に示すAI統合システム1に統合される種々のAIを示すシステム構成図である。図4では、AI統合システム1は、駆動装置11、制動装置12、操舵装置13、緩衝装置14、UI装置15、認識装置16及び伝達装置17の各々を制御するためのAIが統合されている。各々のAIは機械学習を用いた学習器Lmであって、例えば、ニューラルネットワークを用いて学習させた学習済みモデルである。各々のAIは、ソフトウェア又はLSIで構成される。 FIG. 4 is a system configuration diagram showing various AIs integrated into the AI integrated system 1 shown in FIG. In FIG. 4, the AI integrated system 1 integrates an AI for controlling each of the drive device 11, the braking device 12, the steering device 13, the shock absorber 14, the UI device 15, the recognition device 16, and the transmission device 17. .. Each AI is a learner Lm using machine learning, and is, for example, a trained model trained using a neural network. Each AI is composed of software or LSI.
駆動装置11を制御するAIを駆動制御部21、制動装置12を制御するAIを制動制御部22、操舵装置13を制御するAIを操舵制御部23、緩衝装置14を制御するAIを緩衝制御部24、UI装置15を制御するAIをUI制御部25、認識装置16を制御するAIを認識制御部26、及び、伝達装置17を制御するAIを伝達制御部27とする。 The AI that controls the drive device 11 is the drive control unit 21, the AI that controls the braking device 12 is the braking control unit 22, the AI that controls the steering device 13 is the steering control unit 23, and the AI that controls the shock absorber 14 is the buffer control unit. 24, the AI that controls the UI device 15 is the UI control unit 25, the AI that controls the recognition device 16 is the recognition control unit 26, and the AI that controls the transmission device 17 is the transmission control unit 27.
図5は、図3に示すAI統合システム1の統括部30を説明するためのシステム構成図である。AI統合システム1は統括部30を備える。なお、統括部30はAI統合装置とも扱うものとする。また、統括部30はAI統合プログラム、AI統合回路及びAI統合データとも扱うものとする。 FIG. 5 is a system configuration diagram for explaining the control unit 30 of the AI integrated system 1 shown in FIG. The AI integrated system 1 includes a control unit 30. The control unit 30 is also treated as an AI integrated device. In addition, the control unit 30 shall also handle the AI integrated program, the AI integrated circuit, and the AI integrated data.
統括部30は、駆動制御部21、制動制御部22、操舵制御部23、緩衝制御部24、UI制御部25、認識制御部26及び伝達制御部27に対し、統合学習を行わせる。
また、統括部30は、車両2が走行する環境である走行シーンDsの推定、AI統合システム1に統合された各々のAIの制御の評価、各々のAIの制御に用いられる学習済みモデルの切り替えなどを行う。統括部30での処理の詳細については後述する。
The control unit 30 causes the drive control unit 21, the braking control unit 22, the steering control unit 23, the buffer control unit 24, the UI control unit 25, the recognition control unit 26, and the transmission control unit 27 to perform integrated learning.
Further, the control unit 30 estimates the driving scene Ds, which is the environment in which the vehicle 2 travels, evaluates the control of each AI integrated in the AI integrated system 1, and switches the trained model used for controlling each AI. And so on. The details of the processing in the control unit 30 will be described later.
なお、本開示では、車両2が置かれる環境の例として走行シーンDsを扱うが、ここでの環境とは、車両2の走行に関わる状態又は状況に限定されるものではなく、例えば、搭乗者の挙動、体調及び安全状況、並びに、車両2の経年劣化の状態など、車載装置の制御に関わるものを含む。 In this disclosure, the driving scene Ds is treated as an example of the environment in which the vehicle 2 is placed, but the environment here is not limited to the state or situation related to the driving of the vehicle 2, for example, the passenger. This includes those related to the control of the in-vehicle device, such as the behavior, physical condition and safety status of the vehicle 2, and the state of aging deterioration of the vehicle 2.
図6は、図3に示すAI統合システム1の構成を説明するためのシステム構成図である。AI統合システム1は、統括部30及び複数のセンサ処理部31A,31B,・・・,31Nを備える。統括部30は、複数のセンサ処理部31と接続して種々の信号を送受信する。 FIG. 6 is a system configuration diagram for explaining the configuration of the AI integrated system 1 shown in FIG. The AI integrated system 1 includes a control unit 30 and a plurality of sensor processing units 31A, 31B, ..., 31N. The control unit 30 connects to a plurality of sensor processing units 31 to transmit and receive various signals.
また、AI統合システム1には、車両2に搭載される種々のセンサ32A,32B,・・・,32N、車載ネットワーク40に接続する通信装置41、並びに、車両2に搭載される種々の車載装置Vdが接続される。これらのセンサ32、通信装置41及び車載装置Vdは信号伝送路42を介して繋がる。車載ネットワーク40は、車車間通信、地車間通信、並びに、社会インフラとして敷設された広域通信網又はインターネット網と繋がる。 Further, the AI integrated system 1 includes various sensors 32A, 32B, ..., 32N mounted on the vehicle 2, a communication device 41 connected to the vehicle-mounted network 40, and various vehicle-mounted devices mounted on the vehicle 2. Vd is connected. These sensors 32, the communication device 41, and the in-vehicle device Vd are connected via the signal transmission path 42. The in-vehicle network 40 is connected to vehicle-to-vehicle communication, ground-to-vehicle communication, and a wide-area communication network or an Internet network laid as social infrastructure.
図6では、センサ32Aに接続するセンサ処理部31をセンサ処理部31A、センサ32Bに接続するセンサ処理部31をセンサ処理部31Bとしているが、各々のセンサ32とセンサ処理部31との対応は1対1でなくとも良く、1対複数、複数対1、並びに、複数対複数であってもよい。 In FIG. 6, the sensor processing unit 31 connected to the sensor 32A is referred to as the sensor processing unit 31A, and the sensor processing unit 31 connected to the sensor 32B is referred to as the sensor processing unit 31B. It does not have to be one-to-one, and may be one-to-many, multiple-to-one, or multiple-to-many.
種々のセンサ32A,32B,・・・,32Nは、車両2に搭載される装置であって、例えば、走行速度情報、サスペンションに掛かる荷重情報、及び、温度情報など、車両2に関わる種々の情報を検知して出力する。 Various sensors 32A, 32B, ..., 32N are devices mounted on the vehicle 2, and various information related to the vehicle 2, such as running speed information, load information applied to the suspension, and temperature information, are used. Is detected and output.
例えば、複数あるセンサ32のいずれかは、車体3及び個々の車輪の少なくともいずれかにおいて、速度及び加速度、ロール・ピッチ・ヨーを含む角度及び角速度、振動の振幅及び周波数、並びに、正のトルク及び負のトルク(つまり、駆動力及び制動力)などの走行制御に関わる動的な情報と、車体3内部、車体3外部及び車体3そのものの少なくともいずれかにおいて、温度、湿度、照度及び重量などの走行条件に関わる静的な情報とを検知して、検知した情報を対応するセンサ処理部31に出力する。 For example, any of the plurality of sensors 32 may include velocity and acceleration, angles and angular velocities including roll pitch yaw, vibration amplitude and frequency, and positive torque and at least one of the vehicle body 3 and individual wheels. Dynamic information related to driving control such as negative torque (that is, driving force and braking force) and temperature, humidity, illuminance, weight, etc. in at least one of the inside of the vehicle body 3, the outside of the vehicle body 3, and the vehicle body 3 itself. It detects static information related to driving conditions and outputs the detected information to the corresponding sensor processing unit 31.
また、他のセンサ32のいずれかは、カメラ8での撮像、レーダ9による物体検知、並びに、例えばGPSを用いた位置検知及び通信デバイスを利用したデータ通信などの周囲の状況の認識に関わる情報を取得して、取得した情報を対応するセンサ処理部31に出力する。 Further, any of the other sensors 32 is information related to image pickup by the camera 8, object detection by the radar 9, and recognition of surrounding conditions such as position detection using GPS and data communication using a communication device. Is acquired, and the acquired information is output to the corresponding sensor processing unit 31.
種々の装置を統合するシステム1としての車両2は、上述した種々のセンサ32及び通信から得られる情報に基づいて自車両2の状態を認識し、認識した状態Stにおいて取るべき行動Actを関連付けたデータセット(行動Act,状態St)を扱うことにより、自車両2が置かれた環境とインタラクションをする。以下、種々の装置を全て統合した車両2を完成車と呼ぶ。 The vehicle 2 as a system 1 that integrates various devices recognizes the state of the own vehicle 2 based on the information obtained from the various sensors 32 and communication described above, and associates the action Act to be taken in the recognized state St. By handling the data set (behavior Act, state St), it interacts with the environment in which the own vehicle 2 is placed. Hereinafter, the vehicle 2 in which all the various devices are integrated is referred to as a completed vehicle.
自車両2の状態Stとは、上述のセンサ32及び通信を介して得られる、例えば、位置情報、目標との距離情報、速度情報などの種々の情報から導出される。以下、種々のセンサ32及び通信から得られる情報をまとめて検知情報Siとする。また、行動Actとは、車両2に搭載される種々の車載装置Vdに対する制御、並びに、搭乗者に対する種々の情報の通知又は提供を指す。 The state St of the own vehicle 2 is derived from various information obtained via the sensor 32 and communication described above, for example, position information, distance information to the target, speed information, and the like. Hereinafter, the information obtained from various sensors 32 and communication is collectively referred to as detection information Si. Further, the action Act refers to the control of various in-vehicle devices Vd mounted on the vehicle 2 and the notification or provision of various information to the passengers.
センサ処理部31は、制御対象の車載装置Vdと双方向の信号伝送が可能に接続している。センサ処理部31は、制御対象の車載装置Vd又は信号伝送路42に接続された他の装置に実装される。
センサ処理部31は、上述した種々のセンサ32及び通信を介して入力した検知情報Siをもとに制御信号Csを生成して、制御対象の車載装置Vdに対して生成した制御信号Csを送信する。センサ処理部31は、入力した検知情報Siと、車載装置Vdを制御するために生成した制御信号Csとを統括部30に対して逐次出力する。
The sensor processing unit 31 is connected to the vehicle-mounted device Vd to be controlled so as to be capable of bidirectional signal transmission. The sensor processing unit 31 is mounted on the vehicle-mounted device Vd to be controlled or another device connected to the signal transmission path 42.
The sensor processing unit 31 generates control signals Cs based on the various sensors 32 described above and the detection information Si input via communication, and transmits the generated control signals Cs to the vehicle-mounted device Vd to be controlled. do. The sensor processing unit 31 sequentially outputs the input detection information Si and the control signal Cs generated for controlling the in-vehicle device Vd to the control unit 30.
統括部30は、記憶部30mを備えており、入力した情報、導出した情報又は設定された情報など種々の情報を、一時的又は消去されるまで、記憶部30mに保持することができる。 The control unit 30 includes a storage unit 30m, and can hold various information such as input information, derived information, or set information in the storage unit 30m until it is temporarily or erased.
統括部30は、検知情報Si、制御信号Cs及び出力元のセンサ処理部31を関連付けた関連情報Riを保持する。統括部30に入力される検知情報Si及び制御信号Csには、出力元のセンサ処理部31を示す情報が付与されている。このことは、センサ処理部31が検知情報Si及び制御信号Csを出力するときに出力元の情報として付与したり、信号伝送路42上の装置が出力元のセンサ処理部31の情報と検知情報Si及び制御信号Csとを関連付けるために付与したりすることで可能となる。 The control unit 30 holds the related information Ri associated with the detection information Si, the control signal Cs, and the sensor processing unit 31 of the output source. Information indicating the sensor processing unit 31 of the output source is added to the detection information Si and the control signal Cs input to the control unit 30. This is given as output source information when the sensor processing unit 31 outputs the detection information Si and the control signal Cs, or the device on the signal transmission path 42 gives the information and detection information of the output source sensor processing unit 31. It is possible by adding it to associate Si and the control signal Cs.
このように、上述のデータセット(行動Act,状態St)は、検知情報Siと制御信号Csとを関連付けたデータセット(制御信号Cs,検知情報Si)として扱うことができる。 As described above, the above-mentioned data set (behavior Act, state St) can be treated as a data set (control signal Cs, detection information Si) in which the detection information Si and the control signal Cs are associated with each other.
本開示では、センサ処理部31が実装するAIの一例としてニューラルネットワークを扱う。センサ処理部31は、複数のニューラルネットワークデータを保持しており、これらの複数のニューラルネットワークデータを切り替えて設定し、制御対象の車載装置Vdの制御を行う。センサ処理部31に設定されたニューラルネットワークデータは、検知情報Siを入力すると、検知情報Siと対応した制御信号Csを生成する。以下、ニューラルネットワークデータをNNデータNdと記載する。 In the present disclosure, a neural network is treated as an example of AI implemented by the sensor processing unit 31. The sensor processing unit 31 holds a plurality of neural network data, switches and sets the plurality of neural network data, and controls the vehicle-mounted device Vd to be controlled. When the detection information Si is input to the neural network data set in the sensor processing unit 31, the control signal Cs corresponding to the detection information Si is generated. Hereinafter, the neural network data will be referred to as NN data Nd.
図6では、センサ処理部31Aは、学習器Lm、NNデータNdA1,NdA2,・・・,NdAn、及び、教師データTdAを保持している。また、センサ処理部31Bは、学習器Lm、NNデータNdB1,NdB2,・・・,NdBn、及び、教師データTdBを保持している。以降、センサ処理部31Nまで同様である。 In FIG. 6, the sensor processing unit 31A holds the learner Lm, the NN data NdA1, NdA2, ..., NdAn, and the teacher data TdA. Further, the sensor processing unit 31B holds the learner Lm, the NN data NdB1, NdB2, ..., NdBn, and the teacher data TdB. After that, the same applies to the sensor processing unit 31N.
センサ処理部31が保持する複数のNNデータNdは学習済みモデルである。この学習済みモデルは、ニューラルネットワークで構築される学習モデルに対して、入力する検知情報Siと出力する制御に関わる情報(又は制御信号Cs)とのデータセット、つまり教師データTdを用いて学習を行わせることにより獲得される。教師データTdは正解データとも呼ばれる。なお、センサ処理部31が保持する複数のNNデータNdの学習は、センサ処理部31が車両2に統合されるのに先立って行われる。 The plurality of NN data Nd held by the sensor processing unit 31 is a trained model. This trained model trains a training model constructed by a neural network using a data set of input detection information Si and output information related to control (or control signal Cs), that is, teacher data Td. Obtained by letting them do it. The teacher data Td is also called correct answer data. The learning of the plurality of NN data Nd held by the sensor processing unit 31 is performed prior to the integration of the sensor processing unit 31 into the vehicle 2.
なお、AIの学習モデルは、ニューラルネットワーク以外の機械学習の手法、例えば強化学習によって、構築されてもよい。 The learning model of AI may be constructed by a machine learning method other than the neural network, for example, reinforcement learning.
図7は、センサ処理部31の構成を説明するための模式図である。
センサ処理部31は、AI(つまり、学習器Lm)、複数のNNデータNdを保持するための図示しない記憶装置、複数のNNデータNdのうちの1つをAIに設定するための図示しない設定部、及び、車内外の他の装置との通信を行う図示しない通信部を備える。
センサ処理部31が備える複数のNNデータNdは、センサ処理部31が車両2に統合されるのに先立って、教師データ群を用いた学習器Lmでの学習が収束した学習済みモデルである。
FIG. 7 is a schematic diagram for explaining the configuration of the sensor processing unit 31.
The sensor processing unit 31 has an AI (that is, a learner Lm), a storage device (not shown) for holding a plurality of NN data Nd, and a setting (not shown) for setting one of the plurality of NN data Nd in the AI. A communication unit (not shown) for communicating with other devices inside and outside the vehicle is provided.
The plurality of NN data Nd included in the sensor processing unit 31 is a learned model in which learning with the learner Lm using the teacher data group has converged before the sensor processing unit 31 is integrated into the vehicle 2.
図7に示すように、センサ処理部31AのAIは、ニューラルネットワークで実現されており、パーセプトロンを有する。そして、複数のNNデータNdA1,NdA2,・・・,NdAnの各々は、パーセプトロンのネットワークを構成するためのパラメータデータであって、設定部は、複数のNNデータNdA1,NdA2,・・・,NdAnのうち1つをAIのパーセプトロンに設定する。つまり、パーセプトロンの構成を、複数のNNデータNdA1,NdA2,・・・,NdAnによって変えることができる。 As shown in FIG. 7, the AI of the sensor processing unit 31A is realized by a neural network and has a perceptron. Each of the plurality of NN data NdA1, NdA2, ..., NdAn is parameter data for configuring the network of the perceptron, and the setting unit is a plurality of NN data NdA1, NdA2, ..., NdAn. One of them is set to the AI perceptron. That is, the configuration of the perceptron can be changed by a plurality of NN data NdA1, NdA2, ..., NdAn.
これにより、センサ処理部31AのAIは、設定されたNNデータNdに依存して、ある入力情報(つまり、検知情報Si)に対して異なる出力情報(つまり、制御信号Cs)を生成することが可能となる。その結果、各々のセンサ処理部31は、車両2が置かれた環境に適応できるNNデータNdをAIに設定することにより、環境に応じて入力される検知情報Siに基づいて、適応性の高い(言い換えると、環境に対してインタラクティブな)制御信号Csを生成して、制御対象となる車載装置Vdを制御することができる。 As a result, the AI of the sensor processing unit 31A can generate different output information (that is, control signals Cs) for certain input information (that is, detection information Si) depending on the set NN data Nd. It will be possible. As a result, each sensor processing unit 31 has high adaptability based on the detection information Si input according to the environment by setting the NN data Nd that can be adapted to the environment in which the vehicle 2 is placed in AI. The control signal Cs (in other words, interactive with the environment) can be generated to control the vehicle-mounted device Vd to be controlled.
なお、図7に示す例は、パラメータデータを変えることによってパーセプトロンの構成、つまりAIの制御特性を変化させるものであるが、この他の実装方法として、例えば、センサ処理部31が構成の異なるパーセプトロンを複数保持して、パーセプトロン自体を切り替えることによって、AIの制御特性を変化させたりする構成としたり、学習器Lm内にパーセプトロンを持たせる構成としたりしてもよい。つまり、パーセプトロンの配置は任意に設計されるものとする。 In the example shown in FIG. 7, the configuration of the perceptron, that is, the control characteristic of AI is changed by changing the parameter data. As another mounting method, for example, the sensor processing unit 31 has a different configuration of the perceptron. By holding a plurality of perceptrons and switching the perceptron itself, the control characteristics of the AI may be changed, or the perceptron may be provided in the learner Lm. That is, the arrangement of the perceptron shall be arbitrarily designed.
図8は、AIに行わせる学習と、学習によって獲得される学習済みモデルとを説明するための図である。図8では、例えば、制動装置12を制御するための学習モデルを扱っている。 FIG. 8 is a diagram for explaining the learning performed by AI and the trained model acquired by the learning. FIG. 8 deals with, for example, a learning model for controlling the braking device 12.
図8(a)に示すように、学習モデルの学習過程において入力される情報B1は、車両2の走行速度、走行地点(又は走行位置)、目標地点(又は、目標距離若しくは目標位置)、及び、目標速度などが挙げられる。また、学習過程において教師データTdとなる入力情報B2は、車両2の走行速度、走行地点(又は走行位置)、目標地点(又は、目標距離若しくは目標位置)、目標速度、及び、制動力(又は、制動量若しくは制動時間)などが挙げられる。 As shown in FIG. 8A, the information B1 input in the learning process of the learning model includes the traveling speed of the vehicle 2, the traveling point (or the traveling position), the target point (or the target distance or the target position), and the target point. , Target speed, etc. Further, the input information B2 that becomes the teacher data Td in the learning process is the traveling speed of the vehicle 2, the traveling point (or the traveling position), the target point (or the target distance or the target position), the target speed, and the braking force (or). , Braking amount or braking time) and the like.
教師データTdを用いて学習を行った学習モデルは、入力情報B1に対して適切な制動力(又は、制動量若しくは制動時間)を推定し、出力情報Cとして出力できるようになる。 The learning model trained using the teacher data Td can estimate an appropriate braking force (or braking amount or braking time) for the input information B1 and output it as output information C.
さらに、図8(b)の「<1>データ取得」に示すように、学習のために取得するデータは、実際の走行車両2で得られる、制動制御Bcに関わる種々の情報を扱ってもよい。
制動制御Bcに関わる種々の情報とは、センサ32を用いて得られる又は導出される走行状態(例えば、各々のタイヤ4に掛かる荷重、車体3の傾き、及び車体3に働く慣性モーメントなど)及び路面状態、車載カメラ8又はレーダ(LiDAR)などを用いて認識される周囲の状況及び目標地点(又は、目標距離若しくは目標位置)、並びに、GPS、車車間通信又はVICS(Vehicle Information and Communication System(登録商標))などの車載ネットワーク40の通信を用いて得られる走行地点(又は走行位置)、天候の状況及び交通状況などが挙げられる。
Further, as shown in "<1> Data acquisition" of FIG. 8B, the data acquired for learning may handle various information related to the braking control Bc obtained in the actual traveling vehicle 2. good.
Various information related to the braking control Bc includes the running state (for example, the load applied to each tire 4, the inclination of the vehicle body 3, and the inertial moment acting on the vehicle body 3) obtained or derived by using the sensor 32. Road surface conditions, surrounding conditions and target points (or target distances or target positions) recognized using the vehicle-mounted camera 8 or radar (LiDAR), as well as GPS, vehicle-to-vehicle communication, or VICS (Vehicle Information and Communication System). Examples thereof include a traveling point (or traveling position) obtained by using communication of an in-vehicle network 40 such as (registered trademark)), weather conditions, and traffic conditions.
また、車両2及び環境のモデルを構築し、構築したモデルに対してシミュレーション解析を行うことで導出されるデータを扱ってもよい。その他に、予め得られる既知の情報として、車両2の固有情報(例えば、車体3重量、静止時の重心、二輪駆動又は四輪駆動、並びに、前輪駆動又は後輪駆動など)を扱ってもよい。 Further, data derived by constructing a model of the vehicle 2 and the environment and performing simulation analysis on the constructed model may be handled. In addition, as known information obtained in advance, unique information of the vehicle 2 (for example, vehicle body 3 weight, center of gravity at rest, two-wheel drive or four-wheel drive, front-wheel drive or rear-wheel drive, etc.) may be handled. ..
そして、図8(b)の「<2>データ前処理」に示すように、取得したデータに対して、次のように前処理を行うことが考えられる。実測によってデータを取得する場合、センサ32での検出情報の数値化及び変換処理、車載カメラ8での撮像画像の検出及び認識処理、レーダでの反射波情報の測定及び三次元マッピング処理、GPS情報に基づいた走行地点(又は走行位置)及び目標地点(又は、目標距離若しくは目標位置)の導出処理などが挙げられる。シミュレーション解析の場合、構築した車両2及び環境モデルに対する動的解析の外乱を含めた精度向上と演算量の削減、並びに誤差の修正処理などが挙げられる。 Then, as shown in "<2> Data preprocessing" in FIG. 8B, it is conceivable to perform preprocessing on the acquired data as follows. When acquiring data by actual measurement, digitization and conversion processing of detection information by sensor 32, detection and recognition processing of captured image by in-vehicle camera 8, measurement of reflected wave information by radar and three-dimensional mapping processing, GPS information The derivation process of the traveling point (or traveling position) and the target point (or the target distance or the target position) based on the above can be mentioned. In the case of simulation analysis, accuracy improvement including disturbance of dynamic analysis for the constructed vehicle 2 and the environmental model, reduction of calculation amount, error correction processing, and the like can be mentioned.
そして、図8(b)の「<3>AI学習」に示すように、制動装置12を制御の対象とするセンサ処理部31のAIは、上述の入力情報B1及びB2並びに出力情報Cを用いて学習を経ることで、例えば、走行条件及び交通規制などに応じた走行速度の制限、走行路上での衝突回避のための停止、カーブ又は見通しの悪い交差点での走行動作に伴う減速、目的地点での停車、といった動作に際して適宜、制動制御Bcの決定を行えるようになる。 Then, as shown in "<3> AI learning" of FIG. 8B, the AI of the sensor processing unit 31 whose control target is the braking device 12 uses the above-mentioned input information B1 and B2 and the output information C. Through learning, for example, limiting the driving speed according to driving conditions and traffic regulations, stopping to avoid collisions on the driving road, decelerating due to driving movement at curves or intersections with poor visibility, destination points. The braking control Bc can be appropriately determined at the time of an operation such as stopping at the vehicle.
さらに、上述の学習を各々のセンサ処理部31のAIに対して行うことで、種々の走行シーンDsに対応した複数の学習済みモデルとしてのNNデータNdを獲得することができる。種々の走行シーンDsは、完成車が走行する様々な環境(例えば、交通規制、走行場所、路面状態、気候、及び気温など)を特徴付ける要素の情報で区別される。学習過程で用いられる教師データTdは、この環境を特徴付ける要素の情報を含むように予め用意される。走行シーンDsについては後述する。 Further, by performing the above-mentioned learning on the AI of each sensor processing unit 31, it is possible to acquire NN data Nd as a plurality of learned models corresponding to various driving scenes Ds. The various driving scenes Ds are distinguished by information on factors that characterize the various environments in which the finished vehicle travels, such as traffic regulations, driving locations, road conditions, climate, and temperature. The teacher data Td used in the learning process is prepared in advance to include information on the elements that characterize this environment. The driving scene Ds will be described later.
このようにして、車両2に搭載される車載装置Vdは、センサ処理部31がこれらのNNデータNdを切り替えて適用することにより、自装置の動作を環境に合わせることが可能となる。
そして、図8(b)の「<4>AI活用」に示すように、センサ処理部31のAIは、例えば、様々な環境に適用可能な自動運転機能又は運転支援機能を実装した自動車における制動装置12の制動制御に活用される。このことは、完成車メーカーは、完成車に搭載された車載装置Vdに対して所望する信頼性を確保することができるようになる。
In this way, the in-vehicle device Vd mounted on the vehicle 2 can adjust the operation of its own device to the environment by the sensor processing unit 31 switching and applying these NN data Nd.
Then, as shown in "<4> AI utilization" in FIG. 8B, the AI of the sensor processing unit 31 is, for example, braking in an automobile equipped with an automatic driving function or a driving support function applicable to various environments. It is used for braking control of the device 12. This makes it possible for the finished vehicle manufacturer to secure the desired reliability for the in-vehicle device Vd mounted on the finished vehicle.
完成車としての車両2が走行する環境には、例えば、比較的低速に走行する市街地又は山岳地、比較的高速に走行する高速道路又は交通規制の緩い郊外の交通網、タイヤ4のグリップが利きやすい舗装道路、並びに、タイヤ4のグリップが利きづらい未舗装路などが挙げられる。 For the environment in which the vehicle 2 as a completed vehicle travels, for example, an urban area or a mountainous area traveling at a relatively low speed, a highway traveling at a relatively high speed or a traffic network in a suburban area with loose traffic restrictions, and a grip of a tire 4 are effective. Examples include easy paved roads and unpaved roads where the grip of the tire 4 is difficult to use.
そして、これらの環境における検知情報Siとして、例えば、市街地では、走行車線、標識、信号、他の車両2、歩行者、自転車及び周囲若しくは周辺の見通しを妨げる建物等の障害物などが多く存在する。また、山岳地では、走行車線において頻度が高く度合いの厳しい起伏、勾配及び湾曲などが多く存在する。また、高速道路では、周囲の走行車両2との位置関係、トンネルへの出入りによる明暗、道路への乗り降りを含む車線の変更、及び、交通規制又は標識などが多く存在する。 As the detection information Si in these environments, for example, in urban areas, there are many obstacles such as traveling lanes, signs, traffic lights, other vehicles 2, pedestrians, bicycles, and buildings that obstruct the visibility of the surroundings or the surroundings. .. Further, in mountainous areas, there are many frequent and severe undulations, slopes and curves in the traveling lane. Further, on the expressway, there are many positional relationships with the surrounding traveling vehicle 2, light and darkness due to entering and exiting the tunnel, lane changes including getting on and off the road, and traffic restrictions or signs.
さらに、車両2が走行する環境には、例えば、車載カメラ8が撮像する映像から情報を認識しづらい豪雨の状況、ステアリング制御とおりに走行しづらい強風の状況、並びに、水たまり、凍結及び深雪といった路面状態を悪化させる天候及び気象の状況などが挙げられる。 Further, in the environment in which the vehicle 2 travels, for example, a heavy rain situation in which it is difficult to recognize information from an image captured by an in-vehicle camera 8, a strong wind situation in which it is difficult to drive according to steering control, and a road surface such as a puddle, freezing, and deep snow. Examples include the weather and weather conditions that worsen the condition.
このように、車両2が走行する環境における検知情報Siは、種々の交通規制、周囲若しくは周辺の状況及び路面状態、種々の天候及び気候の状況、並びに、これらの状態及び状況が複合的に影響を及ぼす場合に依存する。複合的に影響を及ぼす走行シーンDsとは、例えば、晴天の都市部の一般道路、雨天時の郊外の舗装されていない走行路、及び、豪雪時の高速道路などであって、環境を特徴付ける複数の要素を組み替えたり組み合わせたりすることで構成される。センサ処理部31は、これら複数の要素を検知情報Siとして入力する。 As described above, the detection information Si in the environment in which the vehicle 2 travels is affected by various traffic regulations, surrounding or surrounding conditions and road surface conditions, various weather and climatic conditions, and these conditions and conditions in a complex manner. Depends on the case. The driving scenes Ds that have multiple influences are, for example, general roads in urban areas in fine weather, unpaved driving roads in the suburbs in rainy weather, and highways in heavy snowfall, which characterize the environment. It is composed by rearranging and combining the elements of. The sensor processing unit 31 inputs these plurality of elements as detection information Si.
なお、環境を特徴付ける複数の要素は、シミュレーションによって導出した情報であってもよいし、実際の車両2を実環境で走行させたときに取得した情報であってもよい。 The plurality of elements that characterize the environment may be information derived by simulation, or may be information acquired when the actual vehicle 2 is driven in the actual environment.
車両2が置かれた環境に応じて、種々の車載装置Vdの制御を行うセンサ処理部31のAIは、車両2に統合される前段階において学習を行う。多様な特徴を含む環境を表現した走行シーンDsごとに複数の教師データTd(以下、教師データ群と呼ぶ)が予め用意される。これらの走行シーンDsごとの教師データ群を用いて学習モデルに学習をさせることによって学習済みモデルを獲得する。1つの走行シーンDsと対応した学習済みモデルが1つのNNデータNdであって、センサ処理部31は複数のNNデータNdを保持している。 The AI of the sensor processing unit 31 that controls various in-vehicle devices Vd according to the environment in which the vehicle 2 is placed performs learning before being integrated into the vehicle 2. A plurality of teacher data Td (hereinafter referred to as teacher data group) are prepared in advance for each driving scene Ds expressing an environment including various features. A trained model is acquired by training a training model using the teacher data group for each of these running scenes Ds. The trained model corresponding to one running scene Ds is one NN data Nd, and the sensor processing unit 31 holds a plurality of NN data Nd.
そして、センサ処理部31は、車両2に統合された後、入力する検知情報Siをもとに、統括部30から切り替え指示SwがあったNNデータNdを用いて制御を行う。統括部30の切り替え指示Swについては後述する。このときのセンサ処理部31でのNNデータNdの切り替えは、統括部30からの切り替え指示Swがあった直後でなくとも、センサ処理部31の制御又は車載装置Vdの動作が安定しているタイミングで行うようにしてもよい。 Then, after being integrated into the vehicle 2, the sensor processing unit 31 performs control using the NN data Nd for which the switching instruction Sw has been given from the control unit 30 based on the detection information Si to be input. The switching instruction Sw of the control unit 30 will be described later. The switching of the NN data Nd in the sensor processing unit 31 at this time is the timing at which the control of the sensor processing unit 31 or the operation of the in-vehicle device Vd is stable even if it is not immediately after the switching instruction Sw from the control unit 30 is received. You may do it at.
このように、統合された車両2において各々のセンサ処理部31が複数あるNNデータNdのうち環境に適したNNデータNdに切り替えることで、例えば、市街地において速度の変化度合いを緩くするとともに車載カメラ8及び車載レーダ9の検知対象を比較的小さなもの(例えば、歩行者、自転車及び信号など)に向けて特に感度を高める、高速道路において速度の変化度合いを強くするとともにステアリングの変化度合いを大きく緩やかにする、山岳地においてステアリングの変化度合いを細かく機微にするとともに車載カメラ8及び車載レーダの検知対象を比較的大きなもの(例えば、湾曲した断崖及び傾斜など)に向けて特に感度を高める、などといった制御を車載装置Vdに対して行わせることが可能となる。 In this way, by switching to the NN data Nd suitable for the environment among the NN data Nd in which each sensor processing unit 31 has a plurality of NN data Nd in the integrated vehicle 2, for example, the degree of change in speed is relaxed in the urban area and the in-vehicle camera is used. The detection target of 8 and the in-vehicle radar 9 is aimed at relatively small objects (for example, pedestrians, bicycles, signals, etc.), and the sensitivity is particularly increased. In addition to making the degree of steering change finer and more sensitive in mountainous areas, the sensitivity of the in-vehicle camera 8 and the in-vehicle radar is particularly increased toward relatively large objects (for example, curved cliffs and slopes). It is possible to control the vehicle-mounted device Vd.
つまり、センサ処理部31は、例えば、右折を行う制御であっても、走行する環境に応じて、制御の量、期間(又は時間)、変化率(又は変化速度)及び他の制御との優先度などが異なる制御信号Csを生成して出力することが可能となる。 That is, the sensor processing unit 31 has priority over the amount of control, the period (or time), the rate of change (or the speed of change), and other controls, for example, even in the control of making a right turn, depending on the traveling environment. It is possible to generate and output control signals Cs having different degrees.
統括部30は、種々の走行シーンDsを推定するための情報、各々のセンサ処理部31が保持する複数のNNデータNdと走行シーンDsとの関連付けを示す情報、並びに、センサ処理部31と検知情報Si及び制御信号Csとを関連付ける関連情報Riを予め保持している。この関連情報Riは、図8(a)に示した、システム1への統合前のAI対して行われる学習に関する情報、つまり、入力情報B1及びB2と出力情報Cとのデータセットをもとに生成される。また、統括部30は、各々のセンサ処理部31に設定中のNNデータNdの情報を保持している。 The control unit 30 detects information for estimating various driving scenes Ds, information indicating the association between the plurality of NN data Nd held by each sensor processing unit 31 and the driving scene Ds, and the sensor processing unit 31. The related information Ri associated with the information Si and the control signal Cs is stored in advance. This related information Ri is based on the information related to the learning performed for the AI before integration into the system 1, that is, the data set of the input information B1 and B2 and the output information C shown in FIG. 8A. Generated. Further, the control unit 30 holds the information of the NN data Nd set in each sensor processing unit 31.
図6に説明を戻す。統括部30は、評価部30e及び選択部30sを備える。以下、統括部30の評価部30eについて説明する。 The explanation is returned to FIG. The control unit 30 includes an evaluation unit 30e and a selection unit 30s. Hereinafter, the evaluation unit 30e of the control unit 30 will be described.
評価部30eは、入力した検知情報Siをもとに、車両2が置かれた環境を特徴付ける要素を抽出又は導出し、センサ処理部31に設定中のNNデータNdでの制御が適切であるかどうかを評価する。 Based on the input detection information Si, the evaluation unit 30e extracts or derives an element that characterizes the environment in which the vehicle 2 is placed, and whether control by the NN data Nd set in the sensor processing unit 31 is appropriate. Please evaluate.
評価部30eは、上述の関連情報Riを用いて、車両2に統合後の各々のセンサ処理部31が出力する検知情報Siと制御信号Csとのデータセットの評価を行う。つまり、評価部30eは、車両2が環境とインタラクションするときの、車両2の状態St(つまり、入力する検知情報Si)と行動Act(つまり、出力する制御信号Cs)のデータセットを用いて、各々のAIに設定されたNNデータNdの制御を評価する。 The evaluation unit 30e evaluates the data set of the detection information Si and the control signal Cs output by each sensor processing unit 31 after being integrated into the vehicle 2 by using the above-mentioned related information Ri. That is, the evaluation unit 30e uses the data set of the state St (that is, the input detection information Si) and the action Act (that is, the output control signal Cs) of the vehicle 2 when the vehicle 2 interacts with the environment. The control of the NN data Nd set in each AI is evaluated.
図9は、統括部30の評価部30eが、検知情報Si及び制御信号CsのデータセットをもとにNNデータNdを評価する過程を説明するための模式図である。 FIG. 9 is a schematic diagram for explaining a process in which the evaluation unit 30e of the control unit 30 evaluates the NN data Nd based on the data set of the detection information Si and the control signal Cs.
センサ処理部31Aに設定されるNNデータNdA1は、市街地での走行において駆動装置11(例えば、エンジン又はモーター)を制御するために獲得された学習済みモデルである。また、センサ処理部31Aに設定されるNNデータNdA2は、高速道路での走行において駆動装置11(例えば、エンジン又はモーター)を制御するために獲得された学習済みモデルである。 The NN data NdA1 set in the sensor processing unit 31A is a learned model acquired for controlling the drive device 11 (for example, an engine or a motor) in traveling in an urban area. Further, the NN data NdA2 set in the sensor processing unit 31A is a learned model acquired for controlling the drive device 11 (for example, an engine or a motor) in traveling on a highway.
センサ処理部31Bに設定されるNNデータNdB1は、市街地での走行において制動装置12(例えばブレーキ装置)を制御するために獲得された学習済みモデルである。また、センサ処理部31Bに設定されるNNデータNdB2は、高速道路での走行において制動装置12(例えばブレーキ装置)を制御するために獲得された学習済みモデルである。 The NN data NdB1 set in the sensor processing unit 31B is a learned model acquired for controlling the braking device 12 (for example, the braking device) in traveling in an urban area. Further, the NN data NdB2 set in the sensor processing unit 31B is a learned model acquired for controlling the braking device 12 (for example, the braking device) in traveling on a highway.
センサ処理部31Cに設定されるNNデータNdC1は、市街地での走行において操舵装置13(例えばステアリング装置)を制御するために獲得された学習済みモデルである。また、センサ処理部31Cに設定されるNNデータNdC2は、高速道路での走行において操舵装置13(例えばステアリング装置)を制御するために獲得された学習済みモデルである。 The NN data NdC1 set in the sensor processing unit 31C is a learned model acquired for controlling the steering device 13 (for example, the steering device) in traveling in an urban area. Further, the NN data NdC2 set in the sensor processing unit 31C is a learned model acquired for controlling the steering device 13 (for example, the steering device) in traveling on a highway.
センサ処理部31Dに設定されるNNデータNdD1は、市街地での走行において緩衝装置14(例えばサスペンション装置)を制御するために獲得された学習済みモデルである。また、センサ処理部31Dに設定されるNNデータNdD2は、高速道路での走行において緩衝装置14(例えばサスペンション装置)を制御するために獲得された学習済みモデルである。 The NN data NdD1 set in the sensor processing unit 31D is a learned model acquired for controlling the shock absorber 14 (for example, the suspension device) in traveling in an urban area. Further, the NN data NdD2 set in the sensor processing unit 31D is a learned model acquired for controlling the shock absorber 14 (for example, a suspension device) in traveling on a highway.
評価部30eは、車両2への統合後のセンサ処理部31A~Dから入力する検知情報Si及び制御信号Csのデータセットをもとに、当該センサ処理部31に設定中のNNデータNdが環境に適しているかどうか評価を行う。 In the evaluation unit 30e, the NN data Nd set in the sensor processing unit 31 is the environment based on the data set of the detection information Si and the control signal Cs input from the sensor processing units 31A to D after integration into the vehicle 2. Evaluate whether it is suitable for.
図9(a)は、センサ処理部31Aが、NNデータNdA1及びNNデータNdA2を用いて駆動装置11(例えば、エンジン又はモーター)を制御したときの、車両2の走行速度vと、車両2の振動及び傾きから導出される走行の不安定度合いIsとの関係を示している。この他にも、走行の不安定度合いIsには、評価対象の車載装置Vdに対する制御に関わる情報とセンサ32に関わる情報とのデータセットを用いても良く、例えば、各々のタイヤ4の摩擦力及び荷重(又は衝撃力)、並びに、車両2の運動エネルギー及び慣性モーメントなどを用いてもよい。駆動装置11に対する制御を駆動制御Dcとする。 FIG. 9A shows the traveling speed v of the vehicle 2 and the traveling speed v of the vehicle 2 when the sensor processing unit 31A controls the drive device 11 (for example, an engine or a motor) using the NN data NdA1 and the NN data NdA2. It shows the relationship with the degree of instability Is of running derived from vibration and inclination. In addition to this, as the running instability degree Is, a data set of information related to control for the in-vehicle device Vd to be evaluated and information related to the sensor 32 may be used, for example, the frictional force of each tire 4. And the load (or impact force), and the kinetic energy and moment of inertia of the vehicle 2 may be used. The control for the drive device 11 is referred to as a drive control Dc.
図9(a)に示すように、NNデータNdA1は、学習過程によれば、走行速度vが時速40km~時速60kmの状態のとき、NNデータNdA2よりも走行の不安定度合いIsを低く抑制することができる。また、NNデータNdA2は、学習過程によれば、走行速度vが時速80km~時速100kmの状態のとき、NNデータNdA1よりも走行の不安定度合いIsを低く抑制することができる。 As shown in FIG. 9A, according to the learning process, the NN data NdA1 suppresses the degree of instability Is of traveling lower than that of the NN data NdA2 when the traveling speed v is 40 km / h to 60 km / h. be able to. Further, according to the learning process, the NN data NdA2 can suppress the degree of instability Is of traveling lower than that of the NN data NdA1 when the traveling speed v is in a state of 80 km / h to 100 km / h.
評価部30eは、統括部30が保持する情報又は統括部30がセンサ処理部31Aから入力する情報に基づいて、センサ処理部31AにはNNデータNdA1が設定中であり、検知情報Si及び制御信号Csのデータセットをもとに走行速度vが時速40km~時速60kmの状態であると判断した場合、センサ処理部31AにNNデータNdA1が設定されていることは適切であると評価する。また、検知情報Si及び制御信号Csのデータセットをもとに走行の不安定度合いIsが低く抑制されていると判断した場合、NNデータNdA1の制御は適切であると評価する。 The evaluation unit 30e is setting NN data NdA1 in the sensor processing unit 31A based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31A, and the detection information Si and the control signal. When it is determined that the traveling speed v is in the state of 40 km / h to 60 km / h based on the Cs data set, it is evaluated that it is appropriate that the NN data NdA1 is set in the sensor processing unit 31A. Further, when it is determined that the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs, it is evaluated that the control of the NN data NdA1 is appropriate.
一方で、評価部30eは、統括部30が保持する情報又は統括部30がセンサ処理部31Aから入力する情報に基づいて、センサ処理部31AにはNNデータNdA2が設定中であり、検知情報Si及び制御信号Csのデータセットをもとに走行速度vが時速80km~時速100kmの状態であると判断した場合、センサ処理部31AにNNデータNdA2が設定されていることは適切であると評価する。また、検知情報Si及び制御信号Csのデータセットをもとに走行の不安定度合いIsが低く抑制されていると判断した場合、NNデータNdA2の制御は適切であると評価する。 On the other hand, in the evaluation unit 30e, NN data NdA2 is being set in the sensor processing unit 31A based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31A, and the detection information Si When it is determined that the traveling speed v is in the state of 80 km / h to 100 km / h based on the data set of the control signal Cs, it is evaluated that it is appropriate that the NN data NdA2 is set in the sensor processing unit 31A. .. Further, when it is determined that the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs, it is evaluated that the control of the NN data NdA2 is appropriate.
図9(b)は、センサ処理部31Bが、NNデータNdB1及びNNデータNdB2を用いて制動装置12(例えばブレーキ装置)を制御したときの、車両2の走行速度vと、走行の不安定度合いIsとの関係を示している。走行の不安定度合いIsについては、図9(a)と同様である。制動装置12に対する制御を制動制御Bcとする。 FIG. 9B shows the traveling speed v of the vehicle 2 and the degree of instability of traveling when the sensor processing unit 31B controls the braking device 12 (for example, the braking device) using the NN data NdB1 and the NN data NdB2. It shows the relationship with Is. The degree of instability in running Is is the same as in FIG. 9A. The control for the braking device 12 is referred to as braking control Bc.
図9(b)に示すように、NNデータNdB1は、学習過程によれば、走行速度vが時速40km~時速60kmの状態のとき、NNデータNdB2よりも走行の不安定度合いIsを低く抑制することができる。また、NNデータNdB2は、学習過程によれば、走行速度vが時速80km~時速100kmの状態のとき、NNデータNdB1よりも走行の不安定度合いIsを低く抑制することができる。 As shown in FIG. 9B, according to the learning process, the NN data NdB1 suppresses the degree of instability Is of traveling lower than the NN data NdB2 when the traveling speed v is in a state of 40 km / h to 60 km / h. be able to. Further, according to the learning process, the NN data NdB2 can suppress the degree of instability Is of traveling lower than that of the NN data NdB1 when the traveling speed v is in a state of 80 km / h to 100 km / h.
評価部30eは、統括部30が保持する情報又は統括部30がセンサ処理部31Bから入力する情報に基づいて、センサ処理部31BにはNNデータNdB1が設定中であり、検知情報Si及び制御信号Csのデータセットをもとに走行速度vが時速40km~時速60kmの状態であると判断した場合、センサ処理部31BにNNデータNdB1が設定されていることは適切であると評価する。また、検知情報Si及び制御信号Csのデータセットをもとに走行の不安定度合いIsが低く抑制されていると判断した場合、NNデータNdB1の制御は適切であると評価する。 The evaluation unit 30e is setting NN data NdB1 in the sensor processing unit 31B based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31B, and the detection information Si and the control signal. When it is determined that the traveling speed v is in the state of 40 km / h to 60 km / h based on the Cs data set, it is evaluated that it is appropriate that the NN data NdB1 is set in the sensor processing unit 31B. Further, when it is determined that the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs, it is evaluated that the control of the NN data NdB1 is appropriate.
一方で、評価部30eは、統括部30が保持する情報又は統括部30がセンサ処理部31Bから入力する情報に基づいて、センサ処理部31BにはNNデータNdB2が設定中であり、検知情報Si及び制御信号Csのデータセットをもとに走行速度vが時速80km~時速100kmの状態であると判断した場合、センサ処理部31BにNNデータNdB2が設定されていることは適切であると評価する。また、検知情報Si及び制御信号Csのデータセットをもとに走行の不安定度合いIsが低く抑制されていると判断した場合、NNデータNdB2の制御は適切であると評価する。 On the other hand, in the evaluation unit 30e, NN data NdB2 is being set in the sensor processing unit 31B based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31B, and the detection information Si When it is determined that the traveling speed v is in the state of 80 km / h to 100 km / h based on the data set of the control signal Cs, it is evaluated that it is appropriate that the NN data NdB2 is set in the sensor processing unit 31B. .. Further, when it is determined that the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs, it is evaluated that the control of the NN data NdB2 is appropriate.
図9(c)は、センサ処理部31Cが、NNデータNdC1及びNNデータNdC2を用いて操舵装置13(例えばステアリング装置)を制御したときの、車両2の走行速度vと、走行の不安定度合いIsとの関係を示している。走行の不安定度合いIsについては、図9(a)と同様である。操舵装置13に対する制御を操舵制御Scとする。 FIG. 9C shows the traveling speed v of the vehicle 2 and the degree of instability of traveling when the sensor processing unit 31C controls the steering device 13 (for example, the steering device) using the NN data NdC1 and the NN data NdC2. It shows the relationship with Is. The degree of instability in running Is is the same as in FIG. 9A. The control for the steering device 13 is defined as steering control Sc.
図9(c)に示すように、NNデータNdC1は、学習過程によれば、走行速度vが時速40km~時速60kmの状態のとき、NNデータNdC2よりも走行の不安定度合いIsを低く抑制することができる。また、NNデータNdC2は、学習過程によれば、走行速度vが時速80km~時速100kmの状態のとき、NNデータNdC1よりも走行の不安定度合いIsを低く抑制することができる。 As shown in FIG. 9C, according to the learning process, the NN data NdC1 suppresses the degree of instability Is of traveling lower than the NN data NdC2 when the traveling speed v is in a state of 40 km / h to 60 km / h. be able to. Further, according to the learning process, the NN data NdC2 can suppress the degree of instability Is of traveling lower than that of the NN data NdC1 when the traveling speed v is in a state of 80 km / h to 100 km / h.
評価部30eは、統括部30が保持する情報又は統括部30がセンサ処理部31Cから入力する情報に基づいて、センサ処理部31CにはNNデータNdC1が設定中であり、検知情報Si及び制御信号Csのデータセットをもとに走行速度vが時速40km~時速60kmの状態であると判断した場合、センサ処理部31CにNNデータNdC1が設定されていることは適切であると評価する。また、検知情報Si及び制御信号Csのデータセットをもとに走行の不安定度合いIsが低く抑制されていると判断した場合、NNデータNdC1の制御は適切であると評価する。 The evaluation unit 30e is setting NN data NdC1 in the sensor processing unit 31C based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31C, and the detection information Si and the control signal. When it is determined that the traveling speed v is in the state of 40 km / h to 60 km / h based on the Cs data set, it is evaluated that it is appropriate that the NN data NdC1 is set in the sensor processing unit 31C. Further, when it is determined that the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs, it is evaluated that the control of the NN data NdC1 is appropriate.
一方で、評価部30eは、統括部30が保持する情報又は統括部30がセンサ処理部31Cから入力する情報に基づいて、センサ処理部31CにはNNデータNdC2が設定中であり、検知情報Si及び制御信号Csのデータセットをもとに走行速度vが時速80km~時速100kmの状態であると判断した場合、センサ処理部31CにNNデータNdC2が設定されていることは適切であると評価する。また、検知情報Si及び制御信号Csのデータセットをもとに走行の不安定度合いIsが低く抑制されていると判断した場合、NNデータNdC2の制御は適切であると評価する。 On the other hand, in the evaluation unit 30e, NN data NdC2 is being set in the sensor processing unit 31C based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31C, and the detection information Si When it is determined that the traveling speed v is in the state of 80 km / h to 100 km / h based on the data set of the control signal Cs, it is evaluated that it is appropriate that the NN data NdC2 is set in the sensor processing unit 31C. .. Further, when it is determined that the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs, it is evaluated that the control of the NN data NdC2 is appropriate.
図9(d)は、センサ処理部31Dが、NNデータNdD1及びNNデータNdD2を用いて緩衝装置14(例えばサスペンション装置)を制御したときの、車両2の走行速度vと、走行の不安定度合いIsとの関係を示している。走行の不安定度合いIsについては、図9(a)と同様である。緩衝装置14に対する制御を緩衝制御Ccとする。 FIG. 9D shows the traveling speed v of the vehicle 2 and the degree of instability of traveling when the sensor processing unit 31D controls the shock absorber 14 (for example, the suspension device) using the NN data NdD1 and the NN data NdD2. It shows the relationship with Is. The degree of instability in running Is is the same as in FIG. 9A. The control for the shock absorber 14 is referred to as buffer control Cc.
図9(d)に示すように、NNデータNdD1は、学習過程によれば、走行速度vが時速40km~時速60kmの状態のとき、NNデータNdD2よりも走行の不安定度合いIsを低く抑制することができる。また、NNデータNdD2は、学習過程によれば、走行速度vが時速80km~時速100kmの状態のとき、NNデータNdD1よりも走行の不安定度合いIsを低く抑制することができる。 As shown in FIG. 9D, according to the learning process, the NN data NdD1 suppresses the degree of instability Is of traveling lower than that of the NN data NdD2 when the traveling speed v is 40 km / h to 60 km / h. be able to. Further, according to the learning process, the NN data NdD2 can suppress the degree of instability Is of traveling lower than that of the NN data NdD1 when the traveling speed v is in a state of 80 km / h to 100 km / h.
評価部30eは、統括部30が保持する情報又は統括部30がセンサ処理部31Dから入力する情報に基づいて、センサ処理部31DにはNNデータNdD1が設定中であり、検知情報Si及び制御信号Csのデータセットをもとに走行速度vが時速40km~時速60kmの状態であると判断した場合、センサ処理部31DにNNデータNdD1が設定されていることは適切であると評価する。また、検知情報Si及び制御信号Csのデータセットをもとに走行の不安定度合いIsが低く抑制されていると判断した場合、NNデータNdD1の制御は適切であると評価する。 The evaluation unit 30e is setting NN data NdD1 in the sensor processing unit 31D based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31D, and the detection information Si and the control signal. When it is determined that the traveling speed v is in the state of 40 km / h to 60 km / h based on the Cs data set, it is evaluated that it is appropriate that the NN data NdD1 is set in the sensor processing unit 31D. Further, when it is determined that the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs, it is evaluated that the control of the NN data NdD1 is appropriate.
一方で、評価部30eは、統括部30が保持する情報又は統括部30がセンサ処理部31Dから入力する情報に基づいて、センサ処理部31DにはNNデータNdD2が設定中であり、検知情報Si及び制御信号Csのデータセットをもとに走行速度vが時速80km~時速100kmの状態であると判断した場合、センサ処理部31DにNNデータNdD2が設定されていることは適切であると評価する。また、検知情報Si及び制御信号Csのデータセットをもとに走行の不安定度合いIsが低く抑制されていると判断した場合、NNデータNdD2の制御は適切であると評価する。 On the other hand, in the evaluation unit 30e, NN data NdD2 is being set in the sensor processing unit 31D based on the information held by the control unit 30 or the information input by the control unit 30 from the sensor processing unit 31D, and the detection information Si When it is determined that the traveling speed v is in the state of 80 km / h to 100 km / h based on the data set of the control signal Cs, it is evaluated that it is appropriate that the NN data NdD2 is set in the sensor processing unit 31D. .. Further, when it is determined that the degree of instability Is of traveling is suppressed low based on the data set of the detection information Si and the control signal Cs, it is evaluated that the control of the NN data NdD2 is appropriate.
このようにして、評価部30eは、各々のセンサ処理部31が複数あるNNデータNdの1つを用いて制御対象の車載装置Vdを制御するときの検知情報Si及び制御信号Csのデータセットに基づいて、センサ処理部31に設定中のNNデータNdが適しているかどうかを評価するとともに、当該NNデータNdの制御による車載装置Vdの動作及び状態の安定性を評価する。 In this way, the evaluation unit 30e can be used as a data set of detection information Si and control signal Cs when the in-vehicle device Vd to be controlled is controlled by using one of the NN data Nd in which each sensor processing unit 31 has a plurality of NN data Nd. Based on this, it is evaluated whether or not the NN data Nd set in the sensor processing unit 31 is suitable, and the stability of the operation and state of the in-vehicle device Vd under the control of the NN data Nd is evaluated.
評価部30eは、さらに、複数のセンサ処理部31に設定中のNNデータNdを組み合わせたときの制御が適切に機能しているかどうかを判定及び評価してもよい。 The evaluation unit 30e may further determine and evaluate whether or not the control when the NN data Nd being set is combined with the plurality of sensor processing units 31 is functioning appropriately.
以下、統括部30の選択部30sについて説明をする。評価部30eでの評価の内容が、設定中のNNデータNdの制御が適切ではなく、そのため車両2の走行が不安定であることを示す場合、選択部30sは、入力した検知情報Si及び制御信号Csのデータセットをもとに、上述した複数の走行シーンDsのうち、いずれに近似するか、又は、いずれの走行シーンDsが相応しいか推定を行う。 Hereinafter, the selection unit 30s of the control unit 30 will be described. When the content of the evaluation by the evaluation unit 30e indicates that the control of the NN data Nd being set is not appropriate and therefore the running of the vehicle 2 is unstable, the selection unit 30s determines the input detection information Si and the control. Based on the data set of the signal Cs, it is estimated which of the above-mentioned plurality of driving scenes Ds is close to or which driving scene Ds is suitable.
選択部30sが走行シーンDsを推定するための情報としては、例えば、車両2の走行速度、振動、傾き、運動エネルギー及び慣性モーメント、タイヤ4に掛かる摩擦力及び荷重(又は衝撃力)、並びに、走行路の勾配又は湾曲度合い(つまりRの度合い)、気温又は気候などが挙げられる。選択部30sは、この他に、車載ネットワーク40から得られる情報を用いて走行シーンDsを推定してもよい。 Information for the selection unit 30s to estimate the traveling scene Ds includes, for example, the traveling speed, vibration, inclination, kinetic energy and moment of inertia of the vehicle 2, the frictional force and load (or impact force) applied to the tire 4, and The degree of slope or curvature of the road (that is, the degree of R), temperature or climate, etc. may be mentioned. In addition to this, the selection unit 30s may estimate the driving scene Ds using the information obtained from the vehicle-mounted network 40.
なお、統括部30は、環境を特徴付ける要素の情報と、当該要素の情報を対応付けた走行シーンDsの情報とを予め保持しているものとする。 It is assumed that the control unit 30 holds in advance the information of the elements that characterize the environment and the information of the driving scene Ds associated with the information of the elements.
ここで、実際の環境から得られる検知情報Siが、予め保持する走行シーンDsを構成する要素の情報と一致することは稀であると考えられる。そこで、環境を特徴付ける要素のうち走行の安全性又は信頼性に関わるものに予め高い優先度を設けておき、選択部30sが、総合的に優先度の高い走行シーンDsを推定するようにしたり、近似する走行シーンDsの候補を複数推定して、そのうち最も走行の安全性が高い制御と対応する走行シーンDsを推定するようにしたりしてもよい。最も走行の安全性が高い制御とは、例えば、走行速度の制御範囲が低速なもの、又は、周囲の対象物の検知若しくは周辺の状況の認識に基づいて走行制御を行うものなどである。 Here, it is considered that the detection information Si obtained from the actual environment rarely matches the information of the elements constituting the traveling scene Ds held in advance. Therefore, among the elements that characterize the environment, those related to driving safety or reliability are given high priority in advance, and the selection unit 30s may estimate the driving scene Ds having high priority comprehensively. A plurality of candidates for similar driving scenes Ds may be estimated, and the driving scene Ds corresponding to the control having the highest driving safety may be estimated. The control having the highest traveling safety is, for example, one in which the control range of the traveling speed is low, or one in which the traveling control is performed based on the detection of a surrounding object or the recognition of the surrounding situation.
そして、選択部30sは、各々のセンサ処理部31が保持するNNデータNdのうち、推定した走行シーンDsに適するものを選択する。また、選択部30sは、入力した検知情報Si及び制御信号Csのデータセットをもとに、実際の環境に近似又は相応する特徴を含む教師データ群を用いて学習を行ったものを選択したり、車載ネットワーク40から得られる情報を用いてNNデータNdを選択したりしてもよい。 Then, the selection unit 30s selects the NN data Nd held by each sensor processing unit 31 that is suitable for the estimated driving scene Ds. Further, the selection unit 30s selects a data set obtained by learning using a teacher data group including features that are close to or corresponding to the actual environment based on the input detection information Si and the control signal Cs data set. , NN data Nd may be selected using the information obtained from the vehicle-mounted network 40.
その後、選択部30sは、各々のセンサ処理部31に対し、選択したNNデータNdを設定するように切り替え指示Swを行う。 After that, the selection unit 30s issues a switching instruction Sw to each sensor processing unit 31 so as to set the selected NN data Nd.
さらに、選択部30sは、評価部30eでの評価内容によらず、あるセンサ処理部31から入力した検知情報Si及び制御信号Csのデータセットをもとに、複数の走行シーンDsのうちから新たに近似する又は相応しいものを推定したり、あるセンサ処理部31に設定中のNNデータNdよりも走行シーンDsに適する他のNNデータNdを選択し、当該センサ処理部31に対して当該他のNNデータNdを設定させるための切り替え指示Swを行ったりしてもよい。 Further, the selection unit 30s is new from a plurality of driving scenes Ds based on the data set of the detection information Si and the control signal Cs input from a certain sensor processing unit 31 regardless of the evaluation content of the evaluation unit 30e. Other NN data Nd suitable for the driving scene Ds is selected from the NN data Nd set in a certain sensor processing unit 31, and the other NN data Nd is selected for the sensor processing unit 31. The switching instruction Sw for setting the NN data Nd may be performed.
図10は、AI統合システム1における統括部30での処理を説明するためのフローチャート図である。 FIG. 10 is a flowchart for explaining the processing in the control unit 30 in the AI integrated system 1.
図10(a)は、評価部30eでの処理を示す。
処理Sp81aでは、評価部30eが、各々のセンサ処理部31から検知情報Si及び制御信号Csのデータセットを入力する。
処理Sp82aでは、評価部30eが、処理Sp81aで入力したデータセットと対応する各々のセンサ処理部31のNNデータNdでの制御を評価する。
処理Sp83aでは、評価部30eが、処理Sp82aでの評価をもとに各々のNNデータNdでの制御が適切であるかどうかを判定する。制御が適切であるNNデータNdにおいては処理Sp81aに進む。制御が適切ではないNNデータNdにおいては処理Sp84aに進む。
処理Sp84aでは、図10(b)の処理を行う。
FIG. 10A shows the processing in the evaluation unit 30e.
In the processing Sp81a, the evaluation unit 30e inputs the data set of the detection information Si and the control signal Cs from each sensor processing unit 31.
In the processing Sp82a, the evaluation unit 30e evaluates the control of the NN data Nd of each sensor processing unit 31 corresponding to the data set input in the processing Sp81a.
In the processing Sp83a, the evaluation unit 30e determines whether or not the control in each NN data Nd is appropriate based on the evaluation in the processing Sp82a. In the NN data Nd for which control is appropriate, the process proceeds to the process Sp81a. For NN data Nd for which control is not appropriate, the process proceeds to processing Sp84a.
In the process Sp84a, the process of FIG. 10B is performed.
図10(b)は、選択部30sでの処理を示す。
処理Sp81bでは、選択部30sが、各々のセンサ処理部31から検知情報Siを入力する。
処理Sp82bでは、選択部30sが、処理Sp81bで入力した検知情報Siをもとに車両2が走行する走行シーンDsを推定する。
処理Sp83bでは、選択部30sが、各々のセンサ処理部31に対して、処理Sp82bで推定した走行シーンDsに適するNNデータNdを選択する。
処理Sp84bでは、選択部30sが、各々のセンサ処理部31に対して、処理Sp83bで選択したNNデータNdに切り替えさせる切り替え指示を送信する。
FIG. 10B shows the processing in the selection unit 30s.
In the processing Sp81b, the selection unit 30s inputs the detection information Si from each sensor processing unit 31.
In the processing Sp82b, the selection unit 30s estimates the traveling scene Ds in which the vehicle 2 travels based on the detection information Si input in the processing Sp81b.
In the processing Sp83b, the selection unit 30s selects the NN data Nd suitable for the traveling scene Ds estimated by the processing Sp82b for each sensor processing unit 31.
In the processing Sp84b, the selection unit 30s transmits a switching instruction for switching to the NN data Nd selected in the processing Sp83b to each sensor processing unit 31.
なお、図10(a)及び(b)の一連の処理は独立して行われてもよい。また、図10(a)及び(b)の個々の処理は、次の処理の進捗によらず実行を開始させてもよい。例えば、各々のセンサ処理部31からデータセットが送信されれば、逐次入力を行う。 The series of processes shown in FIGS. 10A and 10B may be performed independently. Further, the individual processes of FIGS. 10 (a) and 10 (b) may be started to be executed regardless of the progress of the next process. For example, if a data set is transmitted from each sensor processing unit 31, sequential input is performed.
以上説明したように、実施の形態1によれば、種々の用途の動作を行う複数の装置と、当該複数の装置をそれぞれ制御する複数のAIとを統合したAI統合システム1において、AIを搭載した各々のセンサ処理部31は、システム1が置かれた環境の情報である検知情報Siを入力し、制御対象の装置を制御する制御信号Csを生成して出力する。各々のセンサ処理部31は、NNデータNdを複数保持しており、そのうち1つのNNデータNdをAIに設定することにより制御を行う。複数のNNデータNdは、システム1が置かれる環境に対応して、学習モデルに学習を行わせた学習済みモデルである。統括部30(評価部30e)は、各々のセンサ処理部31のAIに設定されたNNデータNdでの制御対象の装置への制御を評価する。また、統括部30(選択部30s)は、各々のセンサ処理部31に対して、推定される走行シーンDsと対応したNNデータNdを選択するとともに、選択したNNデータNdに切り替える指示を行って設定させる。その結果、AI統合システム1に、種々の用途の動作を行う各々の装置を制御する複数のAIを統合したとき、システム1が置かれた環境に適するNNデータNdを各々のセンサ処理部31に設定させて、種々の用途で動作する各々の装置を適切に制御させることを可能とする。 As described above, according to the first embodiment, the AI is mounted in the AI integrated system 1 in which a plurality of devices that perform operations for various purposes and a plurality of AIs that control the plurality of devices are integrated. Each of the sensor processing units 31 inputs the detection information Si, which is information on the environment in which the system 1 is placed, and generates and outputs control signals Cs for controlling the device to be controlled. Each sensor processing unit 31 holds a plurality of NN data Nd, and controls by setting one of the NN data Nd to AI. The plurality of NN data Nd is a trained model in which the learning model is trained according to the environment in which the system 1 is placed. The control unit 30 (evaluation unit 30e) evaluates the control of each sensor processing unit 31 to the device to be controlled by the NN data Nd set in the AI. Further, the control unit 30 (selection unit 30s) selects the NN data Nd corresponding to the estimated driving scene Ds to each sensor processing unit 31 and instructs each sensor processing unit 31 to switch to the selected NN data Nd. Let it be set. As a result, when a plurality of AIs that control each device that performs operations for various purposes are integrated into the AI integrated system 1, NN data Nd suitable for the environment in which the system 1 is placed is sent to each sensor processing unit 31. It can be set to appropriately control each device operating in various applications.
なお、AI統合システム1としては、実施の形態1で説明した車両2の他に、例えば、産業ロボット(つまり、ファクトリーオートメーション)、監視システム(又は監視装置)、空調システム(又は空調装置)、並びに、ホームエレクトロニクスなど、複数のAIを統合するものが対象となる。 In addition to the vehicle 2 described in the first embodiment, the AI integrated system 1 includes, for example, an industrial robot (that is, factory automation), a monitoring system (or monitoring device), an air conditioning system (or air conditioning device), and an air conditioning device. , Home electronics, etc. that integrate multiple AIs are targeted.
システム1に統合される複数のAIが複数の学習済みモデルを切り替えて制御を行う場合、複数のAIが個別に制御を行うと、制御対象の装置間において動作の整合が取れなくなるおそれがある。特に、システム1に統合されるAI間の制御の関連性が複雑となると、設計者が各々のAIの制御の整合性を考慮して、システム1が安定した状態となるようにAIの学習済みモデルを切り替えさせることは困難となる。この問題に対して、システム1が統合された複数のAIの制御によって不安定な状態に陥らないように、統括部30が各々のAIの制御を評価して、適切な学習済みモデルを設定させるようにしたので、種々の用途の動作を行う装置類をそれぞれ制御するAIを個別に学習させたとしても、学習済みモデルで制御を行う複数のAIをシステム1に統合したときに、システム1が安定して動作することが可能となる。 When a plurality of AIs integrated in the system 1 switch and control a plurality of trained models, if the plurality of AIs individually control, there is a possibility that the operation cannot be matched among the devices to be controlled. In particular, when the control relevance between AIs integrated into system 1 becomes complicated, the designer has learned AI so that the system 1 is in a stable state in consideration of the consistency of control of each AI. It is difficult to switch models. For this problem, the control unit 30 evaluates the control of each AI and sets an appropriate trained model so that the system 1 does not fall into an unstable state due to the control of a plurality of integrated AIs. Therefore, even if the AIs that control the devices that perform operations for various purposes are individually trained, when a plurality of AIs that are controlled by the trained model are integrated into the system 1, the system 1 will be used. It is possible to operate stably.
なお、AI統合システム1の統括部30は、1つの車載装置Vdと対応した1つのセンサ処理部に対して、複数の学習済みモデルのうちから1つを選択し設定させてもよい。 The control unit 30 of the AI integrated system 1 may select and set one of a plurality of trained models for one sensor processing unit corresponding to one in-vehicle device Vd.
<変形例>
図11は、統括部30の第1の変形例を説明するための模式図である。また、図12は、統括部30の第2の変形例を説明するための模式図である。図11に示すように、AI統合システム1の統括部30は、各々のセンサ処理部31のうち少なくともいずれかを含んでいてもよい。また、図12に示すように、AI統合システム1の統括部30が車両2の外部サーバにあり、車載ネットワーク40の通信を介して、車両2のセンサ処理部31などと情報のやり取りを行うようにしてもよい。
<Modification example>
FIG. 11 is a schematic diagram for explaining a first modification of the control unit 30. Further, FIG. 12 is a schematic diagram for explaining a second modification of the control unit 30. As shown in FIG. 11, the control unit 30 of the AI integrated system 1 may include at least one of the sensor processing units 31. Further, as shown in FIG. 12, the control unit 30 of the AI integrated system 1 is located on the external server of the vehicle 2, and information is exchanged with the sensor processing unit 31 of the vehicle 2 and the like via the communication of the in-vehicle network 40. You may do it.
実施の形態2.
実施の形態1では、種々の車載装置Vdを車両2に統合するのに先立って、車載装置Vdを制御するセンサ処理部31のAIに対し、複数の走行シーンDsごとに学習を行わせて走行シーンDsと対応した複数のNNデータNdを獲得させた。そして、車載装置Vdの車両2への統合後には、各々のセンサ処理部31に対し、車両2が走行する環境に応じて獲得した複数のNNデータNdを切り替えて設定させた。実施の形態2では、各々のセンサ処理部31が車両2の置かれた環境に応じて設定するNNデータNdに対して統合学習を行う。
Embodiment 2.
In the first embodiment, prior to integrating various vehicle-mounted devices Vd into the vehicle 2, the AI of the sensor processing unit 31 that controls the vehicle-mounted device Vd is made to learn for each of a plurality of driving scenes Ds. A plurality of NN data Nd corresponding to the scene Ds were acquired. Then, after the in-vehicle device Vd was integrated into the vehicle 2, each sensor processing unit 31 was made to switch and set a plurality of NN data Nd acquired according to the environment in which the vehicle 2 travels. In the second embodiment, integrated learning is performed on the NN data Nd set by each sensor processing unit 31 according to the environment in which the vehicle 2 is placed.
各々のセンサ処理部31が保持している複数のNNデータNdは、最終的な製品(つまり、完成車)として統合される前段階での学習によって獲得された学習済みモデルであり、最終的な製品として統合された状態において、制御対象となる車載装置Vdに対して適切な制御を行える保証はない。つまり、各々のセンサ処理部31が、学習済みの複数のNNデータNdを予め保持してこれらを切り替えながら制御対象となる車載装置Vdを制御することが可能だとしても、実際の環境を走行する完成車というシステム1で統合されたとき、切り替え指示Swにより設定されたNNデータNdが各々の車載装置Vdを安定して制御し続けることができるかどうか、信頼性を確証できないおそれがある。 The plurality of NN data Nd held by each sensor processing unit 31 is a trained model acquired by the learning in the previous stage integrated as the final product (that is, the finished vehicle), and is the final model. There is no guarantee that appropriate control can be performed for the in-vehicle device Vd to be controlled in the integrated state as a product. That is, even if each sensor processing unit 31 can hold a plurality of learned NN data Nd in advance and switch between them to control the in-vehicle device Vd to be controlled, the vehicle travels in an actual environment. When integrated in the system 1 of a completed vehicle, it may not be possible to confirm the reliability of whether or not the NN data Nd set by the switching instruction Sw can continue to stably control each in-vehicle device Vd.
ここで、制御における高いロバスト性とは、各々のセンサ処理部31が車両2に統合された後の、実際に車両2が走行する環境において、センサ処理部31のAI(つまり、学習過程の学習モデル又は学習済みモデル)が車載装置Vdの制御を行うとき、例えば、学習過程での教師データTdに含まれていなかった特徴を含む検知情報Siが入力されたり、教師データTdには含まれていたが組み合わせとして入力しなかった複数の特徴を含む検知情報Siが入力されたり、学習過程において考慮されなかった外乱を含む検知情報Siが入力されたりしても、制御対象の車載装置Vdを不安定な動作状態のままとすることなく、速やかに安定した動作状態に遷移させて制御を継続できる性質を指すものとする。 Here, the high robustness in control means the AI of the sensor processing unit 31 (that is, learning of the learning process) in the environment in which the vehicle 2 actually travels after each sensor processing unit 31 is integrated into the vehicle 2. When the in-vehicle device Vd is controlled by the model or the trained model, for example, the detection information Si including the feature not included in the teacher data Td in the learning process is input or is included in the teacher data Td. However, even if the detection information Si including a plurality of features that are not input as a combination is input, or the detection information Si including a disturbance that is not considered in the learning process is input, the in-vehicle device Vd to be controlled is not input. It refers to the property of being able to quickly transition to a stable operating state and continue control without leaving it in a stable operating state.
図13は、本開示の実施の形態2における、AIでの制御で扱う検知情報Si及び制御信号Csを2次元で表現する制御領域を説明するための模式図である。 FIG. 13 is a schematic diagram for explaining a control region for expressing the detection information Si and the control signal Cs handled in the control by AI in the second embodiment of the present disclosure in two dimensions.
統括部30は、完成車を模擬したシミュレーションによる解析、又は、実際の完成車を用いた実測によって得られた教師データTdに基づき、理論上は安定して車載装置Vdを制御できる検知情報Si及び制御信号Csの範囲によって表される制御可能な領域の情報を、各々のセンサ処理部31と対応させて保持している。 The control unit 30 has the detection information Si and the detection information Si that can theoretically stably control the in-vehicle device Vd based on the teacher data Td obtained by the analysis by the simulation simulating the completed vehicle or the actual measurement using the actual completed vehicle. Information in a controllable region represented by a range of control signals Cs is held in association with each sensor processing unit 31.
図13に示すように、種々の車載装置Vdに対するAIでの制御で扱う検知情報Si及び制御信号Csの2軸で構成される制御領域を考える。なお、種々の車載装置Vdの例として、駆動装置11、制動装置12、操舵装置13、緩衝装置14の4つを扱うものとする。 As shown in FIG. 13, consider a control region composed of two axes of detection information Si and control signal Cs handled by AI control for various in-vehicle devices Vd. As examples of various in-vehicle devices Vd, four devices, a drive device 11, a braking device 12, a steering device 13, and a shock absorber 14, will be handled.
駆動装置11を制御の対象とするセンサ処理部31Aの制御領域は、検知情報Siとして車両2の速度及び加速度を扱い、制御信号Csとして駆動力Dfを扱うものとする。また、制動装置12を制御の対象とするセンサ処理部31Bの制御領域は、検知情報Siとして路面の勾配及び摩擦力(つまり、各々のタイヤ4と路面とのグリップ度合い)を扱い、制御信号Csとして制動力Bfを扱うものとする。また、操舵装置13を制御の対象とするセンサ処理部31Cの制御領域は、検知情報Siとして車両2の向き及び位置の変化(例えば、変化量又は変化率)を扱い、制御信号Csとして操舵反応Sr(例えば、操舵量又は操舵速度)を扱うものとする。また、緩衝装置14を制御の対象とするセンサ処理部31Dの制御領域は、検知情報Siとして車両2の振動及び応力の変化(例えば、変化量又は変化率)を扱い、制御信号Csとして緩衝反応Cr(例えば、緩衝量又は緩衝速度)を扱うものとする。 The control area of the sensor processing unit 31A whose control target is the drive device 11 handles the speed and acceleration of the vehicle 2 as the detection information Si, and handles the driving force Df as the control signal Cs. Further, the control area of the sensor processing unit 31B whose control target is the braking device 12 handles the slope and frictional force of the road surface (that is, the degree of grip between each tire 4 and the road surface) as the detection information Si, and the control signal Cs. It is assumed that the braking force Bf is treated as. Further, the control area of the sensor processing unit 31C whose control target is the steering device 13 handles changes in the direction and position of the vehicle 2 (for example, the amount of change or the rate of change) as the detection information Si, and the steering reaction as the control signal Cs. Sr (for example, steering amount or steering speed) shall be dealt with. Further, the control area of the sensor processing unit 31D whose control target is the shock absorber 14 handles the vibration and stress change (for example, change amount or rate of change) of the vehicle 2 as the detection information Si, and the buffer reaction as the control signal Cs. Cr (eg, buffer amount or buffer rate) shall be dealt with.
制御領域上の楕円A1~A4,B1~B4,C1~C4及びD1~D4は、センサ処理部31A,31B,31C及び31Dが保持するNNデータNdA1~NdA4,NdB1~NdB4,NdC1~NdC4及びNdD1~NdD4を示しており、各々のNNデータNdが扱う検知情報Si及び制御信号Csの範囲を模式的に表している。 The ellipses A1 to A4, B1 to B4, C1 to C4 and D1 to D4 on the control region are NN data NdA1 to NdA4, NdB1 to NdB4, NdC1 to NdC4 and NdD1 held by the sensor processing units 31A, 31B, 31C and 31D. ~ NdD4 is shown, and the range of the detection information Si and the control signal Cs handled by each NN data Nd is schematically shown.
駆動装置11について具体例を挙げると、市街地の走行シーンDsに対応した楕円A1と、高速道路の走行シーンDsに対応した楕円A2とを比較すると、楕円A1より楕円A2の方が、数値が高い範囲において車両2の速度及び加速度を維持するように駆動力を制御することとなる。なお、高速道路への出入りなどの一時的な低速運転又は渋滞などの徐行運転は、市街地又は高速道路と対応する走行シーンDs及び教師データTdに含まれるものとする。また、市街地の走行シーンDsに対応した楕円A1と、山岳地の走行シーンDsに対応した楕円A3とを比較すると、車両2の速度及び加速度を維持する範囲は同程度であるが、楕円A1より楕円A3の方が、数値が高い範囲において駆動力を制御することとなる。また、市街地の走行シーンDsに対応した楕円A1と、郊外の未舗装路の走行シーンDsに対応した楕円A4とを比較すると、楕円A1より楕円A3の方が、数値が高い範囲において車両2の速度及び加速度を維持するとともに、数値が高い範囲において駆動力を制御することとなる。以下、この楕円枠を制御可能域Iaと定義する。 To give a specific example of the drive device 11, when comparing the ellipse A1 corresponding to the driving scene Ds in the urban area and the ellipse A2 corresponding to the driving scene Ds on the highway, the ellipse A2 has a higher numerical value than the ellipse A1. The driving force will be controlled so as to maintain the speed and acceleration of the vehicle 2 in the range. Temporary low-speed driving such as entering and exiting the expressway or slow driving such as traffic congestion shall be included in the driving scene Ds and the teacher data Td corresponding to the urban area or the expressway. Further, comparing the ellipse A1 corresponding to the driving scene Ds in the urban area and the ellipse A3 corresponding to the driving scene Ds in the mountainous area, the range in which the speed and acceleration of the vehicle 2 are maintained is the same, but the ellipse A1 The ellipse A3 controls the driving force in a range where the numerical value is higher. Comparing the ellipse A1 corresponding to the driving scene Ds in the urban area and the ellipse A4 corresponding to the driving scene Ds on the unpaved road in the suburbs, the ellipse A3 has a higher numerical value than the ellipse A1. The driving force will be controlled in the range where the numerical value is high while maintaining the speed and acceleration. Hereinafter, this elliptical frame is defined as the controllable area Ia.
制動装置12についても同様に、入力する検知情報Siと生成する制御信号Csとの範囲として、市街地の走行シーンDsに対応した制御可能域Iaである楕円B1、高速道路の走行シーンDsに対応した制御可能域Iaである楕円B2、山岳地の走行シーンDsに対応した制御可能域Iaである楕円B3、及び、未舗装路の走行シーンDsに対応した制御可能域Iaである楕円B4を考える。 Similarly, for the braking device 12, the range of the detection information Si to be input and the control signal Cs to be generated corresponds to the ellipse B1 which is the controllable area Ia corresponding to the driving scene Ds in the urban area and the driving scene Ds on the highway. Consider an ellipse B2 which is a controllable area Ia, an ellipse B3 which is a controllable area Ia corresponding to a driving scene Ds in a mountainous area, and an ellipse B4 which is a controllable area Ia corresponding to a driving scene Ds on an unpaved road.
操舵装置13についても同様に、入力する検知情報Siと生成する制御信号Csとの範囲として、市街地の走行シーンDsに対応した制御可能域Iaである楕円C1、高速道路の走行シーンDsに対応した制御可能域Iaである楕円C2、山岳地の走行シーンDsに対応した制御可能域Iaである楕円C3、及び、未舗装路の走行シーンDsに対応した制御可能域Iaである楕円C4を考える。 Similarly, for the steering device 13, the range of the detection information Si to be input and the control signal Cs to be generated corresponds to the ellipse C1 which is the controllable area Ia corresponding to the driving scene Ds in the urban area and the driving scene Ds on the highway. Consider an ellipse C2 which is a controllable area Ia, an ellipse C3 which is a controllable area Ia corresponding to a driving scene Ds in a mountainous area, and an ellipse C4 which is a controllable area Ia corresponding to a driving scene Ds on an unpaved road.
緩衝装置14についても同様に、入力する検知情報Siと生成する制御信号Csとの範囲として、市街地の走行シーンDsに対応した制御可能域Iaである楕円D1、高速道路の走行シーンDsに対応した制御可能域Iaである楕円D2、山岳地の走行シーンDsに対応した制御可能域Iaである楕円D3、及び、未舗装路の走行シーンDsに対応した制御可能域Iaである楕円D4を考える。 Similarly, for the shock absorber 14, the range of the detection information Si to be input and the control signal Cs to be generated corresponds to the ellipse D1 which is the controllable area Ia corresponding to the driving scene Ds in the urban area and the driving scene Ds on the highway. Consider an ellipse D2 which is a controllable area Ia, an ellipse D3 which is a controllable area Ia corresponding to a driving scene Ds in a mountainous area, and an ellipse D4 which is a controllable area Ia corresponding to a driving scene Ds on an unpaved road.
NNデータNdA1,NdA2,・・・,NdD4の各々は、車載装置Vd及びセンサ処理部31が車両2に統合されたとき、統合前の学習により獲得した制御可能域Iaにおいて、車両2を環境に適応させて走行制御を行うことを期待されている。 Each of the NN data NdA1, NdA2, ..., NdD4 puts the vehicle 2 in the environment in the controllable area Ia acquired by the learning before the integration when the in-vehicle device Vd and the sensor processing unit 31 are integrated into the vehicle 2. It is expected to adapt and control driving.
統括部30は、図13に示したような、各々のNNデータNdの制御可能域Iaを検知情報Si及び制御信号Csのデータセットで表される情報として予め保持している。評価部30eは、この制御可能域Iaの情報を用いて、車載装置Vdの動作状態の安定性の評価を行ったり、設定中のNNデータNdでの制御が適切に機能しているかどうかの評価を行ったりすることができる。また、選択部30sは、この制御可能域Iaの情報を用いて、現在の走行シーンDsに適するNNデータNdを選択し、切り替え指示Swを行うことができる。 The control unit 30 holds in advance the controllable area Ia of each NN data Nd as information represented by the data set of the detection information Si and the control signal Cs, as shown in FIG. The evaluation unit 30e uses the information in the controllable area Ia to evaluate the stability of the operating state of the in-vehicle device Vd and evaluate whether the control with the NN data Nd being set is functioning properly. Can be done. Further, the selection unit 30s can select the NN data Nd suitable for the current driving scene Ds by using the information of the controllable area Ia, and can perform the switching instruction Sw.
図14は、各々の車載装置Vdでの制御における相互の影響を説明するための模式図である。図14では、車両2が山岳地を走行する場合に、センサ処理部31A,31B,31C及び31DにおいてNNデータNdA3,NdB3,NdC3及びNdD3が制御を行うことを表している。また、実線矢印Eab,Eac及びEadは、センサ処理部31AのNNデータNdA3での制御が他のセンサ処理部31のNNデータNdB3,NdC3及びNdD3での制御と相互に影響し合うことを示す。 FIG. 14 is a schematic diagram for explaining the mutual influence in the control of each in-vehicle device Vd. FIG. 14 shows that when the vehicle 2 travels in a mountainous area, the sensor processing units 31A, 31B, 31C and 31D control the NN data NdA3, NdB3, NdC3 and NdD3. Further, the solid line arrows Eab, Eac and Ead indicate that the control of the sensor processing unit 31A by the NN data NdA3 interacts with the control of the other sensor processing units 31 by the NN data NdB3, NdC3 and NdD3.
駆動装置11におけるNNデータNdA3の制御可能域Iaの点Uaでの制御によって、制動装置12、操舵装置13及び緩衝装置14におけるNNデータNdB3,NdC3及びNdD3のそれぞれは、制御可能域Iaの点Ub,Uc及びUdで制御を行ったとする。このとき、点Ua,Ub,Uc及びUdは制御可能域Iaに含まれるため、評価部30eは、それぞれのNNデータNdの制御が適切に機能していると評価する。 By controlling at the point Ua of the controllable area Ia of the NN data NdA3 in the drive device 11, each of the NN data NdB3, NdC3 and NdD3 in the braking device 12, the steering device 13 and the shock absorber 14 is the point Ub of the controllable area Ia. , Uc and Ud are used for control. At this time, since the points Ua, Ub, Uc and Ud are included in the controllable area Ia, the evaluation unit 30e evaluates that the control of each NN data Nd is functioning appropriately.
また、駆動装置11におけるNNデータNdA3の制御可能域Iaの点Faでの制御によって、制動装置12、操舵装置13及び緩衝装置14におけるNNデータNdB3,NdC3及びNdD3のそれぞれは、制御可能域Iaの点Fb,Fc及びFdで制御を行ったとする。このとき、点Fa及びFbは制御可能域Iaに含まれるが、点Fc及びFdは制御可能域Iaに含まれないため、評価部30eは、それぞれのNNデータNdの制御が適切に機能していないと評価する。しかしながら、点Fcは操舵装置13を制御の対象とするNNデータNdでのロバスト性に基づいた制御によって、次の制御のタイミングにおけるNNデータNdでの制御が制御可能域Iaに含まれたとすると、安定した走行状態に車両2を維持できる可能性が生じる。 Further, by controlling at the point Fa of the controllable area Ia of the NN data NdA3 in the drive device 11, each of the NN data NdB3, NdC3 and NdD3 in the braking device 12, the steering device 13 and the shock absorber 14 is in the controllable area Ia. It is assumed that control is performed at points Fb, Fc and Fd. At this time, the points Fa and Fb are included in the controllable area Ia, but the points Fc and Fd are not included in the controllable area Ia. Evaluate not. However, assuming that the point Fc includes the control in the NN data Nd at the timing of the next control by the control based on the robustness in the NN data Nd for which the steering device 13 is controlled, the controllable range Ia. There is a possibility that the vehicle 2 can be maintained in a stable running state.
このように、一般に、学習済みモデルを含む制御系(言い換えると、制御システム)は、多少のロバスト性を有することが期待される。つまり、センサ処理部31のNNデータNdは、このロバスト性を有することにより、制御可能域Iaを超えた破線楕円枠の領域においても、制御対象となる車載装置Vdの制御を行えることが期待される。以下、この破線楕円枠をロバスト制御域Raと定義する。 As described above, in general, the control system including the trained model (in other words, the control system) is expected to have some robustness. That is, it is expected that the NN data Nd of the sensor processing unit 31 can control the in-vehicle device Vd to be controlled even in the region of the broken line elliptical frame beyond the controllable range Ia by having this robustness. To. Hereinafter, this broken line elliptical frame is defined as the robust control area Ra.
このようなロバスト制御域Raでの制御は、例えば、車両2のタイヤ4の空気圧又はブレーキ圧、並びに、積載物による車体3の重量又は受ける風圧など、一時的又は経時的な走行特性の変化に対しても制御が行えることが望まれる。 Such control in the robust control range Ra is for temporary or temporal changes in running characteristics such as the air pressure or brake pressure of the tire 4 of the vehicle 2 and the weight or wind pressure of the vehicle body 3 due to the load. However, it is desirable to be able to control it.
しかしながら、ロバスト制御域Raは、不確定な領域であって、設計によって意図したとおりのロバスト性を制御系(言い換えると、制御システム)に与えることは困難である。そのため、センサ処理部31が保持するNNデータNdが有するロバスト性では対応できない適応外領域Naが存在する可能性がある。 However, the robust control region Ra is an uncertain region, and it is difficult to provide the control system (in other words, the control system) with the robustness intended by the design. Therefore, there is a possibility that there is an out-of-applicable region Na that cannot be dealt with by the robustness of the NN data Nd held by the sensor processing unit 31.
図14では、実線矢印Eab,Eac及びEadと同様に、センサ処理部31B,31C及び31Dの各々のNNデータNdでの制御が相互に影響し合うことを破線矢印Ebc,Ebd及びEcdで示す。上述のとおり、各々の車載装置Vdは、NNデータNdA3,NdB3,NdC3及びNdD3の制御可能域Ia以外の不確定なロバスト制御域Raにおいて制御される可能性がある。このような不確定なロバスト制御域Raにおいて各々の車載装置Vdが制御された結果、車両2というシステム1のうえで適切に連動するとは限らない。つまり、実線矢印Eab,Eac及びEad、並びに、破線矢印Ebc,Ebd及びEcdに示した相互の影響に起因して、あるNNデータNdでのロバスト制御域Raにおける制御の結果が、他のNNデータNdに対して適応外領域Naでの制御に繋がり、当該他のNNデータNdが制御できない状況に陥るおそれがある。 In FIG. 14, similarly to the solid line arrows Eab, Eac and Ead, the dashed line arrows Ebc, Ebd and Ecd indicate that the controls of the sensor processing units 31B, 31C and 31D in the NN data Nd affect each other. As described above, each in-vehicle device Vd may be controlled in an uncertain robust control area Ra other than the controllable area Ia of the NN data NdA3, NdB3, NdC3 and NdD3. As a result of controlling each in-vehicle device Vd in such an uncertain robust control area Ra, it is not always properly linked on the system 1 of the vehicle 2. That is, due to the mutual influence shown by the solid line arrows Eab, Eac and Ed, and the dashed line arrows Ebc, Ebd and Edd, the result of the control in the robust control area Ra in one NN data Nd is the other NN data. It may lead to control in the non-adaptive region Na with respect to Nd, and may fall into a situation where the other NN data Nd cannot be controlled.
このような適応外領域Naは、車載装置Vdが車両2に統合される前であっても、センサ処理部31の各々のNNデータNdが制御できる範囲を評価することにより、ある程度は予測が可能である。しかしながら、適応外領域Naを狭めるための統合学習(つまり、最終的な制御系である完成車のロバスト性を高めるための追加の学習)を、車両2に統合される前の個々のNNデータNdに対して(つまり、学習済みモデルとしてのNNデータNdを獲得する過程で)行うことは困難である。 Such a non-adaptive region Na can be predicted to some extent by evaluating the range in which each NN data Nd of the sensor processing unit 31 can be controlled even before the in-vehicle device Vd is integrated into the vehicle 2. Is. However, the integrated learning to narrow the non-adaptive region Na (that is, the additional learning to enhance the robustness of the finished vehicle, which is the final control system), is the individual NN data Nd before being integrated into the vehicle 2. (That is, in the process of acquiring NN data Nd as a trained model).
各々の車載装置Vdを制御するセンサ処理部31のNNデータNdが適応外領域Naで制御する状況に陥らないように、各々のNNデータNdでの制御のロバスト性を高めることを考える。 It is considered to enhance the robustness of the control in each NN data Nd so that the NN data Nd of the sensor processing unit 31 that controls each in-vehicle device Vd does not fall into the situation of being controlled in the non-adaptive region Na.
図15は、AIによる制御が有するロバスト性を説明するための模式図である。図14に示す適応外領域Naでの制御に陥らないために、車両2が山岳地を走行する場合、各々の車載装置Vdを制御するNNデータNdA3,NdB3,NdC3及びNdD3のロバスト制御域Raが拡張されることで、あるNNデータNdでの制御の結果、他のNNデータNdでの制御がロバスト制御域Raの内側に含まれる可能性が高まる。これにより、各々のNNデータNdは制御対象の制御を速やかに制御可能域Ia内に遷移させ、その結果、安定した制御を継続する可能性が高まる。 FIG. 15 is a schematic diagram for explaining the robustness of the control by AI. When the vehicle 2 travels in a mountainous area, the robust control areas Ra of the NN data NdA3, NdB3, NdC3 and NdD3 that control each in-vehicle device Vd are set so as not to fall into the control in the non-adaptive region Na shown in FIG. By expanding, as a result of the control in one NN data Nd, the possibility that the control in another NN data Nd is included in the robust control area Ra increases. As a result, each NN data Nd rapidly shifts the control of the controlled object into the controllable area Ia, and as a result, the possibility of continuing stable control increases.
図15では、NNデータNdC3での制御がロバスト制御域Ra内で行われた結果、NNデータNdD3での制御は拡張されたロバスト制御域Ra内となったことを示す。これにより、次の制御のタイミングにおいて、NNデータNdC3及びNdD3は、速やかに制御可能域Ia内での制御に復帰できるようになる。 FIG. 15 shows that as a result of the control in the NN data NdC3 being performed in the robust control area Ra, the control in the NN data NdD3 is in the extended robust control area Ra. As a result, at the timing of the next control, the NN data NdC3 and NdD3 can quickly return to the control within the controllable area Ia.
図6に示すように、統括部30は、学習部30aを備える。統括部30は、図13に示したような、各々のセンサ処理部31における複数のNNデータNdA1,NdA2,・・・,NdD4のデータセットの制御可能域Iaの情報を予め保持している。そして、センサ処理部31に設定中のNNデータNdが制御可能域Ia外において制御を行い、その後、速やかに制御可能域Ia内での制御に復帰した場合、学習部30aは、このときの制御可能域Ia外での制御に対応したデータセットを当該NNデータNdのロバスト制御域Raとして、予め保持する制御可能域Iaの情報に対し追加及び更新を行う。 As shown in FIG. 6, the control unit 30 includes a learning unit 30a. The control unit 30 previously holds information on the controllable area Ia of the data sets of the plurality of NN data NdA1, NdA2, ..., NdD4 in each sensor processing unit 31 as shown in FIG. Then, when the NN data Nd set in the sensor processing unit 31 controls outside the controllable area Ia and then promptly returns to the control within the controllable area Ia, the learning unit 30a controls at this time. The data set corresponding to the control outside the possible area Ia is set as the robust control area Ra of the NN data Nd, and the information of the controllable area Ia held in advance is added and updated.
ここで、NNデータNdでの制御が速やかに制御可能域Ia内に復帰したかどうかの判定は、車両2を構成する種々の車載装置Vdの動作状態が安定した状態にあるか、又は、安定した状態に遷移しているか、評価部30eが予め設計で定められた期間における安定性を評価することにより行うことができる。 Here, in the determination of whether or not the control in the NN data Nd is promptly returned to the controllable range Ia, the operating state of the various in-vehicle devices Vd constituting the vehicle 2 is stable or stable. This can be done by whether the transition to the above-mentioned state has occurred or by the evaluation unit 30e evaluating the stability in a period predetermined by the design.
図16は、AIでの制御におけるロバスト性が拡張された状態を説明するための模式図である。図16に示すように、山岳地での走行シーンDsと同様に、他の走行シーンDsにおいてもNNデータNdのロバスト制御域Raが拡張されれば、より安定して車両2の走行を制御することが可能となる。 FIG. 16 is a schematic diagram for explaining a state in which robustness in control by AI is expanded. As shown in FIG. 16, if the robust control range Ra of the NN data Nd is expanded in other driving scenes Ds as well as the driving scene Ds in the mountainous area, the traveling of the vehicle 2 can be controlled more stably. It becomes possible.
図17は、走行シーンDsと対応したAIの制御領域の重複部分を示す模式図である。ところで、図17に示すように、センサ処理部31が保持する走行シーンDsに対応した複数のNNデータNdの制御可能域Ia及びロバスト制御域Raを合わせた領域には、重複する重複制御域Daが存在する可能性がある。このような重複制御域Daに含まれるデータセットの扱いとしては、例えば、他の設定中のNNデータNdの走行シーンDsとの整合性又は親和性がある走行シーンDsに対応したNNデータNdに含まれるものと判定したり、重複制御域Daに遷移したデータセットがその直前に含まれていた領域のNNデータNdに含まれるものと判定したりすることが考えられる。つまり、重複制御域Daに含まれるデータセットが、次に遷移する見込みの高い領域と対応するNNデータNdに含まれるように判定する。 FIG. 17 is a schematic diagram showing an overlapping portion of the AI control area corresponding to the traveling scene Ds. By the way, as shown in FIG. 17, the overlapping control area Da is in the area where the controllable area Ia and the robust control area Ra of a plurality of NN data Nd corresponding to the traveling scene Ds held by the sensor processing unit 31 are combined. May exist. As the handling of the data set included in such an overlapping control area Da, for example, the NN data Nd corresponding to the driving scene Ds having consistency or affinity with the driving scene Ds of the NN data Nd being set may be used. It is conceivable that it is determined that the data set is included, or that the data set that has transitioned to the overlapping control area Da is included in the NN data Nd of the region that was included immediately before that. That is, it is determined that the data set included in the overlap control region Da is included in the NN data Nd corresponding to the region with a high possibility of transitioning next.
図18は、システム1としての車両2と、種々の車載装置Vdを制御するサブシステムとしてのAIと、さらに統括部30との関係を説明するための模式図である。 FIG. 18 is a schematic diagram for explaining the relationship between the vehicle 2 as the system 1, the AI as a subsystem that controls various in-vehicle devices Vd, and the control unit 30.
システム1としての車両2(つまり、完成車)において、統括部30は、センサ処理部31とその制御対象の車載装置Vdとの組をそれぞれサブシステムとして扱う。 In the vehicle 2 as the system 1 (that is, the completed vehicle), the control unit 30 treats each pair of the sensor processing unit 31 and the in-vehicle device Vd to be controlled as a subsystem.
各々のサブシステムは、統合前の学習によって獲得されたNNデータNdを用いて、車両2というシステム1への統合後、検知情報Siの入力と制御信号Csの生成を行いながら、各々のサブシステムの用途の動作を行う。 Each subsystem uses the NN data Nd acquired by the learning before the integration, and after the integration into the system 1 called the vehicle 2, inputs the detection information Si and generates the control signal Cs, and each subsystem Perform the intended use of.
ところで、用途が同じサブシステムであっても、統合されるシステム1としての完成車の車種(例えば、車体3重量、重心、車幅、及びホイールベースなど)が異なったり、車種が同じであっても、仕様(例えば、ハイブリッドエンジン若しくはモーターエンジン、二輪駆動若しくは四輪駆動、並びに、排気量など)又は装備(例えば、タイヤ4、ホイール及びヘッドライト6など)が異なったりすれば、同じ環境で、同じコースを、同じ速度で走行したとしても、各々のサブシステムのセンサ処理部31が入力する検知情報Siと生成する制御信号Csとは異なってくる。 By the way, even if the subsystems have the same purpose, the vehicle type (for example, the weight of the vehicle body 3 weight, the center of gravity, the vehicle width, the wheel base, etc.) of the completed vehicle as the integrated system 1 is different, or the vehicle type is the same. However, if the specifications (eg, hybrid engine or motor engine, two-wheel drive or four-wheel drive, and exhaust volume, etc.) or equipment (eg, tire 4, wheels, headlight 6, etc.) are different, in the same environment. Even if the same course is traveled at the same speed, the detection information Si input by the sensor processing unit 31 of each subsystem and the generated control signal Cs are different.
つまり、システム1としての完成車はプロパティ(つまり、車種、仕様又は装備などの種々のパラメータによって定まる特性)がそれぞれで異なる。言い換えると、完成車はそれぞれ複数の固有のパラメータを持っている。そのため、種々の車載装置Vdを制御するNNデータNdを連動させて統合学習を行わせるためには、車両2が持つ複数の固有のパラメータが及ぼす車載装置Vdへの影響を考慮する必要がある。 That is, the completed vehicle as the system 1 has different properties (that is, characteristics determined by various parameters such as vehicle type, specifications, and equipment). In other words, each finished vehicle has multiple unique parameters. Therefore, in order to perform integrated learning by interlocking the NN data Nd that controls various in-vehicle devices Vd, it is necessary to consider the influence of a plurality of unique parameters of the vehicle 2 on the in-vehicle device Vd.
このことは、統合前の学習によって獲得されるNNデータNdの制御可能域Iaが同じであっても、車両2のプロパティが異なれば、制御によりデータセットが遷移する領域も異なってくることを意味する。そのため、車両2(つまり、完成車)ごとにNNデータNdの制御可能域Iaが適切に形成されていることが望ましく、その結果としてロバスト制御域Raが適切に拡張されることが期待できる。 This means that even if the controllable area Ia of the NN data Nd acquired by the learning before integration is the same, if the properties of the vehicle 2 are different, the area where the data set transitions will also be different due to the control. do. Therefore, it is desirable that the controllable area Ia of the NN data Nd is appropriately formed for each vehicle 2 (that is, the completed vehicle), and as a result, the robust control area Ra can be expected to be appropriately expanded.
そこで、車両2への統合前の学習によって獲得されるNNデータNdの制御可能域Iaを基礎として、車両2への統合後に、NNデータNdに対して統合学習を行わせることを考える。つまり、NNデータNdの制御可能域Iaを再形成させ、さらに再形成された制御可能域Iaを基礎としてロバスト制御域Raを拡張させることを目的に、統合後の統合学習を行わせる。 Therefore, based on the controllable area Ia of the NN data Nd acquired by the learning before the integration into the vehicle 2, it is considered that the NN data Nd is subjected to the integrated learning after the integration into the vehicle 2. That is, the integrated learning after integration is performed for the purpose of reforming the controllable region Ia of the NN data Nd and further expanding the robust control region Ra based on the reformed controllable region Ia.
以下、各々のセンサ処理部31が保持するNNデータNdを統合学習させる方法について説明する。なお、統合後のNNデータNdの統合学習は、実際の環境において、又は、ある走行シーンDsを模擬した環境において、完成車を走行させることにより行われる。これらの環境には統合前の学習での走行シーンDsに含まれる特徴的な要素と同等の要素が含まれるものとする。 Hereinafter, a method for integrated learning of the NN data Nd held by each sensor processing unit 31 will be described. The integrated learning of the NN data Nd after the integration is performed by driving the completed vehicle in an actual environment or in an environment simulating a certain driving scene Ds. It is assumed that these environments include elements equivalent to the characteristic elements included in the driving scene Ds in the learning before integration.
車両2への統合後、選択部30sは、入力する検知情報Siをもとに走行シーンDsを推定し、各々のセンサ処理部31が保持する複数のNNデータNdのうち、推定した走行シーンDsに対応するNNデータNdを選択して、各々のセンサ処理部31に対して設定させる。ここで、統合学習においては、選択部30sが行う走行シーンDsの推定とNNデータNdの選択及び設定とを人が行ってもよい。 After integration into the vehicle 2, the selection unit 30s estimates the driving scene Ds based on the input detection information Si, and among the plurality of NN data Nd held by each sensor processing unit 31, the estimated driving scene Ds. The NN data Nd corresponding to is selected and set for each sensor processing unit 31. Here, in the integrated learning, a person may perform the estimation of the traveling scene Ds performed by the selection unit 30s and the selection and setting of the NN data Nd.
また、各々のセンサ処理部31が保持するNNデータNdには、学習済みモデルを変化させない追加学習非対応モードと、追加の学習によって学習済みモデルを変化させることが可能な追加学習対応モードとを、切り替えて設定できるものとする。なお、車両2への統合時点では、各々のセンサ処理部31のNNデータNdは追加学習非対応モードに設定されるものとする。なお、学習部30aからの追加学習対応モードの指示を追加学習対応モードAL、学習部30aからの追加学習非対応モードの切り替え指示を追加学習非対応モードNLとする。 Further, the NN data Nd held by each sensor processing unit 31 has an additional learning non-compatible mode in which the trained model is not changed and an additional learning compatible mode in which the trained model can be changed by additional learning. , Can be switched and set. At the time of integration into the vehicle 2, the NN data Nd of each sensor processing unit 31 is set to the additional learning non-compliant mode. The instruction of the additional learning compatible mode from the learning unit 30a is referred to as the additional learning compatible mode AL, and the instruction of switching the additional learning non-compatible mode from the learning unit 30a is referred to as the additional learning non-compatible mode NL.
先ず、学習部30aが、各々のセンサ処理部31に対して設定中のNNデータNdを追加学習対応モードに切り替えさせ、各々のセンサ処理部31のNNデータNdに独立して一斉に統合学習を行わせる例について説明する。 First, the learning unit 30a switches the NN data Nd being set for each sensor processing unit 31 to the additional learning compatible mode, and the NN data Nd of each sensor processing unit 31 independently and simultaneously performs integrated learning. An example to be performed will be described.
ここでは、統合学習を行うための走行シーンDsとして高速道路を扱う。そして、車両2ごとに異なるプロパティとして、2つの車両2A及び2Bを扱うものとする。車両2Aは、排気量が大きく、馬力が高く、車体3の重量が重い車両2(例えば、排気量が4000cc以上のもの)とする。車両2Bは、排気量が小さく、馬力が低く、車体3の重量が軽い車両2(例えば、排気量が660cc以下のもの)とする。 Here, the expressway is treated as a driving scene Ds for performing integrated learning. Then, it is assumed that two vehicles 2A and 2B are handled as different properties for each vehicle 2. The vehicle 2A is a vehicle 2 having a large displacement, a high horsepower, and a heavy weight of the vehicle body 3 (for example, a vehicle having a displacement of 4000 cc or more). The vehicle 2B is a vehicle 2 having a small displacement, a low horsepower, and a light weight of the vehicle body 3 (for example, a vehicle having a displacement of 660 cc or less).
高速道路では、車両2A及び車両2Bは、走行速度の維持又は他車の追い越しをするための加速、ジャンクション付近での車線変更又はカーブのための減速、並びに、進行方向を変えるための操舵を行う。 On highways, vehicles 2A and 2B accelerate to maintain driving speed or overtake other vehicles, lane change or decelerate for curves near junctions, and steer to change direction of travel. ..
以下、種々の車載装置Vdのうち、駆動装置11、制動装置12及び操舵装置13、並びに、これら3つの車載装置Vdを制御の対象とするセンサ処理部31A、センサ処理部31Bの及びCを扱うものとする。
また、センサ処理部31A、センサ処理部31Bの及びCに設定される、高速道路に対応したNNデータNdをNNデータNdA2,NdB2及びNdC2とする。NNデータNdA2,NdB2及びNdC2は制御対象の車載装置Vdの制御を行いつつ、独立して一斉に統合学習を行わせる。
Hereinafter, among various in-vehicle devices Vd, the drive device 11, the braking device 12, the steering device 13, and the sensor processing unit 31A, the sensor processing unit 31B, and C that control these three in-vehicle devices Vd are handled. It shall be.
Further, the NN data Nd corresponding to the expressway set in the sensor processing unit 31A, the sensor processing unit 31B and C is referred to as NN data NdA2, NdB2 and NdC2. The NN data NdA2, NdB2, and NdC2 control the vehicle-mounted device Vd to be controlled, and independently perform integrated learning all at once.
高速道路において、車両2AがRの小さいカーブに差し掛かるとき、車両2Aの制動装置12を制御するNNデータNdB2aは、車両2Aの重量が重いため、カーブに差し掛かる前に十分に走行速度を落としきれなかったとする。NNデータNdB2aは、カーブに差し掛かる時点で走行速度が速すぎるため、制御可能域Iaでの制御では走行の安全を確保できないと判定し、他のNNデータNdの制御とは独立して、制御可能域Iaを超える大きい制動力で制動装置12を制御する。また、車両2Aの操舵装置13を制御するNNデータNdC2aは、カーブに差し掛かる時点で走行速度が速すぎるため、制御可能域Iaでの制御では走行の安全を確保できないと判定し、他のNNデータNdの制御とは独立して、制御可能域Iaを超える大きい操舵量と操舵速度で操舵装置13を制御する。その結果、車両2Aは強めのブレーキと急なステアリングによって、タイヤ4のグリップが利きづらくなり、スリップしてしまうことが考えられる。 On the highway, when the vehicle 2A approaches a curve with a small radius, the NN data NdB2a that controls the braking device 12 of the vehicle 2A slows down sufficiently before approaching the curve because the weight of the vehicle 2A is heavy. Suppose you couldn't finish it. Since the traveling speed of the NN data NdB2a is too fast at the time of approaching the curve, it is determined that the driving safety cannot be ensured by the control in the controllable range Ia, and the control is performed independently of the control of the other NN data Nd. The braking device 12 is controlled with a large braking force exceeding the possible range Ia. Further, the NN data NdC2a that controls the steering device 13 of the vehicle 2A determines that the traveling safety cannot be ensured by the control in the controllable range Ia because the traveling speed is too fast at the time of approaching the curve, and other NNs. Independent of the control of the data Nd, the steering device 13 is controlled with a large steering amount and steering speed exceeding the controllable range Ia. As a result, it is conceivable that the vehicle 2A will slip due to the hard grip of the tire 4 due to the strong braking and the sudden steering.
このように、制動制御Bcより先に又は一斉に操舵制御Scを行ってしまうと、タイヤ4と路面との摩擦力が車体3の慣性力に負けてスリップし、車両2の進行方向を目標に近づけることが困難となる。 In this way, if the steering control Sc is performed before the braking control Bc or all at once, the frictional force between the tire 4 and the road surface loses to the inertial force of the vehicle body 3 and slips, aiming at the traveling direction of the vehicle 2. It becomes difficult to get close.
車両2Aの駆動装置11を制御するNNデータNdA2aは、車両2Bよりも重い車体3を加速させるために、車両2Bの駆動装置11を制御するNNデータNdA2bに比べて高い駆動力の範囲で制御を行うこととなる。その結果、駆動力を制御するための制御量の変動(つまり、変化量又は変化率)が急峻となる。このため、学習を行うにしたがって、NNデータNdA2aの制御はNNデータNdA2bに比べて、駆動力を高める傾向が強くなると考えられる。 The NN data NdA2a that controls the drive device 11 of the vehicle 2A controls within a range of higher driving force than the NN data NdA2b that controls the drive device 11 of the vehicle 2B in order to accelerate the vehicle body 3 that is heavier than the vehicle 2B. Will be done. As a result, the fluctuation of the controlled amount for controlling the driving force (that is, the amount of change or the rate of change) becomes steep. Therefore, it is considered that the control of the NN data NdA2a has a stronger tendency to increase the driving force as compared with the NN data NdA2b as the learning is performed.
また、車両2Aの制動装置12を制御するNNデータNdB2aは、車両2Bよりも重い車体3を減速させるために、車両2Bの制動装置12を制御するNNデータNdB2bに比べて高い制動力の範囲で制御を行うこととなる。その結果、制動力を制御するための制御量の変動(つまり、変化量又は変化率)が急峻となる。このため、学習を行うにしたがって、NNデータNdB2aの制御はNNデータNdB2bに比べて、制動力を高める傾向が強くなると考えられる。 Further, the NN data NdB2a that controls the braking device 12 of the vehicle 2A has a higher braking force range than the NN data NdB2b that controls the braking device 12 of the vehicle 2B in order to decelerate the vehicle body 3 that is heavier than the vehicle 2B. It will be controlled. As a result, the fluctuation of the controlled amount for controlling the braking force (that is, the amount of change or the rate of change) becomes steep. Therefore, it is considered that the control of the NN data NdB2a has a stronger tendency to increase the braking force as compared with the NN data NdB2b as the learning is performed.
また、車両2Aの操舵装置13を制御するNNデータNdC2aは、車両2Bよりも重くホイールベースが長い車体3を操舵するために、車両2Bの制動装置12を制御するNNデータNdC2bに比べて操舵角の制御が複雑となる。それは、車体3が重いほど進行方向への慣性が大きくなるため、新たな目標となる進行方向に向かうフィードバック制御が安定しづらくなるからである。また、ホイールベースが長いほど回転半径が大きくなるため、新たな目標となる進行方向に向かうには広い範囲で操舵角を制御することとなるからである。その結果、操舵を制御するための制御量の変動(つまり、変化量又は変化率)が大きくなる。このため、学習を行うにしたがって、NNデータNdC2aの制御はNNデータNdC2bに比べて、操舵角を大きく頻繁に変える傾向が強くなると考えられる。 Further, the NN data NdC2a that controls the steering device 13 of the vehicle 2A has a steering angle as compared with the NN data NdC2b that controls the braking device 12 of the vehicle 2B in order to steer the vehicle body 3 that is heavier than the vehicle 2B and has a long wheelbase. Control becomes complicated. This is because the heavier the vehicle body 3, the greater the inertia in the traveling direction, which makes it difficult to stabilize the feedback control toward the traveling direction, which is a new target. In addition, the longer the wheelbase, the larger the turning radius, so the steering angle must be controlled over a wide range in order to move toward the new target direction of travel. As a result, the fluctuation (that is, the amount of change or the rate of change) of the control amount for controlling the steering becomes large. Therefore, it is considered that the control of the NN data NdC2a has a stronger tendency to change the steering angle more frequently than the NN data NdC2b as the learning is performed.
一方で、高速道路において、車両2BがRの小さいカーブに差し掛かるとき、車両2Bの制動装置12を制御するNNデータNdB2bは、車両2Bの重量が軽いため、カーブに差し掛かる前に十分に走行速度を落とすことができる。そのため、NNデータNdB2bは制御可能域Iaにおいて制動装置12を制御することができる。 On the other hand, on the highway, when the vehicle 2B approaches a curve with a small R, the NN data NdB2b that controls the braking device 12 of the vehicle 2B travels sufficiently before approaching the curve because the weight of the vehicle 2B is light. You can slow down. Therefore, the NN data NdB2b can control the braking device 12 in the controllable area Ia.
また、車両2Bの操舵装置13を制御するNNデータNdC2bは、カーブに差し掛かる時点で十分に走行速度が落ちているため、制御可能域Iaにおいて操舵装置13を制御することができる。その結果、車両2Bは適度なブレーキとステアリングにより、タイヤ4のグリップが利いており、スリップすることはない。 Further, since the NN data NdC2b that controls the steering device 13 of the vehicle 2B has a sufficiently low traveling speed at the time of approaching the curve, the steering device 13 can be controlled in the controllable range Ia. As a result, the vehicle 2B has a good grip of the tire 4 due to appropriate braking and steering, and does not slip.
車両2Bは、排気量が小さく、車体3の重量が軽く、ホイールベースが短いため、車両2Aでの制御よりも制御量の変動(つまり、変化量又は変化率)が緩やかとなる。 Since the vehicle 2B has a small displacement, the weight of the vehicle body 3 is light, and the wheelbase is short, the fluctuation of the control amount (that is, the change amount or the rate of change) is slower than that of the control by the vehicle 2A.
統合前の学習済みモデルであるNNデータNdをそのまま車両2Aの制御に用いたとすると、NNデータNdが事前の学習過程(又は、教師データTd)をもとに車両2Aを制御しようとしても、実際の環境での車両2Aの動作の変動(つまり、制御信号Csに対する車両2の状態を示すデータセットの変動)が激しいため、事前の教師データTdで獲得した制御可能域Iaから外れてロバスト制御域Raでの制御をとる可能性が比較的大きくなると考えられる。このことは、ロバスト制御域Raでの制御から速やかに制御可能域Iaでの制御に遷移しづらくする。このような場合、統合学習を行わせたとしても、統合前の学習済みモデルの制御可能域Iaを車両2Aに適した制御可能域Iaに再形成させられる可能性は低い。 Assuming that the NN data Nd, which is a trained model before integration, is used as it is for controlling the vehicle 2A, even if the NN data Nd tries to control the vehicle 2A based on the prior learning process (or the teacher data Td), it is actually Since the fluctuation of the operation of the vehicle 2A in the environment of (that is, the fluctuation of the data set indicating the state of the vehicle 2 with respect to the control signal Cs) is severe, the robust control region deviates from the controllable range Ia acquired by the prior teacher data Td. It is considered that the possibility of taking control by Ra is relatively large. This makes it difficult to quickly transition from the control in the robust control area Ra to the control in the controllable area Ia. In such a case, even if integrated learning is performed, it is unlikely that the controllable area Ia of the trained model before integration can be reformed into the controllable area Ia suitable for the vehicle 2A.
一方で、統合前の学習済みモデルであるNNデータNdをそのまま車両2Bの制御に用いたとすると、NNデータNdが事前の学習過程(又は、教師データTd)をもとに車両2Bを制御したとき、実際の環境での車両2Bの動作の変動(つまり、制御信号Csに対する車両2の状態を示すデータセットの変動)が緩やかであるため、事前の教師データTdで獲得した制御可能域Iaから外れてロバスト制御域Raでの制御をとる可能性が比較的小さくなると考えられる。このことは、ロバスト制御域Raでの制御から速やかに制御可能域Iaでの制御に遷移しやすくする。このような場合、統合学習を行わせることで、統合前の学習済みモデルの制御可能域Iaを車両2Bに適した制御可能域Iaに再形成させられる可能性は高い。 On the other hand, assuming that the NN data Nd, which is a trained model before integration, is used as it is for controlling the vehicle 2B, when the NN data Nd controls the vehicle 2B based on the prior learning process (or teacher data Td). Since the fluctuation of the operation of the vehicle 2B in the actual environment (that is, the fluctuation of the data set indicating the state of the vehicle 2 with respect to the control signal Cs) is gradual, it deviates from the controllable range Ia acquired by the prior teacher data Td. Therefore, it is considered that the possibility of taking control in the robust control range Ra is relatively small. This facilitates a rapid transition from control in the robust control area Ra to control in the controllable area Ia. In such a case, it is highly possible that the controllable area Ia of the trained model before integration can be reformed into the controllable area Ia suitable for the vehicle 2B by performing the integrated learning.
このように、各々のNNデータNdを統合させるシステム1によっては、統合学習を秩序立てずに、独立して一斉に行わせると、各々のNNデータNdでの制御が環境に対して適切に機能せず、その結果、学習がスムーズに収束せず、統合後の各々のAIがシステム1における用途を果たせない場合が生じるおそれがある。つまり、統合学習を行ったとしてもNNデータNdの制御可能域Iaが適切に再形成されず、システム1の動作が不安定のままとなるおそれがある。 In this way, depending on the system 1 that integrates each NN data Nd, if the integrated learning is performed independently and simultaneously without ordering, the control by each NN data Nd functions appropriately for the environment. As a result, the learning may not converge smoothly, and each AI after integration may not be able to fulfill the purpose in the system 1. That is, even if integrated learning is performed, the controllable area Ia of the NN data Nd may not be properly reshaped, and the operation of the system 1 may remain unstable.
次に、学習部30aが、制御の優先度Pを考慮し、優先度Pの高いセンサ処理部31に対して設定中のNNデータNdを追加学習対応モードに切り替えさせ、各々のセンサ処理部31のNNデータNdに秩序立てて統合学習を行わせる例について説明する。 Next, the learning unit 30a considers the control priority P, causes the sensor processing unit 31 having a high priority P to switch the NN data Nd being set to the additional learning compatible mode, and each sensor processing unit 31. An example of causing the NN data Nd of the above to perform integrated learning in an orderly manner will be described.
図19は、制御の優先度Pを考慮したときの各々のNNデータNdにおける統合学習の収束度合いを説明するための模式図である。車両2の走行シーンDsとして、高速道路を扱うものとする。 FIG. 19 is a schematic diagram for explaining the degree of convergence of integrated learning in each NN data Nd when the control priority P is taken into consideration. It is assumed that the highway is treated as the traveling scene Ds of the vehicle 2.
図19(a1)~(a3)は、制御の優先度Pとして、駆動制御Dc、制動制御Bc、及び操舵制御Scの順に統合学習を行っている。 In FIGS. 19 (a1) to 19 (a3), integrated learning is performed in the order of drive control Dc, braking control Bc, and steering control Sc as control priority P.
図19(a1)は、センサ処理部31Aが、NNデータNdA2を用いて駆動制御Dcを行ったときの、駆動装置11への制御信号Csの送信回数に対する制御の安定評価Seの遷移を示している。 FIG. 19A1 shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the drive device 11 when the sensor processing unit 31A performs the drive control Dc using the NN data NdA2. There is.
制御信号Csの送信回数に記した1Mはある所定の回数を示しており、10Mは1Mの10倍を示す。また、制御の安定評価Seは、検知情報Siから得られる情報をもとに導出される評価値によって、評価部30eで判定することができる。そして、評価部30eは、評価値の変動が収束判断値Conv以下となったことで学習が収束したと判断する。 1M described in the number of transmissions of the control signal Cs indicates a predetermined number of times, and 10M indicates 10 times 1M. Further, the control stability evaluation Se can be determined by the evaluation unit 30e based on the evaluation value derived based on the information obtained from the detection information Si. Then, the evaluation unit 30e determines that the learning has converged because the fluctuation of the evaluation value is equal to or less than the convergence determination value Conv.
駆動装置11及び制動装置12に対する制御の安定評価Seは、例えば、走行速度の変化量及び変化率、目標の走行距離における目標の走行速度への到達度合い、並びに、燃費などの情報を用いて、評価部30eで判定することができる。また、操舵装置13に対する制御の安定評価Seは、例えば、目標の走行距離における目標の走行軌道への到達度合い、並びに、当該軌道上での車両2の姿勢などの情報を用いて、評価部30eで判定することができる。また、緩衝装置14に対する制御の安定評価Seは、例えば、車両2に働く慣性モーメント、各タイヤ4に掛かる荷重、車両2の傾き、並びに、車両2の振動などの情報を用いて、評価部30eで判定することができる。 The stability evaluation Se of the control for the drive device 11 and the braking device 12 uses, for example, information such as the amount and rate of change in the traveling speed, the degree of achievement of the target traveling speed in the target mileage, and fuel efficiency. It can be determined by the evaluation unit 30e. Further, the stability evaluation Se of the control for the steering device 13 uses, for example, information such as the degree of arrival at the target traveling track at the target mileage and the posture of the vehicle 2 on the track, and the evaluation unit 30e. Can be determined by. Further, the stability evaluation Se of the control for the shock absorber 14 uses information such as the moment of inertia acting on the vehicle 2, the load applied to each tire 4, the inclination of the vehicle 2, and the vibration of the vehicle 2 to be used in the evaluation unit 30e. Can be determined by.
図19(a1)に示すように、NNデータNdA2は、制御信号Csの送信回数が10Mを過ぎたあたりで評価値の変動は収束判断値Conv以下に収束する。このときの制御信号Csの送信回数をTa1とする。 As shown in FIG. 19 (a1), in the NN data NdA2, the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs exceeds 10M. The number of transmissions of the control signal Cs at this time is set to Ta1.
図19(a2)は、センサ処理部31Dが、NNデータNdD2を用いて制動制御Bcを行ったときの、制動装置12への制御信号Csの送信回数に対する制御の安定評価Seの遷移を示している。制御信号Csの送信回数及び制御の安定評価Seについては、図19(a1)と同様とする。図19(a2)に示すように、NNデータNdD2は、制御信号Csの送信回数が10Mを過ぎたあたりで評価値の変動は収束判断値Conv以下に収束する。このときの制御信号Csの送信回数をTa2とする。 FIG. 19A2 shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the braking device 12 when the sensor processing unit 31D performs the braking control Bc using the NN data NdD2. There is. The number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1). As shown in FIG. 19A2, in the NN data NdD2, the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs exceeds 10M. The number of transmissions of the control signal Cs at this time is set to Ta2.
図19(a3)は、センサ処理部31Bが、NNデータNdB2を用いて操舵制御Scを行ったときの、操舵装置13への制御信号Csの送信回数に対する制御の安定評価Seの遷移を示している。制御信号Csの送信回数及び制御の安定評価Seについては、図19(a1)と同様とする。図19(a3)に示すように、NNデータNdB2は、制御信号Csの送信回数が10Mに達する手前で評価値の変動は収束判断値Conv以下に収束する。このときの制御信号Csの送信回数をTa3とする。 FIG. 19A3 shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the steering device 13 when the sensor processing unit 31B performs the steering control Sc using the NN data NdB2. There is. The number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1). As shown in FIG. 19 (a3), in the NN data NdB2, the fluctuation of the evaluation value converges to the convergence judgment value Conv or less before the number of transmissions of the control signal Cs reaches 10M. The number of transmissions of the control signal Cs at this time is set to Ta3.
図19(a1)~(a3)では、NNデータNdA2,NdB2及びNdD2は、制御信号Csの送信回数が増加するのに伴い、評価値の変動が収束している。つまり、この優先度Pで制御を行っていくことで、NNデータNdA2,NdB2及びNdD2は制御対象の車載装置Vdを安定して制御できるようになる。 In FIGS. 19 (a1) to 19 (a3), the fluctuations of the evaluation values of the NN data NdA2, NdB2 and NdD2 converge as the number of transmissions of the control signal Cs increases. That is, by performing control with this priority P, the NN data NdA2, NdB2 and NdD2 can stably control the in-vehicle device Vd to be controlled.
図20は、制御の優先度Pを考慮したときの各々のNNデータNdにおける統合学習の収束度合いを説明するための別の模式図である。車両2の走行シーンDsとして、高速道路を扱うものとする。
図20(b1)~(b3)は、制御の優先度Pとして、操舵制御Sc、駆動制御Dc、及び制動制御Bcの順に統合学習を行っている。
FIG. 20 is another schematic diagram for explaining the degree of convergence of integrated learning in each NN data Nd when the control priority P is taken into consideration. It is assumed that the highway is treated as the traveling scene Ds of the vehicle 2.
In FIGS. 20 (b1) to 20 (b3), integrated learning is performed in the order of steering control Sc, drive control Dc, and braking control Bc as control priority P.
図20(b1)は、センサ処理部31Bが、NNデータNdB2を用いて操舵制御Scを行ったときの、操舵装置13への制御信号Csの送信回数に対する制御の安定評価Seの遷移を示している。制御信号Csの送信回数に記した1Mはある所定の回数を示しており、10Mは1Mの10倍を示し、100Mは1Mの100倍を示す。制御信号Csの送信回数及び制御の安定評価Seについては、図19(a1)と同様とする。図20(b1)に示すように、NNデータNdB2は、制御信号Csの送信回数が100Mを超えたところで評価値の変動は収束判断値Conv以下に収束する。このときの制御信号Csの送信回数をTb1とする。 FIG. 20 (b1) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the steering device 13 when the sensor processing unit 31B performs the steering control Sc using the NN data NdB2. There is. 1M described in the number of transmissions of the control signal Cs indicates a predetermined number of times, 10M indicates 10 times of 1M, and 100M indicates 100 times of 1M. The number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1). As shown in FIG. 20 (b1), in the NN data NdB2, the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs exceeds 100M. The number of transmissions of the control signal Cs at this time is Tb1.
図20(b2)は、センサ処理部31Aが、NNデータNdA2を用いて駆動制御Dcを行ったときの、駆動装置11への制御信号Csの送信回数に対する制御の安定評価Seの遷移を示している。制御信号Csの送信回数及び制御の安定評価Seについては、図19(a1)と同様とする。図20(b2)に示すように、NNデータNdA2は、制御信号Csの送信回数が100Mを超えたところで評価値の変動は収束判断値Conv以下に収束する。このときの制御信号Csの送信回数をTb2とする。 FIG. 20 (b2) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the drive device 11 when the sensor processing unit 31A performs the drive control Dc using the NN data NdA2. There is. The number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1). As shown in FIG. 20 (b2), in the NN data NdA2, the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs exceeds 100M. The number of transmissions of the control signal Cs at this time is Tb2.
図20(b3)は、センサ処理部31Dが、NNデータNdD2を用いて制動制御Bcを行ったときの、制動装置12への制御信号Csの送信回数に対する制御の安定評価Seの遷移を示している。制御信号Csの送信回数及び制御の安定評価Seについては、図19(a1)と同様とする。図20(b3)に示すように、NNデータNdD2は、制御信号Csの送信回数が100Mを超えたところで評価値の変動は収束判断値Conv以下に収束する。このときの制御信号Csの送信回数をTb3とする。 FIG. 20 (b3) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the braking device 12 when the sensor processing unit 31D performs the braking control Bc using the NN data NdD2. There is. The number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1). As shown in FIG. 20 (b3), in the NN data NdD2, the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs exceeds 100M. The number of transmissions of the control signal Cs at this time is Tb3.
図20(b1)~(b3)では、NNデータNdA2,NdB2及びNdD2は、図19(a1)~(a3)の場合と比べて、評価値の変動が収束するのに、より多くの制御信号Csの送信を要している。つまり、図20(b1)~(b3)では、この優先度Pで制御を行っていくことで、NNデータNdA2,NdB2及びNdD2は制御対象の車載装置Vdを安定して制御できるようにはなるが、図19(a1)~(a3)の場合と比べると、制御が適切に行われなかったときのデータセットを多く含んでおり、そのため、獲得された制御可能域Ia及びロバスト制御域Raが図19(a1)~(a3)の場合と比べて適切ではないおそれがある。 In FIGS. 20 (b1) to 20 (b3), the NN data NdA2, NdB2 and NdD2 have more control signals even though the fluctuation of the evaluation value converges as compared with the case of FIGS. 19 (a1) to 19 (a3). It requires transmission of Cs. That is, in FIGS. 20 (b1) to 20 (b3), by performing control with this priority P, the NN data NdA2, NdB2 and NdD2 can stably control the in-vehicle device Vd to be controlled. However, as compared with the cases of FIGS. 19 (a1) to 19 (a3), it contains a large amount of data sets when the control is not performed properly, so that the acquired controllable area Ia and robust control area Ra are included. It may not be appropriate as compared with the cases of FIGS. 19 (a1) to 19 (a3).
図21は、制御の優先度Pを考慮したときの各々のNNデータNdにおける統合学習の収束度合いを説明するためのさらに別の模式図である。車両2の走行シーンDsとして、高速道路を扱うものとする。
図21(c1)~(c4)は、制御の優先度Pとして、緩衝制御Cc、駆動制御Dc、制動制御Bc、及び操舵制御Scの順に統合学習を行っている。
FIG. 21 is yet another schematic diagram for explaining the degree of convergence of integrated learning in each NN data Nd when the control priority P is taken into consideration. It is assumed that the highway is treated as the traveling scene Ds of the vehicle 2.
In FIGS. 21 (c1) to 21 (c4), integrated learning is performed in the order of buffer control Cc, drive control Dc, braking control Bc, and steering control Sc as control priority P.
図21(c1)は、センサ処理部31Cが、NNデータNdC2を用いて緩衝制御Ccを行ったときの、緩衝装置14への制御信号Csの送信回数に対する制御の安定評価Seの遷移を示している。制御信号Csの送信回数及び制御の安定評価Seについては、図19(a1)と同様とする。図21(c1)に示すように、NNデータNdC2は、制御信号Csの送信回数が10Mに達しないところで評価値の変動は収束判断値Conv以下に収束する。このときの制御信号Csの送信回数をDc1とする。 FIG. 21 (c1) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the shock absorber 14 when the sensor processing unit 31C performs the buffer control Cc using the NN data NdC2. There is. The number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1). As shown in FIG. 21 (c1), in the NN data NdC2, the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs does not reach 10M. Let Dc1 be the number of times the control signal Cs is transmitted at this time.
図21(c2)は、センサ処理部31Aが、NNデータNdA2を用いて駆動制御Dcを行ったときの、駆動装置11への制御信号Csの送信回数に対する制御の安定評価Seの遷移を示している。制御信号Csの送信回数及び制御の安定評価Seについては、図19(a1)と同様とする。図21(c2)に示すように、NNデータNdA2は、制御信号Csの送信回数が10Mに達しないところで評価値の変動は収束判断値Conv以下に収束する。このときの制御信号Csの送信回数をDc2とする。 FIG. 21 (c2) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the drive device 11 when the sensor processing unit 31A performs the drive control Dc using the NN data NdA2. There is. The number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1). As shown in FIG. 21 (c2), in the NN data NdA2, the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs does not reach 10M. The number of transmissions of the control signal Cs at this time is Dc2.
図21(c3)は、センサ処理部31Dが、NNデータNdD2を用いて制動制御Bcを行ったときの、制動装置12への制御信号Csの送信回数に対する制御の安定評価Seの遷移を示している。制御信号Csの送信回数及び制御の安定評価Seについては、図19(a1)と同様とする。図21(c3)に示すように、NNデータNdD2は、制御信号Csの送信回数が10Mに達しないところで評価値の変動は収束判断値Conv以下に収束する。このときの制御信号Csの送信回数をDc3とする。 FIG. 21 (c3) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the braking device 12 when the sensor processing unit 31D performs the braking control Bc using the NN data NdD2. There is. The number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1). As shown in FIG. 21 (c3), in the NN data NdD2, the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs does not reach 10M. Let Dc3 be the number of transmissions of the control signal Cs at this time.
図21(c4)は、センサ処理部31Bが、NNデータNdB2を用いて操舵制御Scを行ったときの、操舵装置13への制御信号Csの送信回数に対する制御の安定評価Seの遷移を示している。制御信号Csの送信回数及び制御の安定評価Seについては、図19(a1)と同様とする。図21(c4)に示すように、NNデータNdB2は、制御信号Csの送信回数が10Mに達しないところで評価値の変動は収束判断値Conv以下に収束する。このときの制御信号Csの送信回数をDc4とする。 FIG. 21 (c4) shows the transition of the control stability evaluation Se with respect to the number of transmissions of the control signal Cs to the steering device 13 when the sensor processing unit 31B performs the steering control Sc using the NN data NdB2. There is. The number of transmissions of the control signal Cs and the control stability evaluation Se are the same as in FIG. 19 (a1). As shown in FIG. 21 (c4), in the NN data NdB2, the fluctuation of the evaluation value converges to the convergence judgment value Conv or less when the number of transmissions of the control signal Cs does not reach 10M. The number of transmissions of the control signal Cs at this time is Dc4.
図21(c2)~(c4)では、NNデータNdA2,NdB2及びNdD2は、図19(a1)~(a3)の場合と比べて、評価値の変動が収束するのに、より少ない制御信号Csの送信で済んでいる。つまり、図21(c2)~(c4)では、この優先度Pで制御を行っていくことで、NNデータNdA2,NdB2及びNdD2は制御対象の車載装置Vdを安定して制御できるようになる。さらに、図19(a1)~(a3)の場合と比べると、制御が適切に行われたデータセットを多く含んでおり、そのため、獲得された制御可能域Ia及びロバスト制御域Raが図19(a1)~(a3)の場合と比べて適切であることが期待できる。 In FIGS. 21 (c2) to 21 (c4), the NN data NdA2, NdB2 and NdD2 have less control signals Cs even though the fluctuation of the evaluation value converges as compared with the case of FIGS. 19 (a1) to 19 (a3). It is enough to send. That is, in FIGS. 21 (c2) to 21 (c4), by performing control with this priority P, the NN data NdA2, NdB2 and NdD2 can stably control the vehicle-mounted device Vd to be controlled. Further, as compared with the cases of FIGS. 19 (a1) to 19 (a3), a large number of appropriately controlled data sets are included, and therefore, the acquired controllable area Ia and robust control area Ra are shown in FIG. 19 (a1). It can be expected that it is more appropriate than the cases of a1) to (a3).
なお、評価部30eは、各々のNNデータNdでの統合学習が収束するかどうかの判定について、評価値の変動が収束判断値Conv以下に収束する前に、例えば、制御信号Csの送信回数が所定回数に達したり、所定時間が経過したりすることにより行うことができる。
所定回数及び所定時間は、走行シーンDs及びセンサ処理部31ごとに統括部30に予め設定されてもよいし、統括部30が自車両2での統合学習を通じて車両2のプロパティ及び走行シーンDsに基づき導出したり、車載ネットワーク40を介して取得したりすることにより設定されてもよい。
In addition, the evaluation unit 30e determines whether or not the integrated learning in each NN data Nd converges, for example, the number of transmissions of the control signal Cs before the fluctuation of the evaluation value converges to the convergence judgment value Conv or less. This can be done by reaching a predetermined number of times or by elapse of a predetermined time.
The predetermined number of times and the predetermined time may be set in advance in the control unit 30 for each of the driving scene Ds and the sensor processing unit 31, or the control unit 30 sets the properties of the vehicle 2 and the driving scene Ds through integrated learning in the own vehicle 2. It may be set by deriving the vehicle based on the vehicle or acquiring the vehicle via the in-vehicle network 40.
なお、NNデータNdでの統合学習が収束しない場合、学習部30aは、選択部30sに対してNNデータNdの再選択指示Rsと切り替え指示Swとを行わせてもよい。 If the integrated learning with the NN data Nd does not converge, the learning unit 30a may cause the selection unit 30s to perform the reselection instruction Rs and the switching instruction Sw of the NN data Nd.
図19(a1)~(a3)及び図20(b1)~(b3)での駆動制御Dc、制動制御Bc及び操舵制御Scにおける統合学習の収束度合いの違いは、統合学習の優先度Pを駆動制御Dc、制動制御Bc及び操舵制御Scの順に行ったことである。 The difference in the degree of convergence of the integrated learning in the drive control Dc, the braking control Bc, and the steering control Sc in FIGS. 19 (a1) to (a3) and 20 (b1) to (b3) drives the priority P of the integrated learning. The control Dc, the braking control Bc, and the steering control Sc were performed in this order.
駆動制御Dc、制動制御Bc及び操舵制御Scの3つの制御項目Ciの優先度Pは次のように考えられる。駆動制御Dcが適切に行われず、過度な加速により走行速度が非常に高い状態において、制動制御Bc及び操舵制御Scを行ったとしても、タイヤ4と路面との間の摩擦力(つまり、タイヤ4のグリップ)が車体3の慣性力に負けてスリップしてしまうおそれがある。そのため、駆動制御Dcが適切に行われない状態で車両2を安定した状態に維持させることは困難である。したがって、3つの制御項目Ciのうち駆動制御Dcが適切に行われることが最も優先される。 The priority P of the three control items Ci of the drive control Dc, the braking control Bc, and the steering control Sc is considered as follows. Even if the braking control Bc and the steering control Sc are performed in a state where the drive control Dc is not properly performed and the traveling speed is very high due to excessive acceleration, the frictional force between the tire 4 and the road surface (that is, the tire 4) is performed. The grip) may slip due to the inertial force of the vehicle body 3. Therefore, it is difficult to maintain the vehicle 2 in a stable state in a state where the drive control Dc is not properly performed. Therefore, among the three control items Ci, the one in which the drive control Dc is appropriately performed has the highest priority.
また、駆動制御Dcが適切に行われて走行速度が適度である状態において、制動制御Bcより優先的に操舵制御Scを行ってしまうと、タイヤ4と路面との間の摩擦力(つまり、タイヤ4のグリップ)が車体3の慣性力に負けてスリップしてしまうおそれがある。そのため、制動制御Bcが適切に行われない状態で操舵制御Scを行ったとしても車両2を安定した状態に維持させることは困難である。したがって、3つの制御項目Ciのうち制動制御Bcが適切に行われることが次に優先される。 Further, if the steering control Sc is performed preferentially over the braking control Bc in a state where the drive control Dc is appropriately performed and the traveling speed is appropriate, the frictional force between the tire 4 and the road surface (that is, the tire) is performed. The grip of 4) may slip due to the inertial force of the vehicle body 3. Therefore, it is difficult to maintain the vehicle 2 in a stable state even if the steering control Sc is performed in a state where the braking control Bc is not properly performed. Therefore, among the three control items Ci, the one in which the braking control Bc is appropriately performed has the next priority.
また、図19(a1)~(a3)及び図21(c2)~(c4)での駆動制御Dc、制動制御Bc及び操舵制御Scにおける統合学習の収束度合いの違いは、図21(c1)の緩衝制御Ccにおける統合学習を優先的に行ったことである。 Further, the difference in the degree of convergence of the integrated learning in the drive control Dc, the braking control Bc, and the steering control Sc in FIGS. 19 (a1) to (a3) and 21 (c2) to (c4) is shown in FIG. 21 (c1). This is the priority given to the integrated learning in the buffer control Cc.
例えば、車両2が凹凸の激しい未舗装面を走行する場合、タイヤ4と路面との間の摩擦力が低下してグリップが利きづらく、車両2の走行が安定しない状態となる。このような状態では、駆動制御Dc、制動制御Bc及び操舵制御Scを適切に行うことは困難となる。 For example, when the vehicle 2 travels on an unpaved surface with severe unevenness, the frictional force between the tire 4 and the road surface decreases, the grip becomes difficult to use, and the vehicle 2 travels in an unstable state. In such a state, it becomes difficult to properly perform the drive control Dc, the braking control Bc, and the steering control Sc.
車両2の走行時の安定性を確保するうえで、慣性力に抗して車体3の傾き(又は姿勢)及び重心の位置が制御されること、また、各々のタイヤ4が路面に対して均等な荷重で接地することが望ましい。車体3の傾きが抑えられていると、車両2の重心の位置が変動しづらくなり、各々のタイヤ4が路面に対して均等な荷重で接地しやすくなる。また、各々のタイヤ4が路面に対して均等な荷重で接地していると、各々のタイヤ4と路面との間の摩擦力が損なわれにくくグリップが利きた状態を維持しやすくなる。緩衝装置14は、各々のタイヤ4の緩衝度合い(つまり、変化量又は変化速度)を変化させることによって、車両2の重心の位置を変化させる(つまり、車体3の前後左右の傾きと重量のバランスを変化させる)ことができる。これにより、車両2が加速、減速及びカーブを行うとき、車両2の進行方向とは別の方向に働く慣性力に抗して路面に力を伝達させるタイヤ4の摩擦力(つまり、グリップ)をより向上させることが可能となる。 In order to ensure the stability of the vehicle 2 during traveling, the inclination (or posture) of the vehicle body 3 and the position of the center of gravity are controlled against the inertial force, and each tire 4 is equal to the road surface. It is desirable to ground with a moderate load. When the inclination of the vehicle body 3 is suppressed, the position of the center of gravity of the vehicle 2 is less likely to fluctuate, and it becomes easier for each tire 4 to come into contact with the road surface with an even load. Further, when each tire 4 is in contact with the road surface with an even load, the frictional force between each tire 4 and the road surface is not easily impaired, and it becomes easy to maintain a state in which the grip is effective. The shock absorber 14 changes the position of the center of gravity of the vehicle 2 by changing the degree of cushioning (that is, the amount of change or the speed of change) of each tire 4 (that is, the balance between the front / rear / left / right inclination and the weight of the vehicle body 3). Can be changed). As a result, when the vehicle 2 accelerates, decelerates, and curves, the frictional force (that is, grip) of the tire 4 that transmits the force to the road surface against the inertial force acting in the direction different from the traveling direction of the vehicle 2 is applied. It will be possible to improve it further.
例えば、車両2が前方に加速する際には、慣性力によって車両2が後方に傾くが、この慣性力に抗して緩衝装置14が後輪の緩衝度合いを強めるように(つまり、後輪が沈みにくくなるように)動作することで、後輪に慣性力が集中して後輪の摩擦力が負けてしまうのを抑制し、結果としてタイヤ4からの前進する回転力を路面に伝わりやすくすることができる。 For example, when the vehicle 2 accelerates forward, the vehicle 2 tilts backward due to the inertial force, but the shock absorber 14 increases the degree of cushioning of the rear wheels against this inertial force (that is, the rear wheels By operating (to prevent it from sinking), it is possible to prevent the inertial force from concentrating on the rear wheels and losing the frictional force of the rear wheels, and as a result, the forward rotational force from the tire 4 is easily transmitted to the road surface. be able to.
また、車両2が走行中にカーブする際には、慣性力によって車両2がカーブする向きとは反対に傾くが、この慣性力に抗して緩衝装置14がカーブの外側の車輪の緩衝度合いを強めるように(つまり、カーブの外側の車輪が沈みにくくなるように)動作することで、カーブ外側の車輪に慣性力が集中して車輪の摩擦力が負けてしまうのを抑制し、つまり、左右の車輪に適度に慣性力を分散させ、結果としてタイヤ4からの進行方向を変更するための反力を路面に伝わりやすくすることができる。 Further, when the vehicle 2 curves while traveling, the vehicle 2 tilts in the direction opposite to the direction in which the vehicle 2 curves due to the inertial force. By acting to strengthen (that is, to prevent the wheels on the outside of the curve from sinking), it prevents the inertial force from concentrating on the wheels on the outside of the curve and losing the frictional force of the wheels, that is, left and right. The inertial force can be appropriately distributed to the wheels of the tire 4, and as a result, the reaction force for changing the traveling direction from the tire 4 can be easily transmitted to the road surface.
なお、カーブに伴い車両2が減速すると車両2が前方に傾くが、前輪に荷重が掛かることによってカーブの際の摩擦力(つまり、進行方向を変更するための反力)を強める作用が生じる。その一方で、前輪に荷重が掛かり過ぎる(言い換えると、車両2が前方に傾き過ぎる)ことによって後輪の摩擦力を弱める作用(つまり、後輪のスリップ)が生じる。このような過度な作用によって後輪のスリップを生じさせないように、緩衝装置14がカーブの外側の前後の車輪の緩衝度合いを調整することが望ましい。 When the vehicle 2 decelerates along with the curve, the vehicle 2 tilts forward, but the load applied to the front wheels increases the frictional force (that is, the reaction force for changing the traveling direction) at the curve. On the other hand, too much load is applied to the front wheels (in other words, the vehicle 2 tilts too much forward), which causes an action of weakening the frictional force of the rear wheels (that is, slipping of the rear wheels). It is desirable that the shock absorber 14 adjusts the degree of cushioning of the front and rear wheels on the outside of the curve so that the rear wheels do not slip due to such excessive action.
統合学習によって車両2に適した緩衝制御Ccが行われることにより、車体3の傾きを抑制したり、各々のタイヤ4が路面に対して均等な荷重で接地させたりすることが可能となる。その結果、各々のタイヤ4と路面との間の摩擦力の低下を抑制してグリップを利きやすくする作用をもたらすことになる。したがって、駆動制御Dc、制動制御Bc及び操舵制御Scよりも緩衝制御Ccを優先的に行うことは、車両2の走行における安定性を確保しやすくする。 By performing the buffer control Cc suitable for the vehicle 2 by the integrated learning, it becomes possible to suppress the inclination of the vehicle body 3 and to make each tire 4 touch the road surface with an even load. As a result, it is possible to suppress a decrease in the frictional force between each tire 4 and the road surface to bring about an action of making the grip easier to use. Therefore, giving priority to the buffer control Cc over the drive control Dc, the braking control Bc, and the steering control Sc makes it easy to secure the stability of the vehicle 2 in running.
前述の高速道路での走行例では、プロパティが異なる車両2A及びBの駆動制御Dc、制動制御Bc及び操舵制御Scの優先度Pを扱った。そこでは、車両2Bより車両2Aのほうが、排気量、馬力及び車体3の重量が大きいため、その分、制御量の変動も大きくなり、統合前の学習によって獲得されたNNデータNdでは適切に制御が行われないおそれがあることを説明した。 In the above-mentioned driving example on the highway, the priority P of the drive control Dc, the braking control Bc, and the steering control Sc of the vehicles 2A and B having different properties is dealt with. There, since the displacement, horsepower, and weight of the vehicle body 3 are larger in the vehicle 2A than in the vehicle 2B, the fluctuation of the control amount is also large by that amount, and the NN data Nd acquired by the learning before the integration is appropriately controlled. Explained that may not be done.
しかしながら、上述の凹凸の激しい未舗装面を走行する場合、車両2Aに比べて車両2Bは、車体3及び車輪のサイズが小さく重量が軽いため、路面の凹凸を走行するうえで安定性を確保しづらい。このような場合、緩衝制御Ccが適切に行われるように優先的に統合学習を行わせることで、走行時の安定性を確保しやすくなる。ここでの緩衝制御Ccとは、例えば、車体3の傾き、タイヤ4が乗り上げる凹凸の大きさ、又は、カーブで働く慣性力の大きさなどに応じて、各々のタイヤ4に対する緩衝装置14の緩衝度合いの強弱を変えたり、緩衝動作のレスポンスを変えたりすることが考えられる。 However, when traveling on the above-mentioned heavily uneven unpaved surface, the vehicle 2B has a smaller size and lighter weight of the vehicle body 3 and wheels than the vehicle 2A, so that stability is ensured when traveling on the uneven road surface. It's hard. In such a case, by preferentially performing integrated learning so that the buffer control Cc is appropriately performed, it becomes easy to secure stability during traveling. The cushioning control Cc here is, for example, the cushioning of the shock absorber 14 for each tire 4 according to the inclination of the vehicle body 3, the size of the unevenness on which the tire 4 rides, the size of the inertial force acting on the curve, and the like. It is conceivable to change the strength of the degree or change the response of the buffering operation.
上述の凹凸が激しい未舗装面での走行例では、統合学習によって緩衝制御Ccが適切に行われるようになると、駆動制御Dc、制動制御Bc及び操舵制御Scでの統合学習がより適切に行われることが期待される。 In the above-mentioned traveling example on an unpaved surface with severe unevenness, when the buffer control Cc is appropriately performed by the integrated learning, the integrated learning by the drive control Dc, the braking control Bc and the steering control Sc is performed more appropriately. It is expected.
上述のとおり、システム1への統合後のNNデータNdを統合学習させる順序によっては、各々のNNデータNdでの制御が環境に対して適切に機能せず、その結果、学習がスムーズに収束せず、統合後の各々のAIがシステム1における用途を果たせない場合が生じるおそれがある。つまり、統合学習を行ったとしてもNNデータNdの制御可能域Iaが適切に再形成されず、システム1の動作が不安定のままとなるおそれがある。 As described above, depending on the order in which the NN data Nd after integration into the system 1 is integrated and learned, the control in each NN data Nd does not function properly for the environment, and as a result, the learning converges smoothly. However, there is a possibility that each AI after integration may not be able to fulfill the purpose in the system 1. That is, even if integrated learning is performed, the controllable area Ia of the NN data Nd may not be properly reshaped, and the operation of the system 1 may remain unstable.
そのため、学習部30aは、図19~図21に示される安定評価Seを行った結果、統合学習させているNNデータNdでの制御が環境に対して適切に機能していないと判定する場合には、制御の優先度Pを導出する過程にフィードバックして、制御の優先度Pを変更して統合学習を行わせることにより、制御対象となる車載装置Vdへの制御の過不足を抑制することが可能となる。 Therefore, when the learning unit 30a performs the stability evaluation Se shown in FIGS. 19 to 21 and determines that the control in the NN data Nd to be integrated learning is not functioning properly with respect to the environment. Feeds back to the process of deriving the control priority P, changes the control priority P to perform integrated learning, thereby suppressing excess or deficiency of control to the in-vehicle device Vd to be controlled. Is possible.
また、このような安定評価Seを行った結果は、制御に対する各々の車載装置Vdの動作の特性を示す情報として蓄積し、車両2のプロパティ又は走行シーンDsの特徴における影響を分析することにより、統合前の学習における教師データTd及び学習過程での評価指標、並びに、統合後の統合学習における走行シーンDsの推定及び制御の優先度Pの導出などに活用することができる。 Further, the result of performing such stability evaluation Se is accumulated as information indicating the characteristics of the operation of each in-vehicle device Vd with respect to the control, and the influence on the property of the vehicle 2 or the characteristics of the driving scene Ds is analyzed. It can be used for the teacher data Td in the learning before the integration, the evaluation index in the learning process, the estimation of the driving scene Ds in the integrated learning after the integration, and the derivation of the priority P of the control.
図22は、車両2に関わる物理量が制御項目Ciに影響を及ぼすメカニズムの一例を説明するための模式図である。 FIG. 22 is a schematic diagram for explaining an example of a mechanism in which a physical quantity related to the vehicle 2 affects the control item Ci.
制御項目Ciとして、ここでは、駆動制御Dc、制動制御Bc、操舵制御Sc、及び、緩衝制御Ccを扱う。また、車両2に関わる種々のパラメータとして、ここでは、車体3、車幅、ホイールベース、走行速度、重量(又は質量)、及び、重心を扱う。 As the control item Ci, the drive control Dc, the braking control Bc, the steering control Sc, and the buffer control Cc are dealt with here. Further, as various parameters related to the vehicle 2, the vehicle body 3, the vehicle width, the wheelbase, the traveling speed, the weight (or mass), and the center of gravity are dealt with here.
一般に、重量(又は質量)は、完成車である車体3によって決まる。重心は、車体3、車幅及びホイールベースなどによって決まる。重量(又は質量)及び重心に関わるパラメータとして、さらに搭乗者及び積載物を扱ってもよい。 Generally, the weight (or mass) is determined by the vehicle body 3 which is a completed vehicle. The center of gravity is determined by the vehicle body 3, the vehicle width, the wheelbase, and the like. Passengers and loads may also be treated as parameters relating to weight (or mass) and center of gravity.
制御項目Ciの駆動制御Dc、制動制御Bc、操舵制御Sc、及び、緩衝制御Ccは、制御の結果が互いに影響し合う。これらの制御項目Ciにおける相互の影響を表現するために、それぞれの制御項目Ciの制御モデルCaを構成する物理量として、摩擦力Fb、荷重(又は衝撃力)Fp、運動エネルギーE、及び、慣性モーメントIを扱う。ここで、駆動制御Dcの制御モデルCaは制御モデルCaA、また、制動制御Bcの制御モデルCaは制御モデルCaBなど、制御項目Ciごとに制御モデルCaが決まる。また、荷重(又は衝撃力)Fp及び運動エネルギーEについては量又は変動率を扱ってもよい。 The control results of the drive control Dc, the braking control Bc, the steering control Sc, and the buffer control Cc of the control item Ci influence each other. In order to express the mutual influence of these control items Ci, the physical quantities constituting the control model Ca of each control item Ci are frictional force Fb, load (or impact force) Fp, kinetic energy E, and moment of inertia. Handle I. Here, the control model Ca of the drive control Dc is the control model CaA, the control model Ca of the braking control Bc is the control model CaB, and the control model Ca is determined for each control item Ci. Further, for the load (or impact force) Fp and the kinetic energy E, the quantity or volatility may be dealt with.
図22に示すように、駆動制御Dcの制御モデルCaは、少なくとも、車両2に働く摩擦力Fbと、運動エネルギーEとを含めて表現することができる。また、制動制御Bcの制御モデルCaは、少なくとも、車両2に働く摩擦力Fbと、運動エネルギーEとを含めて表現することができる。また、操舵制御Scの制御モデルCaは、少なくとも、車両2に働く摩擦力Fbと、慣性モーメントI(又は運動エネルギーE)とを含めて表現することができる。また、緩衝制御Ccの制御モデルCaは、少なくとも、車両2に働く荷重(又は衝撃力)Fpを含めて表現することができる。ここで、摩擦力Fb及び荷重(若しくは、衝撃力)Fpは、車両2の各タイヤ4に働くものと見做すことができる。 As shown in FIG. 22, the control model Ca of the drive control Dc can be expressed including at least the frictional force Fb acting on the vehicle 2 and the kinetic energy E. Further, the control model Ca of the braking control Bc can be expressed including at least the frictional force Fb acting on the vehicle 2 and the kinetic energy E. Further, the control model Ca of the steering control Sc can be expressed including at least the frictional force Fb acting on the vehicle 2 and the moment of inertia I (or kinetic energy E). Further, the control model Ca of the buffer control Cc can be expressed including at least the load (or impact force) Fp acting on the vehicle 2. Here, the frictional force Fb and the load (or impact force) Fp can be regarded as acting on each tire 4 of the vehicle 2.
駆動制御Dc及び制動制御Bcは、摩擦力Fb及び運動エネルギーEの2つの変動の影響を受ける。そして、操舵制御Scは、摩擦力Fb及び慣性モーメントI(又は運動エネルギーE)の2つの変動の影響を受ける。そして、緩衝制御Ccは、荷重(又は衝撃力)Fpの1つの変動の影響を受ける。 The drive control Dc and the braking control Bc are affected by two fluctuations of the frictional force Fb and the kinetic energy E. Then, the steering control Sc is affected by two fluctuations of the frictional force Fb and the moment of inertia I (or kinetic energy E). Then, the buffer control Cc is affected by one fluctuation of the load (or impact force) Fp.
以下に、車両2に働く物理量を用いて各々の制御モデルCaを表現した例を示す。 The following is an example of expressing each control model Ca using the physical quantity acting on the vehicle 2.
摩擦力Fbは、例えば、数式(1)で表すことができる。 The frictional force Fb can be expressed by, for example, the mathematical formula (1).
Figure JPOXMLDOC01-appb-M000001
…(1)
(μ:摩擦係数、N:垂直抗力)
Figure JPOXMLDOC01-appb-M000001
… (1)
(Μ: friction coefficient, N: normal force)
荷重又は衝撃力Fpは、例えば、数式(2)で表すことができる。 The load or impact force Fp can be expressed by, for example, the mathematical formula (2).
Figure JPOXMLDOC01-appb-M000002
…(2)
(m:質量、mg:重量、v:車両2の走行速度、l:衝撃により進む距離、h:落下した(又は進んだ)距離)
Figure JPOXMLDOC01-appb-M000002
… (2)
(M: mass, mg: weight, v: traveling speed of vehicle 2, l: distance traveled by impact, h: distance dropped (or advanced))
運動エネルギーEは、例えば、数式(3)で表すことができる。 The kinetic energy E can be expressed by, for example, the mathematical formula (3).
Figure JPOXMLDOC01-appb-M000003
…(3)
(m:質量、v:車両2の走行速度、I:慣性モーメント、ω:角速度)
Figure JPOXMLDOC01-appb-M000003
… (3)
(M: mass, v: traveling speed of vehicle 2, I: moment of inertia, ω: angular velocity)
慣性モーメントIは、例えば、数式(4)で表すことができる。 The moment of inertia I can be expressed by, for example, the mathematical formula (4).
Figure JPOXMLDOC01-appb-M000004
…(4)
(m:質量、r:回転半径、d:回転軸と重心との距離)
Figure JPOXMLDOC01-appb-M000004
… (4)
(M: mass, r: radius of gyration, d: distance between axis of rotation and center of gravity)
なお、制御モデルCaは、学習モデルに含まれるものであってもよいし、機械学習に限定されない一般的な制御回路又は制御プログラムに適用されるものであってもよい。 The control model Ca may be included in the learning model, or may be applied to a general control circuit or control program not limited to machine learning.
一般に、ごく短い時間の摩擦力Fbに含まれるNは、各タイヤ4に掛かる荷重又は衝撃力Fpにより変動する。また、各タイヤ4に掛かる荷重又は衝撃力Fpは、走行する車両2の運動エネルギーEにより変動する。また、走行する車両2の運動エネルギーEは、車両2の走行速度vと慣性モーメントIにより変動する。また、慣性モーメントIは、車両2の重心の位置dにより変動する。 Generally, N included in the frictional force Fb for a very short time varies depending on the load applied to each tire 4 or the impact force Fp. Further, the load or impact force Fp applied to each tire 4 varies depending on the kinetic energy E of the traveling vehicle 2. Further, the kinetic energy E of the traveling vehicle 2 fluctuates depending on the traveling speed v of the vehicle 2 and the moment of inertia I. Further, the moment of inertia I fluctuates depending on the position d of the center of gravity of the vehicle 2.
そして、駆動制御Dc及び制動制御Bcは、車両2の走行速度vを変化させる。緩衝制御Ccは、車両2の重心の位置dを変化させる。操舵制御Scは、車両2の横滑り、オーバーステア又はアンダーステアが生じないものとすると、車両2の走行速度及び重心を変化させない、又は、変化させても微小なものと扱うことができる。 Then, the drive control Dc and the braking control Bc change the traveling speed v of the vehicle 2. The buffer control Cc changes the position d of the center of gravity of the vehicle 2. Assuming that the vehicle 2 does not skid, oversteer or understeer, the steering control Sc can be treated as a small one that does not change the traveling speed and the center of gravity of the vehicle 2.
ここで、駆動制御Dc及び制動制御Bcが走行速度vを変化させると、それに伴い運動エネルギーEが変動する。このため、運動エネルギーEによって表現される荷重(又は衝撃力)Fpと、荷重(又は衝撃力)Fpによって表現される摩擦力Fbとのいずれも、駆動制御Dc及び制動制御Bcにより変動する。 Here, when the drive control Dc and the braking control Bc change the traveling speed v, the kinetic energy E changes accordingly. Therefore, both the load (or impact force) Fp expressed by the kinetic energy E and the frictional force Fb expressed by the load (or impact force) Fp vary depending on the drive control Dc and the braking control Bc.
また、緩衝制御Ccが車両2の重心の位置dを変化させると、それに伴い慣性モーメントIが変化する。このため、慣性モーメントIによって表現される運動エネルギーEと、運動エネルギーEによって表現される荷重(又は衝撃力)Fpと、荷重(又は衝撃力)Fpによって表現される摩擦力Fbとのいずれも、緩衝制御Ccにより変動する。 Further, when the buffer control Cc changes the position d of the center of gravity of the vehicle 2, the moment of inertia I changes accordingly. Therefore, both the kinetic energy E expressed by the moment of inertia I, the load (or impact force) Fp expressed by the kinetic energy E, and the frictional force Fb expressed by the load (or impact force) Fp are both. It varies depending on the buffer control Cc.
重心の位置dの変動を直接に受ける慣性モーメントIを制御モデルCaに含んだ操舵制御Scは、重心の位置を変動させる緩衝制御Ccの影響が強い。また、走行速度vの変動を直接に受ける運動エネルギーEを制御モデルCaに含んだ駆動制御Dc、制動制御Bc及び操舵制御Scは、走行速度vを変動させる駆動制御Dc及び制動制御Bcの影響が強い。また、駆動制御Dcと制動制御Bcとを比べると、駆動制御Dcは運動エネルギーEを増加させ、制動制御Bcは運動エネルギーEを減少させるため、駆動制御Dcの方が他の制御項目Ciに与える影響が強い。 The steering control Sc that includes the moment of inertia I that directly receives the fluctuation of the position d of the center of gravity in the control model Ca is strongly influenced by the buffer control Cc that fluctuates the position of the center of gravity. Further, the drive control Dc, the braking control Bc, and the steering control Sc that include the kinetic energy E that directly receives the fluctuation of the traveling speed v in the control model Ca are affected by the driving control Dc and the braking control Bc that fluctuate the traveling speed v. strong. Further, when the drive control Dc and the braking control Bc are compared, the drive control Dc increases the kinetic energy E and the braking control Bc decreases the kinetic energy E, so that the drive control Dc gives the other control item Ci. The influence is strong.
これらの関係をまとめると、操舵制御Scは、駆動制御Dc、制動制御Bc及び緩衝制御Ccの影響を強く受ける。制動制御Bcは、駆動制御Dc及び緩衝制御Ccの影響を強く受ける。駆動制御Dcは、制動制御Bc及び緩衝制御Ccの影響を強く受ける。緩衝制御Ccは、駆動制御Dc及び制動制御Bcの影響を受ける。なお、駆動制御Dcが与える影響は、制動制御Bcが与える影響よりも強い。また、駆動制御Dc、制動制御Bc及び操舵制御Scは慣性モーメントIの影響を受け、慣性モーメントIは操舵制御Scの影響を受ける。したがって、各々の制御項目Ciが影響する度合い、つまり、制御の優先度Pは、緩衝制御Cc>駆動制御Dc>制動制御Bc>操舵制御Scの順となる。 Summarizing these relationships, the steering control Sc is strongly influenced by the drive control Dc, the braking control Bc, and the buffer control Cc. The braking control Bc is strongly influenced by the drive control Dc and the buffer control Cc. The drive control Dc is strongly influenced by the braking control Bc and the buffer control Cc. The buffer control Cc is affected by the drive control Dc and the braking control Bc. The influence of the drive control Dc is stronger than the influence of the braking control Bc. Further, the drive control Dc, the braking control Bc, and the steering control Sc are affected by the moment of inertia I, and the moment of inertia I is affected by the steering control Sc. Therefore, the degree to which each control item Ci affects, that is, the priority P of control, is in the order of buffer control Cc> drive control Dc> braking control Bc> steering control Sc.
このように、車両2に関わる物理量を用いて各々の制御項目Ciを表現した制御モデルCaに基づいて、各々の制御項目Ciに対して統合学習を行わせるときの優先度Pを導出することができる。 In this way, it is possible to derive the priority P when the integrated learning is performed for each control item Ci based on the control model Ca expressing each control item Ci using the physical quantity related to the vehicle 2. can.
なお、車両2の制御モデルCaは、上述の数式(1)から(4)の式に限定されることはなく、ラグランジアン又はハミルトニアンの関数を用い、運動量、ポテンシャル、並びに、摩擦力及び空気抵抗などの外力表現を含んだ種々の物理量を、高速演算が可能なコンピュータ(つまり、各々の制御モデルCaの解析に特化した演算装置又は量子コンピュータなど)に演算させて導出してもよい。このとき、例えば、全て又は一部の解析演算を“運動エネルギー>>ポテンシャル+外力エネルギー”の系と見做せる場合に、運動量の解析として扱うことで演算量を抑制してもよい。また、これらの解析演算の過程において、各々の制御モデルCaの事前の解析結果を、例えば、統括部30、センサ処理部31、車載装置Vd又は車載ネットワーク40上のサーバなどで複数保持しておき、各々の車載装置Vdの挙動との類似性に基づいて保持された解析結果を利用することにより、解析演算の効率化又は高速化を図ってもよい。 The control model Ca of the vehicle 2 is not limited to the above equations (1) to (4), and uses the functions of Lagrangian or Hamiltonian, such as momentum, potential, frictional force, and air resistance. Various physical quantities including the external force expression of the above may be derived by being calculated by a computer capable of high-speed calculation (that is, a calculation device or a quantum computer specialized in the analysis of each control model Ca). At this time, for example, when all or part of the analysis calculation can be regarded as a system of "kinetic energy >> potential + external force energy", the calculation amount may be suppressed by treating it as an analysis of the momentum. Further, in the process of these analysis calculations, a plurality of prior analysis results of each control model Ca are held in, for example, a control unit 30, a sensor processing unit 31, an in-vehicle device Vd, or a server on the in-vehicle network 40. By using the analysis result held based on the similarity with the behavior of each in-vehicle device Vd, the efficiency or speed of the analysis calculation may be improved.
上述した4つの制御項目Ciの優先度Pによれば、先ず、学習部30aは、緩衝装置14を制御の対象とするセンサ処理部31DのNNデータNdを追加学習対応モードに設定し、統合後の車両2に適する緩衝制御Ccを統合学習させる。次に、学習部30aは、駆動装置11を制御の対象とするセンサ処理部31AのNNデータNdを追加学習対応モードに設定し、統合後の車両2に適する駆動制御Dcを統合学習させる。次に、学習部30aは、制動装置12を制御の対象とするセンサ処理部31BのNNデータNdを追加学習対応モードに設定し、統合後の車両2に適する制動制御Bcを統合学習させる。次に、学習部30aは、操舵装置13を制御の対象とするセンサ処理部31CのNNデータNdを追加学習対応モードに設定し、統合後の車両2に適する操舵制御Scを統合学習させる。
このように、制御の優先度Pを考慮することにより、各々の制御項目Ciに関わるNNデータNdの統合学習を時系列で切り替えて行わせることが可能となる。
According to the priority P of the four control items Ci described above, first, the learning unit 30a sets the NN data Nd of the sensor processing unit 31D whose control target is the shock absorber 14 to the additional learning compatible mode, and after integration. The buffer control Cc suitable for the vehicle 2 of the above is integratedly learned. Next, the learning unit 30a sets the NN data Nd of the sensor processing unit 31A whose control target is the drive device 11 to the additional learning compatible mode, and causes integrated learning of the drive control Dc suitable for the vehicle 2 after integration. Next, the learning unit 30a sets the NN data Nd of the sensor processing unit 31B whose control target is the braking device 12 to the additional learning compatible mode, and causes integrated learning of the braking control Bc suitable for the vehicle 2 after integration. Next, the learning unit 30a sets the NN data Nd of the sensor processing unit 31C whose control target is the steering device 13 to the additional learning corresponding mode, and causes integrated learning of the steering control Sc suitable for the vehicle 2 after integration.
In this way, by considering the control priority P, it is possible to switch the integrated learning of the NN data Nd related to each control item Ci in chronological order.
図23は、AI統合システム1に統合された複数のAIを統合学習させる過程を説明するための模式図である。図23に示す2軸は、縦軸にAIと対応する制御項目Ciを、横軸に時系列の量として時間軸Coをとっている。また、縦軸の制御項目Ciは、システム1が動作する環境(例えば、走行シーンDs)における、制御の優先度Pの順で並べられている。なお、横軸には、時系列で扱える量であれば、例えば、演算又は計数のカウント数などでもよい。図23の例では、緩衝制御Cc、駆動制御Dc、制動制御Bc、操舵制御Sc、伝達制御Tc、認識制御Rc、UI制御Ui、及び、バッテリー制御Ecの順に並べられている。 FIG. 23 is a schematic diagram for explaining a process of integrated learning of a plurality of AIs integrated in the AI integrated system 1. The two axes shown in FIG. 23 have a control item Ci corresponding to AI on the vertical axis and a time axis Co as a time-series quantity on the horizontal axis. Further, the control items Ci on the vertical axis are arranged in the order of control priority P in the environment in which the system 1 operates (for example, the traveling scene Ds). The horizontal axis may be, for example, the number of counts for calculation or counting as long as it can be handled in time series. In the example of FIG. 23, the buffer control Cc, the drive control Dc, the braking control Bc, the steering control Sc, the transmission control Tc, the recognition control Rc, the UI control Ui, and the battery control Ec are arranged in this order.
学習部30aは、統合学習の開始である時間T1において、最も優先度Pが高い緩衝制御Ccと対応するセンサ処理部31DのNNデータNdDに対して統合学習Ld1を行わせる。評価部30eは、統合学習Ld1における収束度合い又は安定評価Seなどの情報をもとに、時間T1から開始された統合学習Ld1の進捗を評価する。学習部30aは、評価部30eでの評価をもとに、他のAIの統合学習に切り替えるかどうかを判断する。 The learning unit 30a causes the integrated learning Ld1 to be performed on the NN data NdD of the sensor processing unit 31D corresponding to the buffer control Cc having the highest priority P at the time T1 at which the integrated learning is started. The evaluation unit 30e evaluates the progress of the integrated learning Ld1 started from the time T1 based on the information such as the degree of convergence in the integrated learning Ld1 or the stability evaluation Se. The learning unit 30a determines whether to switch to integrated learning of another AI based on the evaluation by the evaluation unit 30e.
ところで、複数のAIに対して制御の優先度Pを考慮しつつ並行して学習を進めさせる方針のもと、システム1に統合される装置類及び制御するAIの特性に応じて、統合学習Ld1が収束していなくとも、安定評価Seなどから得られる学習の進捗によって他のAIの統合学習に切り替える判断を行うよう、学習部30aを処理させることも可能である。また、制御の優先度Pが高い順に、順次、AIの統合学習を終了させて次のAIの統合学習に切り替えるよう、学習部30aを処理させることも可能である。 By the way, under the policy of allowing a plurality of AIs to proceed with learning in parallel while considering the control priority P, the integrated learning Ld1 is made according to the characteristics of the devices integrated into the system 1 and the AI to be controlled. It is also possible to have the learning unit 30a process so as to make a determination to switch to the integrated learning of another AI according to the progress of learning obtained from the stability evaluation Se or the like even if the learning is not converged. Further, it is also possible to process the learning unit 30a so as to sequentially end the integrated learning of the AI and switch to the integrated learning of the next AI in descending order of the priority P of the control.
学習部30aは、時間T2において、2番目に優先度Pが高い駆動制御Dcと対応するセンサ処理部31AのNNデータNdAに対して統合学習La1を行わせる。評価部30eは、統合学習La1における収束度合い又は安定評価Seなどの情報をもとに、時間T2から開始された統合学習La1の進捗を評価する。学習部30aは、評価部30eでの評価をもとに、他のAIの統合学習に切り替えるかどうかを判断する。 The learning unit 30a causes the NN data NdA of the sensor processing unit 31A corresponding to the drive control Dc having the second highest priority P to perform the integrated learning La1 at the time T2. The evaluation unit 30e evaluates the progress of the integrated learning La1 started from the time T2 based on the information such as the degree of convergence in the integrated learning La1 or the stability evaluation Se. The learning unit 30a determines whether to switch to integrated learning of another AI based on the evaluation by the evaluation unit 30e.
ここで、学習部30aは、統合学習Ld1と同様にして、統合学習La1が収束していなくとも、安定評価Seなどから得られる学習の進捗によって他のAIの統合学習に切り替える判断を行うものとする。 Here, in the same manner as the integrated learning Ld1, the learning unit 30a determines to switch to the integrated learning of another AI according to the progress of learning obtained from the stability evaluation Se or the like even if the integrated learning La1 does not converge. do.
学習部30aは、時間T3において、改めてセンサ処理部31DのNNデータNdDに対して統合学習Ld2を行わせる。 The learning unit 30a causes the NN data NdD of the sensor processing unit 31D to perform the integrated learning Ld2 again at the time T3.
このように、学習部30aは、時間T2における統合学習Ld1から統合学習La1への切り替え判断とともに、統合学習La1での進捗によって再び緩衝制御Ccの統合学習Ld2に切り替える判断を行っておくことも可能である。つまり、学習部30aは、上述の方針のもと、優先度Pが高い統合学習xの終了前に、優先度Pが低い他のAIの統合学習yを途中まで進めさせ、再び優先度Pが高い統合学習xを進めさせることで、複数のAIに対して制御の優先度Pを考慮しつつ並行して学習済みモデルを変更させることができる。 In this way, the learning unit 30a can make a determination to switch from the integrated learning Ld1 to the integrated learning La1 at the time T2 and also to make a determination to switch to the integrated learning Ld2 of the buffer control Cc again according to the progress in the integrated learning La1. Is. That is, based on the above-mentioned policy, the learning unit 30a causes the integrated learning y of another AI having a low priority P to proceed halfway before the end of the integrated learning x having a high priority P, and the priority P is again set. By advancing the high integrated learning x, it is possible to change the trained model in parallel for a plurality of AIs while considering the control priority P.
学習部30aは、時間T4において、3番目に優先度Pが高い制動制御Bcと対応するセンサ処理部31BのNNデータNdBに対して統合学習Lb1を行わせる。評価部30eは、統合学習Lb1における収束度合い又は安定評価Seなどの情報をもとに、時間T4から開始された統合学習Lb1の進捗を評価する。学習部30aは、評価部30eでの評価をもとに、他のAIの統合学習に切り替えるかどうかを判断する。 The learning unit 30a causes the integrated learning Lb1 to be performed on the NN data NdB of the sensor processing unit 31B corresponding to the braking control Bc having the third highest priority P at the time T4. The evaluation unit 30e evaluates the progress of the integrated learning Lb1 started from the time T4 based on the information such as the degree of convergence in the integrated learning Lb1 or the stability evaluation Se. The learning unit 30a determines whether to switch to integrated learning of another AI based on the evaluation by the evaluation unit 30e.
学習部30aは、時間T5において、改めて駆動制御Dcと対応するセンサ処理部31AのNNデータNdAに対して統合学習La2を行わせる。 The learning unit 30a causes the NN data NdA of the sensor processing unit 31A corresponding to the drive control Dc to perform the integrated learning La2 again at the time T5.
以降、同様にして、学習部30aは、時間T6から時間T11までの期間に、緩衝制御Ccと対応するセンサ処理部31DのNNデータNdD、駆動制御Dcと対応するセンサ処理部31AのNNデータNdA、制動制御Bcと対応するセンサ処理部31BのNNデータNdB、及び、操舵制御Scと対応するセンサ処理部31CのNNデータNdCに対して統合学習を行わせる。その結果、統合学習Ld1,統合学習La1,統合学習Ld2,統合学習Lb1,統合学習La2,統合学習Lc1,統合学習Ld3,統合学習La3,統合学習Lb2及び統合学習Lc2に示すように、制御の優先度Pを反映しつつ、NNデータNdの学習を順次終了させることができる。 Hereinafter, similarly, in the period from the time T6 to the time T11, the learning unit 30a has the NN data NdD of the sensor processing unit 31D corresponding to the buffer control Cc and the NN data NdA of the sensor processing unit 31A corresponding to the drive control Dc. , The NN data NdB of the sensor processing unit 31B corresponding to the braking control Bc and the NN data NdC of the sensor processing unit 31C corresponding to the steering control Sc are subjected to integrated learning. As a result, as shown in integrated learning Ld1, integrated learning La1, integrated learning Ld2, integrated learning Lb1, integrated learning La2, integrated learning Lc1, integrated learning Ld3, integrated learning La3, integrated learning Lb2 and integrated learning Lc2, priority is given to control. The learning of the NN data Nd can be sequentially completed while reflecting the degree P.
さらに、学習部30aは、時間T11から時間T17において、制御の優先度Pが低い伝達制御Tc、認識制御Rc、UI制御Ui及びバッテリー制御Ecの各々と対応するセンサ処理部31のNNデータNdに対しても統合学習を行わせることが可能である。 Further, the learning unit 30a is connected to the NN data Nd of the sensor processing unit 31 corresponding to each of the transmission control Tc, the recognition control Rc, the UI control Ui, and the battery control Ec having a low control priority P from the time T11 to the time T17. On the other hand, it is possible to perform integrated learning.
なお、学習部30aは、時間T12のように、伝達制御Tcと対応するセンサ処理部31EのNNデータNdEに対する統合学習Lt1の最中に、並行して認識制御Rcと対応するセンサ処理部31FのNNデータNdFに対する統合学習Lf1を開始させたり、時間T14のように、統合学習Lf1の最中に、並行してUI制御Uiと対応するセンサ処理部31GのNNデータNdGに対する統合学習Lg1、及び、バッテリー制御Ecと対応するセンサ処理部31HのNNデータNdHに対する統合学習Lh1を開始させたりしてもよい。 It should be noted that the learning unit 30a of the sensor processing unit 31F corresponding to the recognition control Rc in parallel during the integrated learning Lt1 for the NN data NdE of the sensor processing unit 31E corresponding to the transmission control Tc, as in the time T12. The integrated learning Lg1 for the NN data NdG of the sensor processing unit 31G corresponding to the UI control Ui in parallel during the integrated learning Lf1 such as starting the integrated learning Lf1 for the NN data NdF, and the integrated learning Lg1 for the NN data NdG, and The integrated learning Lh1 for the NN data NdH of the sensor processing unit 31H corresponding to the battery control Ec may be started.
図23に示すように、制御の優先度Pを考慮することによって、学習部30aは複数のAIに対して分割して統合学習を行わせることが可能となる。これにより、複数のAIは、影響力の強さにしたがった一方的な追加学習(つまり、統合学習)だけでなく、相互に及ぼす影響の結果を反映した双方向的な追加学習(つまり、統合学習)により学習モデルを変更させることが可能となる。 As shown in FIG. 23, by considering the control priority P, the learning unit 30a can be divided into a plurality of AIs to perform integrated learning. This allows multiple AIs to have not only one-sided additional learning (ie, integrated learning) that follows the strength of their influence, but also two-way additional learning (that is, integrated learning) that reflects the consequences of their mutual influence. Learning) makes it possible to change the learning model.
図24は、AI統合システム1における統括部30での処理を説明するためのフローチャート図である。図24は、統括部30が、各々のセンサ処理部31のNNデータNdに統合学習を行わせる処理を示す。 FIG. 24 is a flowchart for explaining the processing in the control unit 30 in the AI integrated system 1. FIG. 24 shows a process in which the control unit 30 causes the NN data Nd of each sensor processing unit 31 to perform integrated learning.
処理Sp161では、選択部30sが、図10(b)の処理Sp81bで入力した検知情報Siをもとに車両2が走行する走行シーンDsを推定する。
処理Sp162では、学習部30aが、処理Sp161で推定した走行シーンDsをもとに、各々のセンサ処理部31のNNデータNdでの制御の優先度Pを導出する。
処理Sp163では、学習部30aが、処理Sp162で導出した制御の優先度Pをもとに、各々のセンサ処理部31のNNデータNdに対して統合学習を行わせる。
処理Sp164では、評価部30eが、処理Sp163でのNNデータNdに対する統合学習により入力したデータセットをもとに、NNデータNdでの制御を評価する。
処理Sp165では、評価部30eが、処理Sp164での評価をもとに、各NNデータNdでの統合学習が収束したかどうかを判定する。統合学習が収束した場合は処理Sp161に進み、選択部30sが新たな走行シーンDsであること、又は走行シーンが変わったことを推定したとき、引き続き、制御の優先度Pを導出し、各々のNNデータNdを追加学習対応モードに設定して統合学習を行わせる。統合学習が収束しない場合は処理Sp166に進み、図10(b)の処理を行う。
In the processing Sp161, the selection unit 30s estimates the traveling scene Ds in which the vehicle 2 travels based on the detection information Si input in the processing Sp81b of FIG. 10B.
In the processing Sp162, the learning unit 30a derives the priority P of control in the NN data Nd of each sensor processing unit 31 based on the traveling scene Ds estimated by the processing Sp161.
In the processing Sp163, the learning unit 30a causes the NN data Nd of each sensor processing unit 31 to perform integrated learning based on the control priority P derived in the processing Sp162.
In the process Sp164, the evaluation unit 30e evaluates the control of the NN data Nd based on the data set input by the integrated learning for the NN data Nd in the process Sp163.
In the process Sp165, the evaluation unit 30e determines whether or not the integrated learning in each NN data Nd has converged based on the evaluation in the process Sp164. When the integrated learning converges, the process proceeds to Sp161, and when it is estimated that the selection unit 30s is a new driving scene Ds or the driving scene has changed, the control priority P is continuously derived and each of them is derived. Set the NN data Nd to the additional learning support mode to perform integrated learning. If the integrated learning does not converge, the process proceeds to process Sp166, and the process of FIG. 10B is performed.
なお、処理Sp167のように、学習部30aは、処理Sp164での評価をもとにして、処理Sp163に進み、図23に示したように各々のNNデータNdでの統合学習の実行を切り替えさせてもよいし、処理Sp162に進み、走行シーンDsの特徴に応じて各々のNNデータNdでの制御の優先度Pを改めて導出し直してもよい。 As in the process Sp167, the learning unit 30a proceeds to the process Sp163 based on the evaluation in the process Sp164, and switches the execution of the integrated learning in each NN data Nd as shown in FIG. 23. Alternatively, the process may proceed to Sp162, and the priority P of control in each NN data Nd may be derived again according to the characteristics of the traveling scene Ds.
なお、図24の一連の処理は、図10(a)及び(b)の処理と独立して行われてもよい。また、図24の個々の処理は、次の処理の進捗によらず実行を開始させてもよい。例えば、各々のNNデータNdでの制御の優先度Pを導出しつつ、既に導出した優先度Pに基づいて個々のNNデータNdに統合学習を行わせてもよい。 The series of processes shown in FIG. 24 may be performed independently of the processes shown in FIGS. 10 (a) and 10 (b). Further, the individual processes of FIG. 24 may be started to be executed regardless of the progress of the next process. For example, while deriving the control priority P for each NN data Nd, the individual NN data Nd may be subjected to integrated learning based on the already derived priority P.
さらに、学習部30aは、各々のNNデータNdの統合学習が収束した後は、各々のNNデータNdを、常時、追加学習非対応モードに設定してもよいし、追加学習対応モードに設定してもよい。 Further, the learning unit 30a may always set each NN data Nd to the additional learning non-compatible mode or set the additional learning compatible mode after the integrated learning of each NN data Nd has converged. You may.
また、複数の車両2において、車種、装備及び仕様などのプロパティが同等と見做せる場合、ある車両2で統合学習を行わせたNNデータNdを他の車両2に対して通信を介して取得させ用いさせてもよい。 In addition, when properties such as vehicle type, equipment, and specifications can be regarded as equivalent in a plurality of vehicles 2, NN data Nd in which integrated learning is performed in one vehicle 2 is acquired by communication with another vehicle 2. It may be used.
また、プロパティが同等な複数の車両2間において、統合学習を行わせた各々のNNデータNdを比較及び分析して情報の差分を抽出し、抽出した差分情報をフィードバックすることにより、各々のNNデータNdでの制御における不具合の改善に利用したり、制御の優先度Pを導出するときの制御モデルCaに反映したり、各々のNNデータNdでの制御がより安定して安全性及び信頼性の高いものとなるように適用させてもよい。このようにすることで、車両2が走行する多様な環境において、車両2が個別に複数のAIをAI統合システム1に統合させるための統合学習を地道に行うことなく、複数の車両2間で未対応の環境に対して相互に補完し合いながら、より高精度でより迅速に、複数のAIを車両2のシステム1に適応させることが可能となる。 Further, among a plurality of vehicles 2 having the same property, each NN data Nd subjected to integrated learning is compared and analyzed to extract a difference in information, and the extracted difference information is fed back to each NN. It can be used to improve problems in control with data Nd, or reflected in the control model Ca when deriving the priority P of control, and the control with each NN data Nd is more stable, safe and reliable. It may be applied so as to have a high value. By doing so, in various environments in which the vehicle 2 travels, the vehicle 2 can individually integrate a plurality of AIs into the AI integrated system 1 without steadily performing integrated learning among the plurality of vehicles 2. It is possible to adapt a plurality of AIs to the system 1 of the vehicle 2 with higher accuracy and more quickly while complementing each other for an unsupported environment.
上述した複数の車両2を扱う場合、車載ネットワーク40に接続するサーバが、複数の車両2に搭載の統括部30と通信して、各々の統括部30及びセンサ処理部31が保持する種々の情報及びNNデータNdの更新を行ってもよい。 When handling the plurality of vehicles 2 described above, the server connected to the in-vehicle network 40 communicates with the control unit 30 mounted on the plurality of vehicles 2 and various information held by each control unit 30 and the sensor processing unit 31. And NN data Nd may be updated.
以上説明したように、実施の形態2によれば、実施の形態1のAI統合システム1において、システム1に統合された後の複数のAIがシステム1の特性に合わせて適切に制御を行えるようにするために、統括部30が複数のAIに対して制御の優先度Pを考慮した統合学習を行わせる。統括部30(学習部30a)は、システム1に統合された各々のセンサ処理部31が保持するNNデータNdに統合学習を行わせる。統括部30(学習部30a)は、統括部30(選択部30s)で推定した走行シーンDsと対応するNNデータNdに対して統合学習を行わせる。また、統括部30(学習部30a)は、走行シーンDsと対応して各々のセンサ処理部31に設定されたNNデータNdのうちから、制御の優先度Pをもとに秩序立てて(つまり、順序を決めて)統合学習を行わせる。ここでの制御の優先度Pは、例えば、各々の制御項目Ciが相互に及ぼす影響度合いの強さ関係、各々の制御項目Ciを制御モデルCaで表現したときに含まれる物理量(つまり、システム1に固有のパラメータ)を介した制御項目Ci間の因果関係、及び、各々のセンサ処理部31のNNデータNdが行う制御により制御対象の装置が行う動作の安定評価Seなどをもとに、学習部30aが導出してもよいし、人によって若しくはネットワークを介して設定されてもよい。その結果、AI統合システム1に、種々の用途の動作を行う各々の装置を制御する複数のAIを統合したとき、システム1が動作する環境において、複数のAIに設定されたNNデータNdを秩序立てて統合学習させることが可能となる。 As described above, according to the second embodiment, in the AI integrated system 1 of the first embodiment, a plurality of AIs after being integrated into the system 1 can be appropriately controlled according to the characteristics of the system 1. In order to achieve this, the control unit 30 causes a plurality of AIs to perform integrated learning in consideration of the control priority P. The control unit 30 (learning unit 30a) causes the NN data Nd held by each sensor processing unit 31 integrated in the system 1 to perform integrated learning. The supervision unit 30 (learning unit 30a) causes integrated learning to be performed on the NN data Nd corresponding to the running scene Ds estimated by the supervision unit 30 (selection unit 30s). Further, the control unit 30 (learning unit 30a) is ordered from among the NN data Nd set in each sensor processing unit 31 corresponding to the driving scene Ds, based on the control priority P (that is,). Let them perform integrated learning (in order). The control priority P here is, for example, the strength relationship of the degree of influence of each control item Ci on each other, and the physical quantity included when each control item Ci is expressed by the control model Ca (that is, system 1). Learning based on the causal relationship between the control items Ci via the parameter) and the stability evaluation Se of the operation performed by the device to be controlled by the control performed by the NN data Nd of each sensor processing unit 31. The unit 30a may be derived, or may be set by a person or via a network. As a result, when a plurality of AIs that control each device that performs operations for various purposes are integrated into the AI integrated system 1, the NN data Nd set in the plurality of AIs is ordered in the environment in which the system 1 operates. It is possible to stand up and make integrated learning.
このように、NNデータNdの統合学習を制御の優先度Pを考慮して行わせると、システム1が動作する環境において安定した動作状態を維持させるのにベースとなる制御項目Ci(言い換えると、クリティカルな制御項目Ci)が先行して適切に機能するようになるため、その他の制御項目Ciがより適切に機能できるようになり、結果として、統合後の複数のAIをシステム1に適応させやすくする効果が得られる。 In this way, when the integrated learning of the NN data Nd is performed in consideration of the control priority P, the control item Ci (in other words, in other words, which is the basis for maintaining a stable operating state in the environment in which the system 1 operates) is used. Since the critical control item Ci) will function properly in advance, the other control item Ci will be able to function more appropriately, and as a result, it will be easier to adapt multiple AIs after integration to the system 1. The effect of
システム1への統合後のAIは、統合学習によって統合前の学習により獲得した学習済みモデルをシステム1に適応して変化させることができる。そして、システム1に適応して制御を変化させたAIは、システム1への統合前と比べて、制御を行ううえでのロバスト性はより高いと言える。 The AI after integration into the system 1 can adapt and change the trained model acquired by the learning before the integration by the integrated learning to the system 1. It can be said that the AI whose control is changed according to the system 1 has higher robustness in controlling than before the integration into the system 1.
また、システム1に新たにセンサ32及びセンサ処理部31を追加したり、センサ処理部31に接続するセンサ32を変更したりして統合させる場合においても、統括部30が改めて統合学習を行わせるNNデータNdの優先度Pを導出して学習させることにより、システム1は安定した動作状態を継続することが可能となる。このとき、学習部30aが追加又は変更があったセンサ32を扱うセンサ処理部31のNNデータNdに対して優先的に統合学習を行わせるよう処理することで、システム1を再び安定して動作させることが期待できる。このことは、システム1に適応して制御を変化させたAIが制御を行ううえでのロバスト性を向上させることに繋がる。 Further, even when a sensor 32 and a sensor processing unit 31 are newly added to the system 1 or the sensor 32 connected to the sensor processing unit 31 is changed and integrated, the control unit 30 causes the integrated learning to be performed again. By deriving and learning the priority P of the NN data Nd, the system 1 can continue a stable operating state. At this time, the learning unit 30a processes the NN data Nd of the sensor processing unit 31 that handles the sensor 32 that has been added or changed so as to preferentially perform integrated learning, so that the system 1 operates stably again. It can be expected to make it. This leads to the improvement of the robustness in the control by the AI whose control is changed according to the system 1.
また、システム1に統合済みのセンサ処理部31のNNデータNdを変更した場合においても、統括部30が変更されたNNデータNdを統合学習させることによって、システム1を安定して動作させることが可能となる。このとき、変更されたNNデータNdだけでなく、制御の優先度Pに基づいて、他のNNデータNdを統合学習させてもよい。制御の優先度Pは統括部30が改めて導出してもよいし、既に導出されたものを用いてもよい。 Further, even when the NN data Nd of the sensor processing unit 31 integrated into the system 1 is changed, the system 1 can be stably operated by the integrated learning of the changed NN data Nd by the control unit 30. It will be possible. At this time, not only the changed NN data Nd but also other NN data Nd may be integratedly learned based on the control priority P. The control priority P may be derived again by the control unit 30, or may already be derived.
また、システム1に搭載された車載装置Vd又は統合されたセンサ処理部31のいずれかが異常状態となった場合においても、統括部30が、異常状態となった車載装置Vd又はセンサ処理部31に関わるNNデータNdを除く、他のNNデータNdを統合学習させることによって、システム1を継続して動作させる可能性が得られる。これにより、システム1における装置動作に対する制御のロバスト性が向上する。 Further, even when either the in-vehicle device Vd mounted on the system 1 or the integrated sensor processing unit 31 becomes abnormal, the control unit 30 causes the in-vehicle device Vd or the sensor processing unit 31 in the abnormal state. By integrating and learning other NN data Nd excluding the NN data Nd related to the above, the possibility of continuously operating the system 1 can be obtained. This improves the robustness of control for the operation of the device in the system 1.
また、AIを活用した装置又は部品の共通化を図る場合、統合する各々のAIを代表的なシステム1において統合学習させておくことで、これらのAIを他のシステム1に統合させた後に行う統合学習を効率化させることが期待できる。これにより、多品種のシステム1に対して、装置及び部品とともに制御を共用化することが可能となる。 In addition, when trying to standardize devices or parts that utilize AI, by having each AI to be integrated be integrated and learned in a representative system 1, these AIs are integrated into another system 1 and then performed. It can be expected to improve the efficiency of integrated learning. This makes it possible to share control with devices and parts for a wide variety of systems 1.
また、上述の説明では、学習部30aは、各々のNNデータNdが行う制御に関して優先度Pを導出して統合学習に利用したが、車載装置Vd自体及び車載装置Vdの動作の少なくともいずれかに関して優先度Pを導出して統合学習に利用してもよい。 Further, in the above description, the learning unit 30a derives the priority P for the control performed by each NN data Nd and uses it for integrated learning, but regarding at least one of the operation of the in-vehicle device Vd itself and the in-vehicle device Vd. The priority P may be derived and used for integrated learning.
また、各々のセンサ処理部31の学習器Lmが単一のNNデータNdを扱う(つまり、センサ処理部31がAIを切り替えない)構成であっても、本実施の形態2を適用することは可能である。この場合、統括部30は、各々のセンサ処理部31で用いられる単一のNNデータNd(つまりAI)に統合学習を行わせるときの優先度Pを上述の要領で導出し、導出した優先度Pに基づいて各々のNNデータNdの学習を実行させる。これにより、上述と同様にして、システム1に統合される前の学習によって制御対象となる車載装置Vdを制御するための学習済みモデルであるAI(つまり、NNデータNd)を、システム1への統合後に秩序立てて統合学習させることが可能となり、その結果、各々のセンサ処理部31のAIに対してより適切に制御を行わせる効果が得られる。このことは、AIが学習によって獲得する制御(言い換えると、自律制御)のロバスト性を向上させることに繋がる。 Further, even if the learning device Lm of each sensor processing unit 31 handles a single NN data Nd (that is, the sensor processing unit 31 does not switch AI), the second embodiment can be applied. It is possible. In this case, the control unit 30 derives the priority P for causing the single NN data Nd (that is, AI) used in each sensor processing unit 31 to perform integrated learning as described above, and derives the priority. The learning of each NN data Nd is executed based on P. As a result, in the same manner as described above, AI (that is, NN data Nd), which is a learned model for controlling the in-vehicle device Vd to be controlled by learning before being integrated into the system 1, is transferred to the system 1. After the integration, the integrated learning can be performed in an orderly manner, and as a result, the effect of more appropriately controlling the AI of each sensor processing unit 31 can be obtained. This leads to improving the robustness of the control (in other words, autonomous control) acquired by AI through learning.
実施の形態3.
実施の形態2では、各々の制御項目Ciを表現する制御モデルCaを用いて制御の優先度Pを導出した。実施の形態3では、各々の制御項目Ciにおける学習過程に基づき、制御の優先度Pを導出する。
Embodiment 3.
In the second embodiment, the control priority P is derived using the control model Ca expressing each control item Ci. In the third embodiment, the control priority P is derived based on the learning process in each control item Ci.
図25は、本開示の実施の形態3における、NNデータNdへの統合学習を通じて制御の優先度Pを決定する過程を説明するための模式図である。 FIG. 25 is a schematic diagram for explaining a process of determining a control priority P through integrated learning to NN data Nd in the third embodiment of the present disclosure.
学習部30aは、各々のセンサ処理部31が保持するNNデータNdの車両2への統合前の学習に関する情報を取得できるものとする。 It is assumed that the learning unit 30a can acquire information regarding learning before integration of the NN data Nd held by each sensor processing unit 31 into the vehicle 2.
NNデータNdの車両2への統合前の学習に関する情報とは、種々の走行シーンDsにおけるNNデータNdの学習過程に関する情報であって、例えば、各々のNNデータNdにおける走行シーンDsごとの制御の貢献度合い、学習の収束度合い、又は、学習に用いるシードを変えたときの学習過程のばらつき度合いなどの種々の評価項目が挙げられる。 The information regarding the learning before the integration of the NN data Nd into the vehicle 2 is the information regarding the learning process of the NN data Nd in various driving scenes Ds, for example, the control of each driving scene Ds in each NN data Nd. Various evaluation items such as the degree of contribution, the degree of convergence of learning, or the degree of variation in the learning process when the seed used for learning is changed can be mentioned.
走行シーンDsごとの制御の貢献度合いとは、車両2の走行に対する影響力であって、例えば、NNデータNdの制御対象となる車載装置Vdが適切に動作したとき(例えば、車載装置Vdの制御とその結果が、走行シーンDsに対応した教師データTdに近似するとき)、又は、適切に動作しなかったとき(例えば、車載装置Vdの制御とその結果が、走行シーンDsに対応した教師データTdに近似しないとき)、並びに、制御の後の走行の安定評価などから導出できる。学習の収束度合いは、例えば、学習が収束したと判定されるまでの制御信号Csの送信回数、又は、学習に用いる教師データ数(若しくはデータセット数)などから導出できる。学習に用いるシードを変えたときの学習過程のばらつき度合いは、例えば、シード数と学習過程のばらつきとの関係などから導出できる。 The degree of contribution of control for each driving scene Ds is an influence on the driving of the vehicle 2, for example, when the vehicle-mounted device Vd to be controlled by the NN data Nd operates appropriately (for example, control of the vehicle-mounted device Vd). And when the result is close to the teacher data Td corresponding to the driving scene Ds), or when it does not operate properly (for example, the control of the in-vehicle device Vd and the result are the teacher data corresponding to the driving scene Ds). It can be derived from (when it is not close to Td) and the stability evaluation of running after control. The degree of convergence of learning can be derived from, for example, the number of transmissions of control signals Cs until it is determined that learning has converged, the number of teacher data (or the number of data sets) used for learning, and the like. The degree of variation in the learning process when the seeds used for learning are changed can be derived from, for example, the relationship between the number of seeds and the variation in the learning process.
以下に、学習部30aが制御の優先度Pを決定する過程について説明する。ここでは、評価部30eが、車両2への統合前の各々のセンサ処理部31のNNデータNdに対し、走行シーンDsごとに上述の評価項目で評価を行う。上述の評価項目とは、制御の貢献度合い、学習の収束度合い及び学習過程のばらつき度合いである。 The process in which the learning unit 30a determines the control priority P will be described below. Here, the evaluation unit 30e evaluates the NN data Nd of each sensor processing unit 31 before integration into the vehicle 2 with the above-mentioned evaluation items for each driving scene Ds. The above-mentioned evaluation items are the degree of contribution of control, the degree of convergence of learning, and the degree of variation in the learning process.
評価部30eは、この評価結果をもとに走行シーンDsごとの評価値Valを導出し、評価値Valが高いものを抽出する。評価値Valを抽出する過程において、評価項目ごとに重み付けを変えてもよい。 The evaluation unit 30e derives the evaluation value Val for each driving scene Ds based on the evaluation result, and extracts the one having a high evaluation value Val. In the process of extracting the evaluation value Val, the weighting may be changed for each evaluation item.
車両2が走行する環境として、凍結又は未舗装などの路面状態が悪い走行シーンDsを扱い、上述の評価項目をもとに評価値Valを導出した結果、各々の制御項目CiにおけるNNデータNdの評価値Valが次のとおりとなったとする。
[駆動制御Dc,制動制御Bc,操舵制御Sc,緩衝制御Cc,UI制御Ui,認識制御Rc,伝達制御Tc,バッテリー制御Ec,・・・]
=[8.5,7.5,7.0,9.0,5.0,3.0,2.0,1.5,・・・]
As a result of handling the driving scene Ds in which the road surface condition is bad such as frozen or unpaved as the environment in which the vehicle 2 travels and deriving the evaluation value Val based on the above evaluation items, the NN data Nd in each control item Ci It is assumed that the evaluation value Val is as follows.
[Drive control Dc, Braking control Bc, Steering control Sc, Buffer control Cc, UI control Ui, Recognition control Rc, Transmission control Tc, Battery control Ec, ...]
= [8.5, 7.5, 7.0, 9.0, 5.0, 3.0, 2.0, 1.5, ...]
この評価値Valをもとに、制御の優先度Pが高いものを抽出する。抽出の方法としては、例えば、評価値Valが6.0以上の制御項目Ciを選択する方法が挙げられる。これにより、緩衝制御Cc(評価値Val:9.0)、駆動制御Dc(評価値Val:8.5)、制動制御Bc(評価値Val:7.5)及び操舵制御Sc(評価値Val:7.0)の4つの制御項目Ciが抽出されたとする。 Based on this evaluation value Val, those having a high control priority P are extracted. As an extraction method, for example, a method of selecting a control item Ci having an evaluation value Val of 6.0 or more can be mentioned. As a result, buffer control Cc (evaluation value Val: 9.0), drive control Dc (evaluation value Val: 8.5), braking control Bc (evaluation value Val: 7.5), and steering control Sc (evaluation value Val: 7.5). It is assumed that the four control items Ci of 7.0) are extracted.
抽出された4つの制御項目Ciに対して、評価値Valの高い順に優先度Pを設けて統合学習を行わせる。さらに、その他の走行シーンDsについても同様にして、評価、抽出及び優先度Pの決定を行い、各々のNNデータNdに統合学習を行わせる。 For the four extracted control items Ci, priority P is set in descending order of the evaluation value Val, and integrated learning is performed. Further, for other driving scenes Ds, evaluation, extraction and determination of priority P are performed in the same manner, and integrated learning is performed on each NN data Nd.
以上説明したように、実施の形態3によれば、実施の形態2のAI統合システム1において、評価部30eが、例えば、各々のセンサ処理部31のNNデータNdにおける統合前の学習過程に関する情報、及び、各々のNNデータNdでの制御の貢献度合いなどをもとに評価値Valを導出し、統括部30(学習部30a)が、導出された評価値Valをもとに各々の制御項目Ciの制御の優先度Pを決定して統合学習を行わせることが可能となる。なお、実施の形態2の方法と合わせて制御の優先度Pを導出することも可能である。 As described above, according to the third embodiment, in the AI integrated system 1 of the second embodiment, the evaluation unit 30e provides information on, for example, the learning process before integration in the NN data Nd of each sensor processing unit 31. , And the evaluation value Val is derived based on the degree of contribution of control in each NN data Nd, and the control unit 30 (learning unit 30a) determines each control item based on the derived evaluation value Val. It is possible to determine the priority P of the control of Ci and perform integrated learning. It is also possible to derive the control priority P in combination with the method of the second embodiment.
実施の形態4.
実施の形態4では、実施の形態3で抽出された制御項目Ciをさらに組み合わせて統合学習を行わせる。
Embodiment 4.
In the fourth embodiment, the control item Ci extracted in the third embodiment is further combined to perform integrated learning.
図26は、本開示の実施の形態4における、NNデータNdへの統合学習を通じて制御の優先度Pを決定する別の過程を説明するための模式図である。実施の形態3の例では、学習部30aは、車両2に統合する種々のセンサ処理部31が担う制御項目Ciのうち、緩衝制御Cc、駆動制御Dc、制動制御Bc及び操舵制御Scの4つを抽出した。 FIG. 26 is a schematic diagram for explaining another process of determining the priority P of control through integrated learning to NN data Nd in the fourth embodiment of the present disclosure. In the example of the third embodiment, the learning unit 30a has four control items Ci, the buffer control Cc, the drive control Dc, the braking control Bc, and the steering control Sc, among the control items Ci carried by the various sensor processing units 31 integrated in the vehicle 2. Was extracted.
本実施の形態では、学習部30aは、抽出された4つの制御項目Ci(つまり、駆動制御Dc、制動制御Bc、操舵制御Sc及び緩衝制御Cc)のうちから2つ以上を組み合わせて、試行的な統合学習を行わせる。 In the present embodiment, the learning unit 30a combines two or more of the four extracted control items Ci (that is, drive control Dc, braking control Bc, steering control Sc, and buffer control Cc) on a trial basis. Have students perform integrated learning.
図26(a)では、学習部30aは、抽出された4つの制御項目Ciのうち2つを組み合わせた6パターンについて、組み合わせ対象の2つの制御項目Ciは追加学習対応モードとし、その他の制御項目Ciは追加学習非対応モードとして、試行的な統合学習を行わせる。そして、評価部30eは、組み合わせた6パターンについて試行的な統合学習を行わせたときの学習過程の評価を行う。学習過程の評価としては、例えば、各々のNNデータNdの出力(つまり、制御信号Cs)のばらつき度合い、又は、図19~図21に示した制御の安定評価Seなどが考えられる。学習過程において、各々のNNデータNdの出力(つまり、制御信号Cs)のばらつき度合いが大きければ、統合学習の候補として低い評価を与えることにより、制御のばらつき度合いが小さいNNデータNdの組み合わせを優先して統合学習させることが可能となる。このような試行的な統合学習での評価をもとに、学習部30aは、優先的に組み合わせて統合学習を行わせるNNデータNdを決定し、組み合わせたそれぞれのNNデータNdの学習が収束するまで統合学習を行わせる。 In FIG. 26A, the learning unit 30a sets the two control item Cis to be combined as the additional learning support mode for the six patterns in which two of the four extracted control item Cis are combined, and other control items. Ci performs trial integrated learning as a mode that does not support additional learning. Then, the evaluation unit 30e evaluates the learning process when trial integrated learning is performed on the combined 6 patterns. As the evaluation of the learning process, for example, the degree of variation in the output of each NN data Nd (that is, the control signal Cs), the stability evaluation Se of the control shown in FIGS. 19 to 21, and the like can be considered. In the learning process, if the degree of variation in the output of each NN data Nd (that is, the control signal Cs) is large, a low evaluation is given as a candidate for integrated learning, and the combination of NN data Nd with a small degree of control variation is prioritized. It becomes possible to make integrated learning. Based on the evaluation in such trial integrated learning, the learning unit 30a determines the NN data Nd to be preferentially combined to perform integrated learning, and the learning of each combined NN data Nd converges. Let them perform integrated learning.
図26(b)では、学習部30aは、抽出された4つの制御項目Ciのうち2つの組み合わせの順序を考慮した12パターンについて、組み合わせ対象の2つの制御項目Ciは順を変えて追加学習対応モードとし、その他の制御項目Ciは追加学習非対応モードとして、試行的な統合学習を行わせる。そして、図26(a)と同様にして、評価部30eは、組み合わせた12パターンについて試行的な統合学習を行わせたときの学習過程の評価を行う。このような試行的な統合学習での評価をもとに、学習部30aは、順序を考慮したうえで優先的に組み合わせて統合学習を行わせるNNデータNdを決定し、組み合わせたNNデータNdの学習が順次収束するまで統合学習を行わせる。 In FIG. 26B, the learning unit 30a supports additional learning by changing the order of the two control item Cis to be combined for 12 patterns considering the order of the combination of two of the four extracted control item Cis. The mode is set, and the other control item Ci is set as a mode that does not support additional learning, and trial integrated learning is performed. Then, in the same manner as in FIG. 26A, the evaluation unit 30e evaluates the learning process when trial integrated learning is performed on the combined 12 patterns. Based on the evaluation in such trial integrated learning, the learning unit 30a determines the NN data Nd to be preferentially combined and performed integrated learning in consideration of the order, and the combined NN data Nd. Let the integrated learning be performed until the learning converges in sequence.
図26(a)及び(b)は一例であって、学習部30aは、種々の制御項目Ciから複数のNNデータNdを組み合わせて試行的な統合学習を行わせるとともに、優先度Pを決定して統合学習を行わせてもよい。 26 (a) and 26 (b) are examples, and the learning unit 30a causes trial integrated learning to be performed by combining a plurality of NN data Nd from various control items Ci, and determines the priority P. You may have them perform integrated learning.
以上説明したように、実施の形態4によれば、実施の形態2のAI統合システム1において、学習部30aが、例えば、各々のセンサ処理部31のNNデータNdを組み合わせた複数のパターンにおけるNNデータNdでの試行的な統合学習を行わせて、評価部30eが、これらの試行的な統合学習に対して評価を行う。そして、学習部30aが、評価部30eでの評価をもとに、NNデータNdの組み合わせの優先度Pを決定し、NNデータNdの統合学習を行わせることが可能となる。なお、実施の形態2及び3の方法と合わせて制御の優先度Pを導出することも可能である。 As described above, according to the fourth embodiment, in the AI integrated system 1 of the second embodiment, the learning unit 30a is, for example, an NN in a plurality of patterns in which the NN data Nd of each sensor processing unit 31 is combined. Trial integrated learning is performed with the data Nd, and the evaluation unit 30e evaluates these trial integrated learning. Then, the learning unit 30a can determine the priority P of the combination of the NN data Nd based on the evaluation by the evaluation unit 30e, and make it possible to perform the integrated learning of the NN data Nd. It is also possible to derive the control priority P in combination with the methods of the second and third embodiments.
上述した実施の形態1から4におけるニューラルネットワークは、一般に、複数のニューロンからなる入力層、複数のニューロンからなる中間層(隠れ層)、及び複数のニューロンからなる出力層で構成される。中間層は、1層、又は2層以上でもよい。
図27は、3層のニューラルネットワークの例を説明するための模式図である。例えば、図27に示すような3層のニューラルネットワークであれば、複数の入力が入力層(X1‐X3)に入力されると、その値に重みW1(w11‐w16)を掛けて中間層(Y1‐Y2)に入力され、その結果にさらに重みW2(w21‐w26)を掛けて出力層(Z1‐Z3)から出力される。この出力結果は、重みW1とW2の値によって変わる。
また、学習器Lmに適用される学習アルゴリズムとしては、特徴量そのものの抽出を学習する、深層学習(Deep Learning)を用いることもでき、他の公知の方法、例えば遺伝的プログラミング、機能論理プログラミング、サポートベクターマシンなどに従って機械学習を実行してもよい。
The neural network according to the above-described embodiments 1 to 4 is generally composed of an input layer composed of a plurality of neurons, an intermediate layer (hidden layer) composed of a plurality of neurons, and an output layer composed of a plurality of neurons. The intermediate layer may be one layer or two or more layers.
FIG. 27 is a schematic diagram for explaining an example of a three-layer neural network. For example, in the case of a three-layer neural network as shown in FIG. 27, when a plurality of inputs are input to the input layer (X1-X3), the value is multiplied by the weight W1 (w11-w16) to form an intermediate layer (w11-w16). It is input to Y1-Y2), and the result is further multiplied by the weight W2 (w21-w26) and output from the output layer (Z1-Z3). This output result depends on the values of the weights W1 and W2.
Further, as a learning algorithm applied to the learner Lm, deep learning that learns the extraction of the feature amount itself can also be used, and other known methods such as genetic programming and functional logic programming can be used. Machine learning may be performed according to a support vector machine or the like.
図28は、上述した各実施の形態並びに変形例に示した本開示の技術を実施するためのハードウェア構成例である。ハードウェアは、少なくとも、演算装置であるCPU、メモリなどの記憶デバイスである補助記憶装置、及び、ハードディスク又は光ディスクなどの主記憶装置で構成される。ハードウェア構成は図28の例に限られることはない。さらに、外部ネットワークと接続する通信デバイスを備えていてもよい。 FIG. 28 is a hardware configuration example for implementing the technique of the present disclosure shown in each of the above-described embodiments and modifications. The hardware is composed of at least a CPU which is an arithmetic unit, an auxiliary storage device which is a storage device such as a memory, and a main storage device such as a hard disk or an optical disk. The hardware configuration is not limited to the example of FIG. 28. Further, it may be provided with a communication device for connecting to an external network.
<各実施の形態並びに変形例の主な構成>
AI統合システムは、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び外部ネットワークの少なくともいずれかを介して入力とし、制御対象の装置を制御するための制御信号を生成する複数の学習済みモデルのうちから、入力した検知情報をもとに、1つを選択する統括部と、選択された学習済みモデルを用いて、制御対象の装置を制御するセンサ処理部とを備える。なお、統括部は、入力した検知情報及び制御信号の少なくともいずれかに基づき、複数の学習済みモデルのうちから1つを選択するようにしてもよい。
<Main configurations of each embodiment and modified examples>
The AI integrated system receives detection information indicating the characteristics of the environment in which the controlled device operates as input via at least one of a sensor and an external network, and generates a plurality of control signals for controlling the controlled device. It includes a control unit that selects one of the trained models based on the input detection information, and a sensor processing unit that controls the device to be controlled by using the selected trained model. The control unit may select one of the plurality of trained models based on at least one of the input detection information and the control signal.
また、AI統合システムは、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び通信ネットワークの少なくともいずれかを介して入力とし、制御対象の装置を制御するための制御信号を生成する学習済みモデルと、複数の制御対象の装置の各々と対応して学習済みモデルを用いて制御する複数のセンサ処理部と、複数のセンサ処理部の各々に対し、学習済みモデルに追加学習させる学習部とを備える。なお、学習部は、複数のセンサ処理部のうち少なくともいずれかに対し追加学習させるようにしてもよい。 Further, the AI integrated system receives detection information indicating the characteristics of the environment in which the controlled device operates as input via at least one of a sensor and a communication network, and generates a control signal for controlling the controlled device. Learning to make the trained model additionally trained for each of the trained model, the plurality of sensor processing units that are controlled by using the trained model corresponding to each of the plurality of controlled devices, and the plurality of sensor processing units. It has a part. The learning unit may have at least one of the plurality of sensor processing units perform additional learning.
また、AI統合装置は、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び外部ネットワークの少なくともいずれかを介して入力として制御対象の装置を制御するための制御信号を生成する複数の学習済みモデルのうち、入力した検知情報をもとに1つを選択して制御させる統括部を備える。なお、統括部は、入力した検知情報及び制御信号の少なくともいずれかに基づき、複数の学習済みモデルのうちから1つを選択するようにしてもよい。 Further, the AI integrated device generates a plurality of control signals for controlling the device to be controlled by inputting detection information indicating the characteristics of the environment in which the device to be controlled is operated via at least one of a sensor and an external network. It is equipped with a control unit that selects and controls one of the trained models of the above based on the input detection information. The control unit may select one of the plurality of trained models based on at least one of the input detection information and the control signal.
また、AI統合装置は、複数の制御対象の装置の各々と対応した、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び通信ネットワークの少なくともいずれかを介して入力として制御対象の装置を制御するための制御信号を生成する各々の学習済みモデルの少なくともいずれかに対し、優先的に追加学習させる統括部を備える。 Further, the AI integrated device is controlled by inputting detection information indicating the characteristics of the environment in which the controlled device operates, which corresponds to each of the plurality of controlled devices, as input via at least one of the sensor and the communication network. It is provided with a control unit that preferentially performs additional learning for at least one of the trained models that generate a control signal for controlling the device.
また、AI統合プログラムは、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び外部ネットワークの少なくともいずれかを介して入力として制御対象の装置を制御するための制御信号を生成する複数の学習済みモデルのうちから、入力した検知情報をもとに、環境に適する1つを選択し、制御対象の装置を制御させる。なお、AI統合プログラムは、入力した検知情報及び制御信号の少なくともいずれかに基づき、複数の学習済みモデルのうちから1つを選択するようにしてもよい。 Further, the AI integrated program generates a plurality of control signals for controlling the controlled device by inputting detection information indicating the characteristics of the environment in which the controlled device operates as an input via at least one of a sensor and an external network. From the trained models of, select one that is suitable for the environment based on the input detection information, and control the device to be controlled. The AI integrated program may select one of a plurality of trained models based on at least one of the input detection information and the control signal.
また、AI統合プログラムは、複数の制御対象の装置の各々と対応した、制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び通信ネットワークの少なくともいずれかを介して入力として制御対象の装置を制御するための制御信号を生成する学習済みモデルの少なくともいずれかに対し、優先的に追加学習させる。 Further, the AI integrated program is controlled by inputting detection information indicating the characteristics of the environment in which the controlled device operates, which corresponds to each of the plurality of controlled devices, as input via at least one of the sensor and the communication network. Priority is given to additional training for at least one of the trained models that generate a control signal for controlling the device.
1 AI統合システム、2 車両、3 車体、4 タイヤ、5 ドア、6 ヘッドライト、7 電子ミラー、8 車載カメラ、9 レーダ、10 トランスミッション、11 駆動装置、12 制動装置、13 操舵装置、14 緩衝装置、15 UI装置、16 認識装置、17 伝達装置、21 駆動制御部、22 制動制御部、23 操舵制御部、24 緩衝制御部、25 UI制御部、26 認識制御部、27 伝達制御部、30 統括部、30e 評価部、30s 選択部、30a 学習部、30m 記憶部、31,31A,31B,・・・,31N センサ処理部、32,32A,32B,・・・,32N センサ、40 車載ネットワーク、41 通信装置、42 信号伝送路、101 サブシステム、Nd,NdA,NdA1,NdA2,・・・,NdB,NdB1,・・・,NdNn NNデータ、Lm 学習器、Si 検知情報、Cs 制御信号、Sw 切り替え指示、Rs 再選択指示、AL 追加学習対応モード、NL 追加学習非対応モード、Ds,DsA,DsB,・・・,DsN 走行シーン、Ca,CaA,CaB,・・・,CaN 制御モデル、Se 安定評価、P 制御の優先度、Ci 制御項目、Dc 駆動制御、Bc 制動制御、Sc 操舵制御、Cc 緩衝制御、Ui UI制御、Rc 認識制御、Tc 伝達制御、Ia 制御可能域、Ra ロバスト制御域、Na 適応外領域。 1 AI integrated system, 2 vehicles, 3 car bodies, 4 tires, 5 doors, 6 headlights, 7 electronic mirrors, 8 in-vehicle cameras, 9 radars, 10 transmissions, 11 drive devices, 12 braking devices, 13 steering devices, 14 shock absorbers. , 15 UI device, 16 recognition device, 17 transmission device, 21 drive control unit, 22 braking control unit, 23 steering control unit, 24 buffer control unit, 25 UI control unit, 26 recognition control unit, 27 transmission control unit, 30 supervision Unit, 30e evaluation unit, 30s selection unit, 30a learning unit, 30m storage unit, 31,31A, 31B, ..., 31N sensor processing unit, 32,32A, 32B, ..., 32N sensor, 40 in-vehicle network, 41 communication device, 42 signal transmission path, 101 subsystem, Nd, NdA, NdA1, NdA2, ..., NdB, NdB1, ..., NdNn NN data, Lm learner, Si detection information, Cs control signal, Sw. Switching instruction, Rs reselection instruction, AL additional learning compatible mode, NL additional learning non-compatible mode, Ds, DsA, DsB, ..., DsN driving scene, Ca, CaA, CaB, ..., CaN control model, Se Stability evaluation, P control priority, Ci control items, Dc drive control, Bc braking control, Sc steering control, Cc buffer control, Ui UI control, Rc recognition control, Tc transmission control, Ia controllable area, Ra robust control area , Na Non-adaptive area.

Claims (13)

  1.  制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び外部ネットワークの少なくともいずれかを介して入力とし、前記制御対象の装置を制御するための制御信号を生成する複数の学習済みモデルのうちから、前記検知情報又は生成された前記制御信号の少なくともいずれかをもとに、1つを選択する統括部と、
     選択された前記学習済みモデルを用いて、前記制御対象の装置を制御するセンサ処理部と、
    を備えるAI統合システム。
    A plurality of trained models that input detection information indicating the characteristics of the environment in which the controlled device operates via at least one of a sensor and an external network and generate a control signal for controlling the controlled device. A control unit that selects one based on at least one of the detection information or the generated control signal.
    A sensor processing unit that controls the device to be controlled using the selected trained model, and a sensor processing unit.
    AI integrated system with.
  2.  前記統括部は、
    前記センサ処理部から入力する前記検知情報をもとに前記環境を判定し、前記複数の学習済みモデルのうちから前記センサ処理部が用いる学習済みモデルを選択する選択部を有する、
    請求項1に記載のAI統合システム。
    The control department
    It has a selection unit that determines the environment based on the detection information input from the sensor processing unit and selects a trained model used by the sensor processing unit from the plurality of trained models.
    The AI integrated system according to claim 1.
  3.  前記統括部は、
    入力する前記検知情報及び前記制御信号をもとに、前記センサ処理部が用いている前記学習済みモデルの制御を評価する評価部を有し、
     前記選択部は、前記評価部での評価に応じて前記複数の学習済みモデルのうちから前記センサ処理部が用いる学習済みモデルを選択する、
    請求項2に記載のAI統合システム。
    The control department
    It has an evaluation unit that evaluates the control of the trained model used by the sensor processing unit based on the detection information and the control signal to be input.
    The selection unit selects a trained model used by the sensor processing unit from the plurality of trained models according to the evaluation by the evaluation unit.
    The AI integrated system according to claim 2.
  4.  前記統括部は、
    複数の前記制御対象の装置の各々と対応した複数の前記センサ処理部の各々に対して、前記制御対象の装置を制御する前記複数の学習済みモデルのうちから1つを選択する、
    請求項1から3のいずれか1項に記載のAI統合システム。
    The control department
    For each of the plurality of sensor processing units corresponding to each of the plurality of controlled devices, one of the plurality of trained models that control the controlled device is selected.
    The AI integrated system according to any one of claims 1 to 3.
  5.  前記統括部は、
    前記複数のセンサ処理部に対して、使用中の前記学習済みモデルに追加学習させる学習部を有する、
    請求項1から4のいずれか1項に記載のAI統合システム。
    The control department
    It has a learning unit for additionally learning the trained model in use for the plurality of sensor processing units.
    The AI integrated system according to any one of claims 1 to 4.
  6.  制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び通信ネットワークの少なくともいずれかを介して入力とし、前記制御対象の装置を制御するための制御信号を生成する学習済みモデルと、
     複数の前記制御対象の装置の各々と対応して前記学習済みモデルを用いて制御する複数のセンサ処理部と、
     前記複数のセンサ処理部の各々に対し、前記学習済みモデルに追加学習させる学習部と、
    を備えるAI統合システム。
    A trained model that inputs detection information indicating the characteristics of the environment in which the controlled device operates via at least one of a sensor and a communication network and generates a control signal for controlling the controlled device.
    A plurality of sensor processing units that are controlled by using the trained model corresponding to each of the plurality of controlled devices, and a plurality of sensor processing units.
    A learning unit that causes each of the plurality of sensor processing units to additionally learn from the trained model.
    AI integrated system with.
  7.  前記学習部は、
    前記複数のセンサ処理部の各々で使用中の前記学習済みモデルの優先度を導出し、導出した優先度に基づき、前記複数のセンサ処理部の少なくともいずれかに対して、使用中の前記学習済みモデルに追加学習させる、
    請求項5または6に記載のAI統合システム。
    The learning unit
    The priority of the trained model used by each of the plurality of sensor processing units is derived, and based on the derived priority, the trained trained unit in use is used for at least one of the plurality of sensor processing units. Let the model learn more,
    The AI integrated system according to claim 5 or 6.
  8.  前記学習部は、
    前記複数のセンサ処理部の各々で使用中の前記学習済みモデルでの制御に基づき、前記優先度を導出する、
    請求項7に記載のAI統合システム。
    The learning unit
    The priority is derived based on the control in the trained model in use by each of the plurality of sensor processing units.
    The AI integrated system according to claim 7.
  9.  前記学習部は、
    前記複数のセンサ処理部の各々が制御する制御対象の装置に基づき、前記優先度を導出する、
    請求項7に記載のAI統合システム。
    The learning unit
    The priority is derived based on the device to be controlled controlled by each of the plurality of sensor processing units.
    The AI integrated system according to claim 7.
  10.  制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び外部ネットワークの少なくともいずれかを介して入力として前記制御対象の装置を制御するための制御信号を生成する複数の学習済みモデルのうちから、前記検知情報又は生成された前記制御信号の少なくともいずれかをもとに、1つを選択する統括部を備えるAI統合装置。 Of a plurality of trained models that generate control signals for controlling the controlled device by inputting detection information indicating the characteristics of the environment in which the controlled device operates via at least one of a sensor and an external network. An AI integrated device including a control unit that selects one based on at least one of the detection information and the generated control signal.
  11.  複数の制御対象の装置の各々と対応した、前記制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び通信ネットワークの少なくともいずれかを介して入力として前記制御対象の装置を制御するための制御信号を生成する各々の学習済みモデルの少なくともいずれかに対し、優先的に追加学習させる統括部を備えるAI統合装置。 To control the controlled device by inputting detection information indicating the characteristics of the environment in which the controlled device operates, which corresponds to each of the plurality of controlled devices, as an input via at least one of a sensor and a communication network. An AI integrated device including a control unit that preferentially performs additional learning for at least one of the trained models that generate the control signal of.
  12.  制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び外部ネットワークの少なくともいずれかを介して入力として前記制御対象の装置を制御するための制御信号を生成する複数の学習済みモデルのうちから、前記検知情報又は生成された前記制御信号の少なくともいずれかをもとに、前記環境に適する1つを選択し、前記制御対象の装置を制御させる、AI統合プログラム。 Of a plurality of trained models that generate control signals for controlling the controlled device by inputting detection information indicating the characteristics of the environment in which the controlled device operates via at least one of a sensor and an external network. An AI integrated program that selects one suitable for the environment based on at least one of the detection information or the generated control signal and controls the device to be controlled.
  13.  複数の制御対象の装置の各々と対応した、前記制御対象の装置が動作する環境の特徴を示す検知情報をセンサ及び通信ネットワークの少なくともいずれかを介して入力として前記制御対象の装置を制御するための制御信号を生成する各々の学習済みモデルの少なくともいずれかに対し、優先的に追加学習させる、AI統合プログラム。 To control the controlled device by inputting detection information indicating the characteristics of the environment in which the controlled device operates, which corresponds to each of the plurality of controlled devices, as an input via at least one of a sensor and a communication network. An AI integrated program that preferentially additionally trains at least one of the trained models that generate the control signal of.
PCT/JP2020/025175 2020-06-26 2020-06-26 Ai integration system, ai integration device, and ai integration program WO2021260910A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2022532199A JP7414995B2 (en) 2020-06-26 2020-06-26 AI integrated system, AI integrated device and AI integrated program
PCT/JP2020/025175 WO2021260910A1 (en) 2020-06-26 2020-06-26 Ai integration system, ai integration device, and ai integration program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/025175 WO2021260910A1 (en) 2020-06-26 2020-06-26 Ai integration system, ai integration device, and ai integration program

Publications (1)

Publication Number Publication Date
WO2021260910A1 true WO2021260910A1 (en) 2021-12-30

Family

ID=79282164

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/025175 WO2021260910A1 (en) 2020-06-26 2020-06-26 Ai integration system, ai integration device, and ai integration program

Country Status (2)

Country Link
JP (1) JP7414995B2 (en)
WO (1) WO2021260910A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018173121A1 (en) * 2017-03-21 2018-09-27 株式会社Preferred Networks Server device, trained model providing program, trained model providing method, and trained model providing system
WO2019193660A1 (en) * 2018-04-03 2019-10-10 株式会社ウフル Machine-learned model switching system, edge device, machine-learned model switching method, and program
WO2020090251A1 (en) * 2018-10-30 2020-05-07 日本電気株式会社 Object recognition device, object recognition method, and object recognition program

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002104104A (en) 2001-06-11 2002-04-10 Hitachi Ltd Cooperative control device for automobile
US20220001858A1 (en) * 2018-11-13 2022-01-06 Nec Corporation Dangerous scene prediction device, dangerous scene prediction method, and dangerous scene prediction program

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018173121A1 (en) * 2017-03-21 2018-09-27 株式会社Preferred Networks Server device, trained model providing program, trained model providing method, and trained model providing system
WO2019193660A1 (en) * 2018-04-03 2019-10-10 株式会社ウフル Machine-learned model switching system, edge device, machine-learned model switching method, and program
WO2020090251A1 (en) * 2018-10-30 2020-05-07 日本電気株式会社 Object recognition device, object recognition method, and object recognition program

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
IBUKI, TAKERO ET AL.: "Development of a deep learning basis that enables the creation and offering of A1 models in a short period of time", NTT DOCOMO TECHNICAL JOURNAL, vol. 25, no. 4, 31 January 2018 (2018-01-31), pages 12 - 18 *

Also Published As

Publication number Publication date
JPWO2021260910A1 (en) 2021-12-30
JP7414995B2 (en) 2024-01-16

Similar Documents

Publication Publication Date Title
US20220379920A1 (en) Trajectory planning method and apparatus
US10235881B2 (en) Autonomous operation capability configuration for a vehicle
US10824155B2 (en) Predicting movement intent of objects
CN113386795A (en) Intelligent decision-making and local track planning method for automatic driving vehicle and decision-making system thereof
CN114391088B (en) Trajectory planner
Ferrara et al. Second order sliding mode control of vehicles with distributed collision avoidance capabilities
EP3814909A2 (en) Using divergence to conduct log-based simulations
CN113835421B (en) Method and device for training driving behavior decision model
KR101876063B1 (en) Method for deciding a road surface using vehicle data
CN109733474A (en) A kind of intelligent vehicle steering control system and method based on piecewise affine hierarchical control
CN112577506B (en) Automatic driving local path planning method and system
US11300968B2 (en) Navigating congested environments with risk level sets
CN110794851A (en) Vehicle remote control safety protection method and device and unmanned vehicle
CA3155591C (en) Functional safety in autonomous driving
US20240010198A1 (en) Methods and Systems for Adjusting Vehicle Behavior Based on Ambient Ground Relative Wind Speed Estimations
EP4219253A1 (en) Electromechanical braking method and electromechanical braking device
Ali et al. Minimizing the inter-vehicle distances of the time headway policy for urban platoon control with decoupled longitudinal and lateral control
WO2023010043A1 (en) Complementary control system for an autonomous vehicle
CN117980212A (en) Planning system based on optimization
Ali et al. Urban platooning using a flatbed tow truck model
WO2021260910A1 (en) Ai integration system, ai integration device, and ai integration program
CN115712950A (en) Automatic driving decision-making method for semi-trailer
US20220242441A1 (en) Systems and methods for updating the parameters of a model predictive controller with learned operational and vehicle parameters generated using simulations and machine learning
CN111857112A (en) Automobile local path planning method and electronic equipment
KR102616457B1 (en) Air Suspension Operation Planning Generation Device for Autonomous Vehicles

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20941742

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022532199

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20941742

Country of ref document: EP

Kind code of ref document: A1