JP2019046371A - Vehicle controller recognizing traffic sign and method of the same - Google Patents

Abstract

To improve recognition accuracy of a traffic sign in a recognition system of the traffic sign.SOLUTION: A vehicle controller executing a control of a vehicle, acquires an image obtained by photographing an environment of a circumference of a vehicle, acquires information of a user operation for a vehicle received by the vehicle, extracts information of a traffic sign included in the image by using an operation mode determined on the basis of the user operation, and controls the vehicle so that the vehicle executes a predetermined processing on the basis of the information of the traffic sign.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a traffic sign recognition technique.

  There is a system for recognizing a traffic sign based on an image taken by a camera attached to a vehicle. Since traffic signs differ for each country or region, it is important to specify the country or region in which the vehicle is traveling in order to evaluate these traffic signs with high accuracy. In Patent Document 1, a country signal indicating the current location of a vehicle is generated based on external information such as a lane and a traffic sign, and is input to a system for recognizing a traffic sign, and the country-specific sign information is used. By doing so, a technique for suppressing erroneous detection is described.

German Patent Application Publication No. 102009057553

  In the technique of Patent Document 1, when the recognition accuracy of external information deteriorates, the accuracy of the country signal also deteriorates. As a result, the recognition accuracy of the traffic sign based on the country signal may also deteriorate. Although it is possible to recognize traffic signs using all of the sign information of many countries and regions, in this case, false detection between different signs in multiple countries with similar shapes and characteristics may occur. May end up.

  The present invention has been made in view of the above problems, and an object of the present invention is to improve the accuracy of traffic sign recognition in a traffic sign recognition system.

  According to the present invention, a vehicle control device that executes control of a vehicle, acquires an image obtained by capturing an environment around the vehicle, acquires information on a user operation on the vehicle received in the vehicle, Using the operation mode determined based on the user operation, the information on the traffic sign included in the image is extracted, and the vehicle is controlled to execute a predetermined process based on the information on the traffic sign. A vehicle control device configured as described above is provided.

  ADVANTAGE OF THE INVENTION According to this invention, the recognition accuracy of a traffic sign can be improved in the traffic sign recognition system.

The block diagram explaining the structure of an apparatus. The flowchart which shows the example of the flow of the process which a vehicle control apparatus performs. The flowchart which shows the example of the flow of the process which a vehicle control apparatus performs.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present invention is applicable to any vehicle equipped with a sign recognition system that recognizes traffic signs. For example, the following discussion can be applied to any vehicle that recognizes a sign by a sign recognition system and executes predetermined vehicle control (such as information output by video or audio) using the recognition result. In some cases, the vehicle control device based on the sign recognition need not be provided in the vehicle. For example, the vehicle transmits sensing information such as an image taken by an in-vehicle camera, information such as an operation by a user of the vehicle to a vehicle control device existing outside the vehicle, and traffic based on the information in the vehicle control device Information on the recognition result of the sign may be received. The information of the recognition result may be a vehicle control command in one example, and the vehicle performs predetermined vehicle control according to this information.

(Device configuration)
FIG. 1 shows a block diagram of a vehicle control device 2 according to the present embodiment for controlling the vehicle 1. The vehicle 1 is a four-wheeled passenger car as an example, but the vehicle 1 may be a vehicle other than a four-wheel vehicle such as a two-wheeled vehicle. The vehicle 1 includes, in one example, a vehicle control device 2, a camera 11, a display unit 12, and an operation unit 13. The vehicle control device 2 includes, in one example, a recognition unit 21, a reception unit 22, a determination unit 23, and an output unit. 24. In addition, each function part of the vehicle control apparatus 2 is comprised, for example by ECU (Electronic Control Unit), for example, is comprised so that communication is possible. The ECU includes a processor represented by a CPU (Central Processing Unit), a storage device such as a semiconductor memory, an interface with an external device, and the like. The storage device stores a program executed by the processor, data used by the processor for processing, and the like. Each ECU may include a plurality of processors, storage devices, interfaces, and the like. FIG. 1 is simplified for explaining the configuration of the vehicle control device 2 according to the present embodiment, and the vehicle 1 is generally used for driving and braking functions for the vehicle 1 to travel. Of course, the vehicle has functions (not shown) of the vehicle.

  The camera 11 captures the surroundings including the front of the vehicle 1 and inputs it to the recognition unit 21 of the vehicle control device 2. Other sensors may be used instead of or in addition to the camera 11. That is, any sensor that can detect the external environment can be used. The display unit 12 displays information based on the signal output from the output unit 24 of the vehicle control device 2. Note that a speaker that presents information by voice may be included instead of or in addition to the display unit 12. That is, an arbitrary information presentation function can be included in the vehicle 1. The operation unit 13 is an arbitrary interface that receives a user operation on the vehicle 1. For example, the operation unit 13 may include an interface for switching the unit of speed or distance display. The operation unit 13 may include an interface that accepts user operations such as radio channel switching.

  The recognition unit 21 acquires an image related to the external environment photographed by the camera 11 based on predetermined dictionary data, and recognizes and extracts a traffic sign included in the image. For example, the recognition unit 21 determines whether or not there is a region in the image data acquired by photographing with the camera 11 that matches the pattern of the traffic sign image data included in the dictionary data. This determination can be executed using a known pattern matching function. The recognition unit 21 transmits the determined result to the determination unit 23 and the output unit 24. The receiving unit 22 acquires information related to a user operation on the operation unit 13, monitors whether or not a user operation has been performed on the operation unit 13, and if there is a user operation, shows the specific operation content. To detect. The reception unit 22 outputs the monitoring and detection results to the determination unit 23. For example, the reception unit 22 may periodically output the monitoring and detection results, and may notify the determination unit 23 that no user operation has been performed, for example. The determination unit 23 determines dictionary data used in the recognition unit 21 based on the recognition result of the external environment by the recognition unit 21 and the user operation detected by the reception unit 22. Details of the processing of the determination unit 23 will be described later. The output unit 24 performs control for causing the display unit 12 to display information based on the traffic sign recognized by the recognition unit 21. Note that the output unit 24 may perform not only information display but also control for performing sound output, or may output arbitrary information for performing predetermined control on the other vehicle 1. Good.

(Process flow)
The vehicle control device 2 according to the present embodiment performs control of the vehicle 1 based on information on traffic signs extracted by the recognition unit 21 from an image obtained by photographing the surrounding environment with the camera 11. For example, the vehicle control device 2 causes the output unit 24 to present information on the display unit 12 in order to notify the driver of the presence of the traffic sign and the content of the traffic sign. In addition, when the vehicle 1 has an automatic driving function, the vehicle control device 2 performs traveling at a speed that does not exceed the speed limit in response to the extraction of a sign related to the speed limit from the captured image. The vehicle 1 can be controlled.

  The extraction of the traffic sign from the photographed image is performed on the basis of the reference information stored in the database by referring to the database of the sign used. For example, when traffic sign image data is used as a reference image, a sign is extracted from the captured image by pattern matching. At this time, for example, extraction of traffic signs using first dictionary data in which reference information for each country or region is aggregated, and extraction of traffic signs using second dictionary data in which reference information for a plurality of countries or regions is aggregated, At least one of the following may be performed. At this time, the extraction of the traffic sign by the first dictionary data does not include the reference information of other countries and regions, so that the traffic sign can be extracted with high accuracy. On the other hand, in order to use the first dictionary data, it is necessary to distinguish between countries and regions. For traffic sign extraction using the second dictionary data, common data can be used in multiple countries and regions, so it is not necessary to distinguish between countries and regions. In some cases, the accuracy of distinguishing between signs having different meanings is not sufficient. If the sign discrimination accuracy is insufficient, the accuracy of vehicle control such as driving control and information presentation may be insufficient, so it is important to use the first dictionary data in as many situations as possible. However, in some countries and regions, the discrimination accuracy of the country or region in which the vehicle is present is low, the first dictionary data cannot be used, and the second dictionary data is used as a result. The traffic sign recognition accuracy may be insufficient.

  The vehicle control device 2 according to the present embodiment uses the first dictionary data or the second dictionary based on a user operation on the operation unit 13 received via the reception unit 22 in order to cope with such a situation. The determination unit 23 determines whether to use data. Note that, when using the first dictionary data, the determination unit 23 further determines which country or region the first dictionary data is to be used for. In addition, if a sign peculiar to a country or region is detected while the second dictionary data is used, the dictionary data to be used is changed according to the country or region related to the country or region accordingly. 1 dictionary data can be changed. On the other hand, the vehicle control apparatus according to the present embodiment uses the first dictionary data based on the user operation even when this change cannot be made or when the user performs an operation related to the country or region in advance. Can be selected.

  For example, the vehicle control device determines an area where the vehicle 1 exists in response to switching of a speed unit or a distance unit used in the vehicle 1 by a user operation. For example, when the unit is switched from kilometers to miles by a user operation, the vehicle control device can determine that the vehicle 1 has entered a country using a mile display such as the United Kingdom. Similarly, the vehicle control device determines, for example, an area where the vehicle 1 is present depending on whether or not a user operation regarding the setting of driving related to whether the vehicle 1 performs left-hand traffic or right-hand traffic is accepted. . For example, when the setting of the vehicle 1 that was traveling in the right-hand traffic is switched to the left-hand traffic by the user operation, the determination unit 23 determines that the vehicle 1 has entered the left-hand traffic country such as the United Kingdom. Can do. At this time, the vehicle control device can determine the country or region based on the sensing result of the external environment in addition to the user operation. The external environment may be, for example, a language or the like that is mainly used in surrounding signboards, or may be a predetermined landmark. For example, many European countries use right-hand traffic and kilometers as the speed unit. On the other hand, for example, a country or region in which the vehicle 1 has a common traffic zone, speed unit, or the like by recognition of a traffic sign using the second dictionary data or identification of a license plate of another vehicle, for example. It is also possible to determine which of them is staying. That is, in the present embodiment, the country or region is determined based on the user operation, but the user operation is one of the elements used for determining the country or region, and the information related to the user operation includes the country or region determination accuracy. Used to improve Note that the user operation is a concept including not only “the operation was performed” but also “the operation was not performed”. For example, after the speed unit is set to kilometers, the vehicle control device determines that the vehicle 1 has not entered the country or region where the speed unit is a mile while the user operation for the setting is not performed. Can be judged. Therefore, in the following, it is referred to as a user operation including that there is no operation.

  When the determining unit 23 can determine the country or region based on the user operation, the recognizing unit 21 uses the first dictionary data corresponding to the country or region to detect the traffic sign from the image captured by the camera 11. Perform extraction. Further, the determination unit 23 selects a radio broadcast channel frequency or language, a traffic information broadcast service channel frequency or language, a network operator that provides a wireless communication service between the vehicle 1 and an external network, Based on the frequency or the like, the country or region in which the vehicle 1 exists can be specified. For example, the determination unit 23 holds a list of radio broadcast channel frequencies in each country or region, and determines whether the radio broadcast frequency selected by the user matches the channel frequency in the list. Identify the country or region that exists. In addition, when the same radio broadcast frequency or the like is used in a plurality of countries or regions, the determination unit 23, in addition to the user operation, for example, the traffic sign recognition result using the second dictionary data or the sensing of the external environment The country or region where the vehicle 1 stays can be specified using the result or the like. Moreover, the determination part 23 may specify the area | region where the vehicle 1 exists roughly according to the country or area identified as having stayed in the past within predetermined time, and may restrict | limit the list | wrist to use. For example, a general vehicle cannot move from France to Hungary within 3 hours. For this reason, when the vehicle 1 has existed in France within the past three hours, the determination unit 23 does not use a list relating to Hungary, for example, but lists countries or regions that can move within three hours from France. Can be used to determine the country or region where the vehicle 1 exists. As described above, the determination unit 23 specifies the country or region in which the vehicle 1 stays based on the user operation for switching that sets the vehicle 1 to correspond to the country or region in which the vehicle 1 stays. Corresponding first dictionary data can be used. As a result, the vehicle control device 2 can perform appropriate traffic sign recognition and vehicle control according to the traffic conditions in each country.

  In addition, the vehicle control apparatus 2 performs the recognition process of a traffic sign using 2nd dictionary data, when the country or area where the vehicle 1 stays cannot be specified based on user operation. Here, the environment in which the country or region cannot be specified based on the user operation may include, for example, a case where the user operation is incorrect. That is, when the user operation is incorrect or the operation that should be performed at the destination is not performed, the country or region specified based on the user operation and the country or region estimated from the surrounding environment conflict There can be. In this case, the determination unit 23 can determine that the country or region could not be specified, and can determine to use the second dictionary data. That is, the determination unit 23 uses the second dictionary data when the first dictionary data corresponding to the predetermined country or region has been used, and the second dictionary data is used until then. If so, it may be decided to continue using the second dictionary data. Thereby, even in an environment where the user performs an operation corresponding to the wrong country or region, the vehicle control device can execute the recognition of the traffic sign and the control of the vehicle.

  In addition, the vehicle control device 2 can move the vehicle to the country or region estimated from the surrounding environment or the like when the country or region specified based on the user operation conflicts with the country or region estimated from the surrounding environment or the like. Control may be performed so as to present a notification that prompts a user operation to make 1 correspond. For example, the vehicle control device 2 may display a screen on the display unit 12 so as to execute setting of a speed unit or a traffic band in a situation where the vehicle 1 is estimated to have moved to a country or region having a different traffic sign system. Information can be notified to the user by voice guidance using a speaker (not shown). Thereby, it becomes possible to make a user set the vehicle 1 appropriately. In addition, the vehicle control device uses the extracted traffic sign information for vehicle control when the country or region specified based on the user operation conflicts with the country or region estimated from the surrounding environment. It can be controlled not to. That is, the vehicle control device 2 determines not to use the extracted information (for example, not to output from the output unit 24) in a predetermined process for the vehicle 1, for example, not to execute the extracted traffic sign presentation process. sell. According to this, for example, dictionary data corresponding to different countries or regions is used, and it is possible to prevent inconvenience due to the use of a traffic sign recognition result having a high probability of being erroneous. In such a case, the vehicle control device 2 performs at least one of setting dictionary data to be used, notifying the user, and not using the extracted information in a predetermined process. It may be. That is, the vehicle control device 2 may execute a combination of the above-described processes.

  In the above description, the vehicle control device 2 has been described as setting dictionary data to be used based on a user operation. However, the vehicle control device 2 may be configured to set an operation mode other than the dictionary data setting. For example, the vehicle control device 2 is based on a user operation on the vehicle 1, instead of or in addition to dictionary data, for recognition of a traffic sign corresponding to a country or region or for control of display or the like. The parameter may be set to be used. For example, the vehicle control device 2 can improve the recognition accuracy of traffic signs by appropriately setting parameters. Moreover, the vehicle control apparatus 2 expands or reduces the traffic sign extraction range as necessary by appropriately setting the parameters, and the vehicle control device 2 has an appropriate range required for information presentation to the user, driving control, and the like. It is also possible to extract traffic signs. In other words, the vehicle control device 2 uses the appropriate dictionary data and operation modes corresponding to various parameters based on user operations on the vehicle 1 to improve the accuracy of traffic sign recognition and appropriate conditions associated with sign recognition. Can be controlled. In addition to the extraction of traffic signs, the vehicle control device 2 causes the driver to drive in accordance with the driver's tendency for each country or region (performs such automatic driving in some cases). User operation can be used to switch various processes such as control for the user. In the above description, the country or region is specified based on the user operation. However, in practice, it is only necessary to set the operation mode based on the user operation, and the country or region is specified in the processing. There may be no stages.

  The above-mentioned “user” is a driver in many cases, but includes other users such as passengers. Therefore, the user operation does not have to be an operation by the driver, and includes an operation performed by a vehicle user other than the driver.

  A summary of the above processing is shown in FIGS. The vehicle control device 2 according to the present embodiment starts the process of FIG. 2 based on a predetermined trigger such as, for example, that the vehicle 1 has been activated or a user has activated a predetermined function of the vehicle 1. .

  In the process of FIG. 2, the vehicle control device 2 obtains user operation information by monitoring whether a predetermined user operation is performed on the vehicle 1 (operation unit 13) via the reception unit 22 (S201). External environment information is acquired through the camera 11 and the recognition unit 21 (S202). Here, the predetermined user operation may be, for example, setting to a unit of speed or distance in kilometers or miles, selection of a radio frequency or language, selection of a radio communication network operator, frequency, or the like. Further, the external environment information may be a traffic sign extracted using the second dictionary data, a language mainly used in surrounding signs, a predetermined landmark, and the like. In addition, although the example in which the acquisition of the user operation information is performed prior to the acquisition of the external environment information has been described, the acquisition of the user operation information and the acquisition of the external environment information may be performed in parallel. Acquisition of information may be performed prior to acquisition of user operation information. Thereafter, the vehicle control device 2 determines an operation mode for extracting traffic sign information based on the acquired user operation information and external environment information by the determination unit 23 (S203).

  The process of S203 will be described with reference to FIG. In the process of S203, the vehicle control device 2 first determines whether the operation mode determined based on the user operation information acquired in S201 is consistent with the external environment information acquired in S202 (S301). When the operation mode based on the user operation information is consistent with the external environment information (YES in S301), the vehicle control device 2 determines the operation mode corresponding to the user operation information as the operation mode for extracting traffic sign information. (S302).

  On the other hand, when the operation mode based on the user operation information is not consistent with the external environment information (NO in S301), the vehicle control device 2 executes, for example, the process of S303, the process of S305, or both. In S303, the vehicle control device 2 performs a notification prompting the user operation in consideration of forgetting to set the unit of speed or distance, for example. When the user operation prompted by this is obtained (YES in S304), the vehicle control device 2 selects the operation mode corresponding to the user operation information as the operation mode for extracting traffic sign information ( S302). On the other hand, when the user operation is not obtained by the notification in S303 (NO in S304), the vehicle control device 2 sets the operation mode for extracting traffic sign information according to the external environment, for example, regardless of the user operation. It is set, or the non-use of the extraction result is determined (S305). For example, if there is no user operation prompted by the notification in S303 and the country or region can be determined based on the external environment information, the vehicle control device 2 uses the first dictionary data regarding the country or region. A mode can be selected. In addition, for example, when there is no user operation prompted by the notification in S303 and the country or region cannot be determined based on the external environment information, the vehicle control device 2 can select the operation mode using the second dictionary data. . Further, the vehicle control device 2 may use the second dictionary data regardless of whether or not the country or region can be determined when there is no user operation prompted by the notification in S303. In addition, the vehicle control device 2 sets the operation mode regardless of the user operation or maintains the previous operation mode, and uses the traffic sign extraction result in the operation mode for vehicle control. You can decide not to.

  Note that the operation mode may be determined only by a user operation. That is, for example, when a user operation is performed to support only a predetermined country or region, the operation mode may be set based on the user operation without performing the determination in S301. That is, the operation mode is determined based on at least a user operation, and may be determined in consideration of external environment information or the like as necessary. For this reason, the processing of S301 and S303 to S305 indicated by the dotted lines in FIG. May not be executed.

  Returning to FIG. 2, the vehicle control device 2 executes the traffic sign extraction process in the recognition unit 21 according to the operation mode determined in S203 (S204), and performs predetermined control on the vehicle 1 based on the extraction result. Run. The predetermined control can include, for example, presenting the extracted traffic sign information to the user and driving control based on the extracted traffic sign information.

  As mentioned above, the vehicle control apparatus 2 can improve the recognition accuracy of a traffic sign by setting an operation mode based on user operation with respect to a vehicle.

<Summary of Embodiment>
1. The vehicle control device of the above embodiment is
A vehicle control device that executes control of a vehicle,
Obtain an image of the surrounding environment of the vehicle,
Obtaining user operation information for the vehicle received in the vehicle;
Using the operation mode determined based on the user operation, the information on the traffic sign included in the image is extracted,
Controlling the vehicle to execute a predetermined process based on the information of the traffic sign,
It is comprised so that it may be comprised.

  According to this embodiment, the vehicle control apparatus can extract a traffic sign by an appropriate operation mode based on an operation by a user of the vehicle.

2. The vehicle control device of the above embodiment is
The user operation relates to switching for associating the setting of the vehicle with a country or region,
It is characterized by that.

  According to this embodiment, in response to an operation corresponding to the country or region where the vehicle stays, the vehicle control device can extract a traffic sign in an operation mode corresponding to the country or region. It becomes like this.

3. The vehicle control device of the above embodiment is
The switching includes switching in units of speed or distance,
It is characterized by that.

  According to this embodiment, the vehicle control device can determine a traffic sign between countries or regions having different speed or distance units with high accuracy.

4). The vehicle control device of the above embodiment is
The operation mode is an operation mode related to recognition of the traffic sign included in the image.
It is characterized by that.

  According to this embodiment, it is possible to determine an operation mode that enables a traffic sign to be recognized with high accuracy based on a user operation.

5. The vehicle control device of the above embodiment is
The operation mode related to the recognition of the traffic sign includes a first mode using reference information corresponding to each country or region, and a second mode using reference information in which information on a plurality of countries or regions is aggregated.
It is characterized by that.

  According to this embodiment, the reference information corresponding to the country or region can be used in the first mode to recognize the traffic sign with high accuracy, and the country or region can be specified in the second mode. It becomes possible to recognize traffic signs even when it is not possible.

6). The vehicle control device of the above embodiment is
When it is determined that the vehicle is traveling in a predetermined country or region based on the user operation, the reference information corresponding to the predetermined country or region is used for the image in the first mode. Configured to recognize the traffic sign included,
It is characterized by that.

  According to this embodiment, when the country or region in which the vehicle is traveling can be specified by user operation, the traffic sign can be extracted with high accuracy using the reference information corresponding to the country or region.

7). The vehicle control device of the above embodiment is
When the country or region in which the vehicle is traveling is not determined based on at least the user operation, the second mode is used to display the image using reference information common to the plurality of countries or regions. Configured to recognize the traffic sign included,
It is characterized by that.

  According to this embodiment, the traffic sign can be recognized even when the country or region in which the vehicle is traveling cannot be specified.

8). The vehicle control device of the above embodiment is
When a target that does not correspond to the operation mode determined based on the user operation is detected in the vehicle, setting of the operation mode not based on the user operation and notification of the user to perform the user operation on the vehicle And non-use in the control of the vehicle of the extracted information of the traffic sign is further configured to perform at least one of:
It is characterized by that.

  According to this embodiment, when a target that does not match the operation mode determined based on the user operation is detected, when the user operation is not appropriate or when the user forgets to perform the operation, the user operation The influence of the determination of the operation mode can be suppressed.

9. The vehicle of the above embodiment is
It has the above-mentioned vehicle control device.

  According to this, it is possible to execute appropriate control in real time by quickly executing the above-described processing inside the vehicle.

10. The method of the above embodiment
A method executed by a vehicle control device that controls a vehicle,
Obtaining an image of the surrounding environment of the vehicle;
Obtaining information of user operations on the vehicle received in the vehicle;
Extracting information on traffic signs included in the image using an operation mode determined based on the user operation;
Controlling the vehicle to execute a predetermined process based on the information of the traffic sign;
According to this embodiment, the vehicle control device can extract a traffic sign by an appropriate operation mode based on an operation by a user of the vehicle.

  The present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the spirit and scope of the present invention. Therefore, in order to make the scope of the present invention public, the following claims are attached.

1: vehicle, 2: vehicle control device, 11: camera, 12: display unit, 13: operation unit, 21: recognition unit, 22: reception unit, 23: determination unit, 24: output unit

Claims (10)

A vehicle control device that executes control of a vehicle,
Obtain an image of the surrounding environment of the vehicle,
Obtaining user operation information for the vehicle received in the vehicle;
Using the operation mode determined based on the user operation, the information on the traffic sign included in the image is extracted,
Controlling the vehicle to execute a predetermined process based on the information of the traffic sign,
A vehicle control device configured as described above. The user operation relates to switching for associating the setting of the vehicle with a country or region,
The vehicle control device according to claim 1. The switching includes switching in units of speed or distance,
The vehicle control device according to claim 2. The operation mode is an operation mode related to recognition of the traffic sign included in the image.
The vehicle control device according to any one of claims 1 to 3, wherein The operation mode related to the recognition of the traffic sign includes a first mode using reference information corresponding to each country or region, and a second mode using reference information in which information on a plurality of countries or regions is aggregated.
The vehicle control device according to claim 4. When it is determined that the vehicle is traveling in a predetermined country or region based on the user operation, the reference information corresponding to the predetermined country or region is used for the image in the first mode. Configured to recognize the traffic sign included,
The vehicle control device according to claim 5. When the country or region in which the vehicle is traveling is not determined based on at least the user operation, the second mode is used to display the image using reference information common to the plurality of countries or regions. Configured to recognize the traffic sign included,
The vehicle control device according to claim 5 or 6. When a target that does not correspond to the operation mode determined based on the user operation is detected in the vehicle, setting of the operation mode not based on the user operation and notification of the user to perform the user operation on the vehicle And non-use in the control of the vehicle of the extracted information of the traffic sign is further configured to perform at least one of:
The vehicle control device according to claim 1, wherein the vehicle control device is a vehicle control device.   A vehicle comprising the vehicle control device according to claim 1. A method executed by a vehicle control device that controls a vehicle,
Obtaining an image of the surrounding environment of the vehicle;
Obtaining information of user operations on the vehicle received in the vehicle;
Extracting information on traffic signs included in the image using an operation mode determined based on the user operation;
Controlling the vehicle to execute a predetermined process based on the information of the traffic sign;
A method comprising the steps of:

Images