JP2018097737A - Operation state controller and operation state control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation state controller capable of improving serviceability for an occupant.SOLUTION: The operation state controller includes an image processing part; a correlation processing and a mode processing part. The image processing part processes image data acquired by a camera including the outside of a vehicle in a visual field and acquires image information in a visual field of an occupant in the vehicle. The correlation processing part creates attribute information of the occupant by correlation processing using the image information. The mode processing part identifies an operation mode for the occupant on the basis of the attribute information.SELECTED DRAWING: Figure 1

Description

  Embodiments described herein relate generally to an operation state control device and an operation state control method.

  For example, as disclosed in Non-Patent Documents 1 and 2, a technique for detecting a person in an image taken by a camera is known. Patent Document 1 also discloses a technique for detecting a person on a wheelchair. In recent years, development of image processing technology has been remarkable in this way, and it has become possible to acquire information such as the state, attributes, and possessions of a person as well as a person from an image.

  It is expected that this type of technology is applied to safe driving of vehicles. For example, the technologies shown in Patent Documents 2 to 4 and the standards such as Non-Patent Document 3 are disclosed. Non-Patent Document 4 also introduces a technology that supports driving at night using an in-vehicle camera.

US Patent No. 7043084 International Publication No. 2016/121174 Pamphlet JP 2015-1333050 A International Publication No. 2014/054153 Pamphlet

Yuki Watanabe, “High-precision human detection technology using new features”, Toshiba Review Vol. 65 No. 4 (2010), p72-73 Van Kwok Vet, “Person tracking technology that can be tracked even if temporarily hidden from view”, Toshiba review Vol. 70 No. 6 (2015), p56-57 Ministry of Land, Infrastructure, Transport and Tourism, Automobile Bureau Press Release “Standards for“ camera monitoring systems ”to replace rearview mirrors, etc.” ”, [Online], [searched November 30, 2016], Internet, <URL: http: //www.mlit.go.jp/report/press/jidosha07_hh_000211.html> “Toyota | Safety Technology | Night View”, [online], [searched November 30, 2016], Internet, <URL: http://www.toyota.co.jp/eng/tech/safety/technology/ technology_file / active / night_view.html> “Survey research on communication policy evaluation and future image of ICT society after communication liberalization”, [online], [searched on November 30, 2016], Internet, <URL: http: //www.soumu. go.jp/johotsusintokei/linkdata/h27_01_houkoku.pdf>

  As mentioned above, in recent years, technologies for supporting vehicle driving with image recognition technology have been developed, but all of them are primarily for safe driving support and improve passenger comfort and hospitality. There are few technologies that focus on this. For example, a technology that can apply the knowledge obtained by image recognition to the field of so-called “automatic driving” is in the process of development.

  There are plans to provide vehicles to the market in the near future using automatic driving technology or automatic driving support technology. Along with this, services such as car sharing and car rental equipped with an automatic driving system will be started. However, in providing this type of service, there is no known technique that can reflect the attributes of the passenger (which may include the driver) in driving.

  For example, in car sharing / rental car, the vehicle desired by the driver cannot be used continuously. At best, it is possible to specify the vehicle type, which is not certain. In such an environment where a dedicated vehicle cannot be used, in order to use automatic driving functions such as route selection, warning about acceleration / deceleration, or lane change, not only safety but also consideration for passengers is more important Bring. Furthermore, as described in Non-Patent Document 5, it is possible to deal with a case where a passenger uses an actuator technology (such as a powered suit) that can assist athletic ability, which will appear in the near future as a walking aid. There is a need to.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a driving state control device and a driving state control method with improved serviceability for passengers.

  According to the embodiment, the driving state control device includes an image processing unit, a correlation processing unit, and a mode processing unit. The image processing unit processes image data acquired by a camera including the outside of the vehicle in the field of view, and acquires image information in the field of view of the passenger of the vehicle. The correlation processing unit generates passenger attribute information by correlation processing using image information. The mode processing unit specifies a driving mode for the passenger based on the attribute information.

FIG. 1 is a functional block diagram illustrating an example of an operation state control apparatus according to the embodiment. FIG. 2 is a diagram illustrating an installation example of the outside camera 31 and the inside camera 32. FIG. 3 is a diagram illustrating an installation example of the camera 31 outside the vehicle. FIG. 4 is a flowchart illustrating an example of a processing procedure in the ECU 20. FIG. 5 is a flowchart illustrating an example of a processing procedure in the ECU 20. FIG. 6 is a diagram showing the correlation between the detection result of each sensor of FIG. 1 and the estimation result of the passenger attribute in time series. FIG. 7 is a diagram illustrating an example of a prompt message displayed on the display 40. FIG. 8 is a diagram illustrating another example of the prompt message displayed on the display 40.

  FIG. 1 is a functional block diagram illustrating an example of an operation state control apparatus according to the embodiment. Hereinafter, an electronic control unit (ECU) denoted by reference numeral 20 in FIG. 1 will be described as an example of an operation state control device.

  The ECU 20 is connected to an automatic driving system 10 for the vehicle and a sensor system 30 provided in the vehicle. In addition, the vehicle includes a display 40, a speaker 50, and a lighting 70. Further, the vehicle may include a rear door lock control unit 60 that controls locking / unlocking of the rear door.

  The display device 40 is installed between a driver seat and a passenger seat, for example, and is a user interface such as a touch panel LCD (Liquid Crystal Display). Various messages are given to the user (passenger) by a GUI (Graphical User Interface) displayed on the display 40 or a voice message from the speaker 50. In addition, the user's instruction given from the GUI of the display 40 is passed to the automatic driving system 10 via the ECU 20.

  The automatic driving system 10 is provided as an opposing device of the ECU 20 and is responsible for control related to automatic driving of the vehicle. For example, the automatic driving system 10 controls driving of the vehicle according to the driving mode specified by the ECU 20.

  The sensor system 30 includes an outside camera 31, an inside camera 32, a door unlock sensor 33, a door opening / closing sensor 34, a beacon receiver 35, a microphone 36, and a seat pressure sensor 37.

  The vehicle exterior camera 31 is a camera that includes the outside of the vehicle in the field of view, and captures the inside of the field of view to acquire image data. That is, the camera 31 outside the vehicle is a camera that captures the periphery of the vehicle, as long as it can capture at least a passenger approaching the door of the vehicle. It is also possible to share the camera monitoring system shown in Non-Patent Document 3 and the camera 31 outside the vehicle. In particular, as long as it is of a type interlocking with the illumination 70, a good image can be obtained even in the dark such as at night, and a camera having a night vision function may be used.

  The in-vehicle camera 32 is an image sensor that includes the inside of the vehicle (inside the vehicle) in the field of view, and captures the inside of the field of view to acquire image data. That is, the in-vehicle camera 32 may be any camera that captures the interior of the vehicle and can capture at least the seating situation of each seat in the vehicle. The number of in-vehicle cameras 32 is not limited. For example, if the rear seat state can be photographed with a camera installed at the front of a small vehicle, there is no need to install a separate camera at the rear. The same applies to the camera 31 outside the vehicle.

  2 and 3 are diagrams showing examples of installation of the outside camera 31 and the inside camera 32. For example, the camera 31 outside the vehicle can be attached to the position of the rearview mirror and used together with the camera monitoring system. The in-vehicle camera 32 is attached, for example, at a position looking down from the ceiling of the vehicle. A wide viewing angle camera such as a fisheye lens may be used.

  Returning to FIG. 1, the description will be continued. The door lock release sensor 33 detects the door lock state of the vehicle. The door open / close sensor 34 is installed at each door of the vehicle and detects the open / closed state of each door. Note that the door opening / closing sensor 34 may be omitted if the opening / closing state of each door can be determined from the image data taken by the outside camera 31 and the inside camera 32 and the information can be passed to the correlation processing unit 26.

  The beacon receiver 35 is loaded on the vehicle and receives a beacon signal from the beacon transmitter 80. The beacon transmitter 80 can be attached to a walking aid such as a wheelchair or a crutch, and wirelessly transmits a beacon signal including information reflecting the attribute of the user. In the embodiment, the beacon transmitter 80 is carried by the passenger himself and transmits different signals according to the attributes of the passenger. As the beacon transmitter 80, a short-range wireless transmitter such as BLE (Bluetooth (registered trademark) Low Energy) can be used.

  The presence of the walking aid can be directly confirmed based on the received beacon signal. Or you may determine the presence or absence of a walk aid by matching a beacon signal and the person detected with the camera 31 outside the vehicle.

  The microphone 36 is loaded on the vehicle, captures sounds and sounds inside and outside the vehicle, and outputs a sound signal. The seat pressure sensor 37 is a pressure sensor provided for each seat of the vehicle, and detects the presence or absence of a seat-like occupant based on the detected pressure value. Information regarding the presence or absence of a passenger on the seat is passed to the correlation processing unit 26.

  Next, the ECU 20 includes an image processing unit 21, a door lock release detection processing unit 22, a door opening / closing detection processing unit 23, a beacon reception processing unit 24, an audio processing unit 25, a correlation processing unit 26, and a user as processing functions according to the embodiment. An interface unit 27, an operation mode selection processing unit 28, and a baggage carry-in support processing unit 29 are provided.

  The image processing unit 21 processes the image data acquired by the vehicle camera 31 to acquire image information mainly outside the vehicle passengers. Further, the image processing unit 21 processes the image data acquired by the in-vehicle camera 32 to acquire in-vehicle image information. In particular, the image processing unit 21 determines the presence or absence of a child seat based on image information in the vehicle, and passes the determination result to the correlation processing unit 26. It is technically possible to determine the presence or absence of an object such as a child seat by utilizing an existing image processing library or the like.

  The door lock release detection processing unit 22 processes detection information from the door lock release sensor 33. The door opening / closing detection processing unit 23 processes detection information from the door opening / closing sensor 34.

  The beacon reception processing unit 24 processes the beacon signal received by the beacon receiver 35 and acquires beacon information. This beacon information is passed to the correlation processing unit 26. The audio processing unit 25 processes the audio signal output from the microphone 36 and acquires audio information.

  The correlation processing unit 26 generates attribute information of a vehicle occupant by correlation processing using image information obtained by imaging at least the outside of the vehicle output from the image processing unit 21. The correlation processing can be easily understood by those skilled in the art by applying a technique such as sensor fusion that integrally processes a plurality of sensing data.

  Passenger attribute information is information indicating, for example, age, sex, presence / absence of walking aids (wheelchairs, crutches, etc.) or whether or not a pet is present. In addition, various attribute information can be considered according to the scene.

  In particular, the correlation processing unit 26 generates passenger attribute information through correlation processing between the passenger's image information and the in-vehicle image information. Moreover, the correlation process part 26 produces | generates a passenger's attribute information by the correlation process using beacon information.

  The correlation processing unit 26 can also generate passenger attribute information by correlation processing using audio information. In particular, the correlation processing unit 26 may start the correlation processing when door unlocking is detected.

  In addition, the correlation processing unit 26 generates attribute information for each occupant who has entered the vehicle from each door based on a time-series change between the open state and the closed state of the vehicle door. In addition, the correlation processing unit 26 generates attribute information of the passenger of the seat to which the seat pressure sensor 37 is sensitive.

  In particular, the correlation processing unit 26 generates attribute information related to whether or not the passenger's walking aid is used. Further, the correlation processing unit 26 reflects the information related to the presence / absence of the child seat in the correlation processing to generate passenger attribute information.

  Next, the user interface unit 27 is connected to the display 40 and the speaker 50 in the vehicle, and serves as an intermediary between the ECU 20 and the user to exchange various information. For example, the user interface unit 27 presents a plurality of operation modes extracted by the operation mode selection processing unit 28 to the user via the display 40 and the speaker 50. The operation mode designated using the touch panel of the display device 40 is input to the ECU 20 via the user interface unit 27.

  The driving mode selection processing unit 28 specifies the driving mode for the passenger based on the passenger attribute information generated by the correlation processing unit 26. In particular, the operation mode selection processing unit 28 extracts a plurality of operation modes based on the passenger's attribute information, and notifies the user of the plurality of operation modes via the display 40 or the speaker 50 (user interface). When any of the plurality of operation modes displayed on the display device 40 is designated by the user, the operation mode selection processing unit 28 specifies the designated operation mode as the operation mode for the passenger.

  In addition, the operation mode selection processing unit 28 notifies the automatic operation system 10 of the specified operation mode. More specifically, the driving mode selection processing unit 28 notifies the automatic driving system 10 of a combination of vehicle control parameters that reflect the specified driving mode. The vehicle control parameter may be, for example, at least one of an acceleration / deceleration range, a speed range, and a movement route, and various other parameters can be considered.

  The luggage carry-in support processor 29 controls the rear door lock controller 60 to open and close the door of the luggage compartment of the vehicle. For example, when it is determined from the video of the camera 31 outside the vehicle that the passenger is accompanied by a large baggage, a wheelchair, or the like, the baggage carry-in support processing unit 29 opens the trunk and the rear hatch to support the baggage storage.

  The ECU 20 estimates an occupant's attribute based on the above function, and notifies the automatic driving system 10 of the result, thereby instructing an appropriate driving mode. For example, if it is determined from the attribute information that an injured person has appeared, a suggestion message for a driving mode such as “friendly” driving that suppresses acceleration / deceleration is sent to the driver and passenger of the vehicle. Display on the display 40.

  The image processing unit 21, door lock release detection processing unit 22, door opening / closing detection processing unit 23, beacon reception processing unit 24, voice processing unit 25, correlation processing unit 26, user interface unit 27, operation mode selection processing shown in FIG. Each function of the unit 28 and the baggage carry-in support processing unit 29 can be stored in a memory (not shown) or the like in the form of a program executable by the processor, for example. That is, ECU20 is a computer which implement | achieves the function corresponding to each program by reading a program from memory and executing it.

  The processor is a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a programmable logic device (for example, a simple programmable logic device: Hardware resources such as SPLD, complex programmable logic device (CPLD), and field programmable gate array (FPGA). Next, the operation of the above configuration will be described.

  4 and 5 are flowcharts illustrating an example of a processing procedure in the ECU 20. In FIG. 4, when the door lock of the vehicle is released, the door lock release sensor 33 detects that and power supply to the ECU 20 is started (step S1). Immediately thereafter, the ECU 20 is activated, and at the same time, the outside camera 31, the inside camera 32, the door opening / closing sensor 34, the seat pressure sensor 37, and the like are also activated (step S2).

  Subsequently, the outside camera 31 captures a moving image of a person approaching the vehicle and outputs image data to the image processing unit 21 of the ECU 20 (step S3). The image processing unit 21 processes the image data to generate image information, and estimates the presence / absence of a walking aid such as a wheelchair of an approaching person and attribute information such as age from the image information. The result is output to the correlation processing unit 26 (step SS5).

  Here, when the presence of the load is estimated from the video of the camera 31 outside the vehicle, the correlation processing unit 26 gives an instruction to the load carry-in support processing unit 29 to open the trunk and the rear hatch, thereby supporting the load storage.

  In parallel, the in-vehicle camera 32 also continuously captures images in the vehicle after activation (step S4). Image data photographed by the in-vehicle camera 32 is processed by the image processing unit 21 to generate image information. Based on the image information, the image processing unit 21 determines whether there is a child seat or the like as a state before the passenger is seated, and outputs the result (step S6). The presence or absence of the child seat is notified to the correlation processing unit 26 (step S11).

  Next, when the vehicle door is opened (step S7), the door open / close sensor 34 reacts (step S8), and the door open / close detection processing unit 23 outputs the door open state to the correlation processing unit 26 (step S9).

  When the correlation processing unit 26 receives the notification of the door open state from the door opening / closing detection processing unit 23, the correlation processing unit 26 confirms the presence / absence of the passenger and the age estimation result from the image processing unit 21. The in-vehicle camera 31 and the in-vehicle camera 32 may be linked, and a moving image of a person boarding may be tracked and photographed by the out-of-vehicle camera 31 and the in-vehicle camera 32. And the correlation process part 26 determines the presence or absence of the passenger with respect to the said seat about the seat of the door vicinity which became the open state, and estimates a passenger's age (step S10).

  Subsequently, when the door is closed, the door opening / closing sensor 34 reacts, and the door opening / closing detection processing unit 23 detects the closed state of the door (step S12) and notifies the correlation processing unit 26 of this (step S21). .

  The correlation processing unit 26 detects that the in-vehicle camera 32 detects the seating of the passenger in the driver's seat after a predetermined time has elapsed from the notification of the door closed state from the door opening / closing detection processing unit 23, or the camera outside the vehicle. When no other occupant candidates are observed in 31, the presence / absence of passengers in each seat and the age estimation are determined (step S <b> 22). The confirmed attribute of the passenger is notified to the operation mode selection processing unit 28.

  Then, the driving mode selection processing unit 28 selects the driving mode of the vehicle based on the rider's attributes, informs the user of the content via the display 40 and the speaker 50, and prompts the passenger to select (step S23). ). When the driving mode is designated by the passenger (step S24), the driving mode selection processing unit 28 notifies the selection result to the automatic driving system 10 (step S25).

  FIG. 6 is a diagram showing the correlation between the detection result of each sensor of FIG. 1 and the estimation result of the passenger attribute in time series. FIG. 6 shows processing at the left rear door of the vehicle. That is, the attribute of the passenger can be estimated on a door basis.

  As shown in FIG. 6 (a), for example, there are five judgment items: door lock release, exterior camera (left), door open (rear left), in-vehicle camera (left rear view), and all doors closed. (# 0 to # 4). When the door lock release is detected for the first time (）), it is assumed that the outside camera and the in-vehicle camera are activated and a passenger is detected in the vicinity of the left rear door (◯). Then, as shown in FIG. 6B, the presence / absence of a walking aid is determined from the image data of the outside camera, and the presence / absence of the child seat is determined from the image data of the in-vehicle camera.

  When time elapses in FIG. 6A and the left rear door is opened (▽), the presence / absence of the passenger and the age of the passenger are estimated in FIG. 6B. Further, when time elapses in FIG. 6A and all doors are closed (all doors closed (▽)), the presence / absence and age of the passenger and the presence / absence of the child seat are determined for the left rear seat.

The procedure when a person gets on the vehicle can be disassembled as (1) to (5) below, for example.
(1) Approach the vehicle and move in front of the door. (2) Open the vehicle door. (3) Board the vehicle and take a seat. (4) Close the vehicle door. (5) Start driving the vehicle.

  At this time, when the passenger uses a walking aid such as a wheelchair or a crutch, a characteristic image and a movement pattern are observed in the outside camera 31. In addition, the act of removing the walking aid after opening the door will be observed.

  On the other hand, when there is a child passenger, it is possible to make a judgment based on the height from the photographed image when approaching the vehicle and in front of the door. Furthermore, when a seating aid such as a baby seat or a child seat is confirmed in the image taken by the in-vehicle camera 32, it can be determined that the infant is included in the passenger. Furthermore, it is possible to determine pet riding with the same determination.

  The attribute information determined for each door includes items shown in the following table, for example.

  When the door opening / closing detection processing unit 23 notifies that all the doors are closed, the correlation processing unit 26 generates the attribute information shown in Table 1 for each door. For example, a small item such as the presence or absence of a passenger can be estimated from the detection result of the door lock release sensor 33. If the passenger presence is 1, the absence is defined as 0. The age and the presence / absence of a walking aid can be estimated from the image data of the camera outside the vehicle. From the image data of the in-vehicle camera, it is possible to estimate the age and the presence or absence of a child seat. The age can also be estimated by analyzing the sound from the microphone.

  The attribute determination value Vm can be calculated using the numerical value (0/1) shown in the remarks column of Table 1.

  For example, consider that an adult without a walking aid such as a walking stick approaches the vehicle. In this case, based on the image information from the camera 31 outside the vehicle, the image processing unit 21 determines from the height that the approaching person is an adult. As a result, 0 is generated as the age estimation value, and 0 is generated without the walking aid. It is determined from the image information from the in-vehicle camera 32 that the person is an adult, and 0 is generated as an age estimated value. The correlation processing unit 26 generates 0 as a value if there is no child seat on the seat.

  In parallel, the voice processing unit 25 discriminates individual speakers based on voice data input from the microphone 36, and performs age estimation. Here, if the passenger is determined to be an adult, 0 is generated as the value of the attribute information.

  Further, when the door is closed, the presence / absence of a passenger from the door is determined, and a determination value 1 is generated as the presence of a passenger. As a result of the above process, item 1 = 1 and items 2 to 6 = 0 in Table 1 are obtained, and 0 is obtained as the attribute determination value Vm.

  In another case, suppose that a passenger using Matsubae approaches. In this case, a crutch is detected from the image data from the camera 31 outside the vehicle, and a determination value 1 is generated as item 3 in Table 1. Each item other than item 3 generates the same determination value as when an adult approaches, and the attribute determination value Vm calculated in this case is 1.

  Further, as another case, consider a case where there is a child passenger. In this case, in item 2, item 4, and item 6 of Table 1, 1 is generated, and 3 is generated as the attribute determination value Vm.

  The attribute determination value Vm is notified from the correlation processing unit 26 to the operation mode selection processing unit 28. When the attribute determination value Vm is not 0 (Vm> 0), the operation mode selection processing unit 28 determines that it is recommended to use the “friendly operation” mode for children and injured persons. Then, the operation mode selection processing unit 28 displays a message as shown in FIG. 7 or FIG.

  FIG. 7 shows an example of a message notifying the passenger that driving with less burden is possible, and prompts the user to select “Yes” or “No” from this screen. The passenger may be notified of the same content through the speaker 50 by voice. In FIG. 8, either (friendly operation) or (normal operation) can be selected.

When the passenger selects “Yes” from the screen of FIG. 7, the correlation processing unit 26 instructs the automatic driving system 10 to perform vehicle control and route selection for driving that reduces the burden on the passenger. That is, the automatic driving system 10 is notified of the optimal combination of vehicle control parameters corresponding to the (friendly driving mode).
Next, some examples according to this embodiment will be described.

[First embodiment]
An embodiment assuming a rental car / car sharing using a vehicle with an automatic driving function or an automatic driving support function will be described.

  When the passenger releases the door lock, the system of the present invention starts operation, and the ECU 20 starts collecting the attributes of the passenger. When it is determined that the passenger does not include an injured person or the like, no special presentation is made from the vehicle side, and the vehicle is controlled as the normal operation mode unless there is an operation / instruction of the passenger.

  When it is estimated that there are passengers using walking aids such as crutches, wheelchairs, canes, etc., automatic driving such as lane change control with a lower acceleration rate or lower lateral acceleration, etc. Or provide driving assistance. In addition, the route to the destination is not time-prioritized, and for example, a route with a small stop by a signal, a route with a small traffic volume, or a route with a small turn is selected and presented.

  Also, if it is estimated that there is a passenger using a walking aid, the passenger will be asked to confirm the unlocking response by voice, etc. to lock the trunk or the rear door when storing the walking aid. You may cancel.

  When it is estimated that there is a child passenger, route selection is performed with time priority. Then, by providing driving assistance similar to the above, automatic driving or driving assistance is presented so that the child can be safely spent in the car while shortening the boarding time so as not to fool.

  As a result, in a vehicle unfamiliar with the driver, the driver can be presented with an optimal driving state and can easily perform a safe and comfortable driving for the passenger without performing complicated operations. . For example, depending on the condition of the passenger, such as an injured person, it may take longer than usual, reducing the acceleration / deceleration rate and minimizing lateral shaking, etc. It is important to secure the surface.

  Systems mounted on vehicles in recent years have become more sophisticated and complex. In particular, in an unfamiliar vehicle that is not used everyday, it is difficult for the passenger to operate the vehicle-mounted system. As a countermeasure, use a screen display or audio output on the vehicle side to present to the passenger the choices of the functions provided in the vehicle, prompt the passenger to make a choice, or ask / desir whether the passenger is injured. A method for confirming the contents to be controlled and controlling the vehicle settings accordingly can be considered.

  However, presenting options / replying various questions tends to result in time consuming from boarding to departure. Furthermore, the passenger is stressed due to the occurrence of an erroneous answer to the question.

  On the other hand, in the embodiment, the vehicle outside camera 31 is used to check the passenger outside the vehicle, that is, the state before boarding, and the attributes of the passenger are checked by the sensor system such as the vehicle camera 32 and the microphone 36. I made it. Based on the result, a recommended driving mode is selected, and optimal vehicle control parameter combination candidates are extracted and presented to the passenger. Then, the result of selection based on the intention of the passenger is reflected in the control of the vehicle. This makes it possible to realize a vehicle system that provides comfort to all passengers.

  Even in a temporary use environment such as car sharing / rental car, it is possible to obtain the best attributes from the menu that reflects the attributes by obtaining the attributes of the passengers. The operation mode can be determined. Therefore, the present invention provides means for a system that can be easily used even by non-experts in a vehicle control system that is different for each vehicle type and that is more sophisticated and complicated.

[Second Embodiment]
For example, the above-described embodiment can be applied to a taxi for automatic driving. During the operation of the taxi for automatic driving, the ECU 20 provided in the taxi may be kept in an operating state at all times.
A person estimated to be a passenger is discovered by the camera 31 outside the vehicle, and its attributes are collected. When the passenger is approaching and appears in the car, it is determined from the collected attributes whether the passenger is injured or the like, and the same message as in the first embodiment is presented to the passenger. .

  By doing in this way, in the taxi of an automatic driving | operation, confirmation of the question and answer to a passenger etc. can be made unnecessary. In addition, the optimum driving state can be presented to the passenger, and it is possible to quickly start from the boarding and to realize safe and comfortable driving for the passenger.

  When the technology according to the embodiment is not applied, a detailed question and answer to the passenger is necessary due to the nature of the system in order to perform driving in accordance with the passenger's condition in the automatic driving taxi. Further, it is necessary to control the operation mode based on a vehicle determination function or a remote operator's determination. This mechanism is used after the passenger gets on the vehicle, and takes time to start.

  On the other hand, according to the embodiment, since the passenger only has to select the driving mode he desires from the presented message, the passenger can enjoy quick start and comfortable driving.

[Third embodiment]
An embodiment applicable to either a rental car / car sharing using a vehicle with an automatic driving function or an automatic driving support function, or a taxi will be described.

  For passengers who use actuator technology to assist athletic ability as walking aids, or who do not wear (use) walking aids even though they normally use walking aids Therefore, it may be difficult to determine the attribute only from the image data captured by the camera.

  In this case, the beacon transmitter 80 is attached to the walking aid used by the passenger. Alternatively, the passenger himself / herself carries a beacon transmitter 80 that emits a different signal according to attributes such as the age of the passenger and the presence or absence of injury. Then, the attribute of the passenger can be estimated by receiving the signal on the vehicle side and associating it with the video from the camera. As a result, similar to the first embodiment and the second embodiment, it is possible to present the optimum driving mode to the occupant and to realize quick start and safe and comfortable driving.

  As described above, according to the embodiment, the out-of-vehicle camera 31 is actively used, and the attribute estimation is started before the person boarding. In addition, images outside the vehicle are also used to determine whether or not a wheelchair or Matsubae is used. Accordingly, it is possible to acquire detailed attributes that are difficult to detect with the in-vehicle camera 32 alone. Furthermore, it is possible to accurately acquire the attributes of the occupant, which was difficult only with the in-vehicle camera 32, and it is possible to realize a more appropriate driving mode proposal.

  Further, according to the embodiment, the passenger can determine the attribute of the passenger from the vehicle side and propose the function provided in the vehicle without knowing the function provided in the vehicle. Therefore, it is possible to provide a more comfortable moving means without taking time and effort to the passenger.

  Further, by starting the sensor system 30 and the ECU 20 from the time when the door lock is released, it becomes possible to start collecting the attributes of the passengers before boarding. Accordingly, it is possible to propose an appropriate driving mode and select by the passenger during the time until it is possible to leave, and to provide a passenger-free use environment.

  Therefore, according to the embodiment, it is possible to provide a driving state control device and a driving state control method that improve serviceability for passengers.

  The present invention is not limited to the above embodiment. For example, an automatic driving system can be read as an automatic driving support system. Further, if there is a camera having a performance capable of acquiring detailed attribute information as much as necessary, an intended process may be performed based only on the attribute information acquired by the camera.

  A program for realizing each device and system described above is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system and executed, thereby executing an execution process. May be. Here, the “computer system” may include an OS and hardware such as peripheral devices.

  Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 10 ... Automatic driving system, 20 ... ECU, 21 ... Image processing part, 22 ... Door lock cancellation | release detection processing part, 23 ... Door opening / closing detection processing part, 24 ... Beacon reception processing part, 25 ... Voice processing part, 26 ... Correlation process 27: User interface unit, 28: Operation mode selection processing unit, 29 ... Luggage carry-in support processing unit, 30 ... Sensor system, 31 ... Outside camera, 32 ... In-vehicle camera, 33 ... Door lock release sensor, 34 ... Door open / close Sensor, 35 ... Beacon receiver, 36 ... Microphone, 37 ... Seat pressure sensor, 40 ... Display, 50 ... Speaker, 60 ... Rear door lock control unit, 70 ... Illumination, 80 ... Beacon transmitter.

Claims (14)

An image processing unit that processes image data acquired by a camera including the outside of the vehicle in a field of view, and acquires image information in the field of view of a passenger of the vehicle;
A correlation processing unit that generates attribute information of the passenger by correlation processing using the image information;
A driving state control device comprising: a mode processing unit that specifies a driving mode for the passenger based on the attribute information. The image processing unit acquires image information in the vehicle by processing image data acquired by an image sensor including the inside of the vehicle in the field of view,
The driving state control device according to claim 1, wherein the correlation processing unit generates attribute information of the passenger by correlation processing between the image information of the passenger and image information in the vehicle. And a beacon reception processing unit for acquiring beacon information by processing a beacon signal received by a beacon receiver loaded on the vehicle,
The driving state control device according to claim 2, wherein the correlation processing unit generates attribute information of the occupant by the correlation processing using the beacon information. And a voice processing unit that acquires voice information by processing a voice signal output from a microphone loaded on the vehicle,
The driving state control device according to claim 2, wherein the correlation processing unit generates attribute information of the occupant through the correlation processing using the voice information. Furthermore, it comprises a door lock release detection processing unit for processing detection information from a lock sensor that detects the door lock state of the vehicle,
The operation state control device according to claim 2, wherein the correlation processing unit starts the correlation processing when door lock release is detected by the lock sensor. And a door opening / closing detection processing unit for processing detection information from a door sensor for detecting an opening / closing state of the vehicle door,
The driving state according to claim 2, wherein the correlation processing unit generates attribute information of a passenger from the door based on a time-series change of the door open state and the closed state detected by the door sensor. Control device.   The driving state control device according to claim 2, wherein the correlation processing unit generates attribute information of a seat occupant to which a pressure sensor that detects a pressure on a seat of the vehicle is sensitive. The mode processing unit
Extracting a plurality of operation modes based on the attribute information,
Presenting the extracted plurality of operation modes to a user interface;
The driving | running state control apparatus of any one of Claim 1 thru | or 7 which specifies the driving | running mode designated in the said user interface as a driving | operation mode for the said passenger.   The driving state control device according to claim 8, wherein the mode processing unit notifies the specified driving mode to an automatic driving system of the vehicle.   The driving state control device according to claim 8, wherein the mode processing unit notifies the vehicle automatic driving system of a combination of vehicle control parameters reflecting the identified driving mode.   The driving state control device according to claim 10, wherein the vehicle control parameter is at least one of an acceleration / deceleration range, a speed range, and a movement route.   The driving state control device according to claim 1, wherein the correlation processing unit generates attribute information related to whether or not the passenger's walking aid is used. The image processing unit passes information on the presence or absence of a child seat acquired based on the image information in the vehicle to the correlation processing unit,
The driving state control device according to claim 2, wherein the correlation processing unit generates attribute information of the occupant by reflecting information on the presence or absence of the child seat in the correlation processing. In a driving state control method applicable to a computer mounted on a vehicle,
The computer processes image data acquired by a camera including the outside of the vehicle in a field of view to acquire image information in the field of view of a passenger of the vehicle;
The computer generates attribute information of the passenger by a correlation process using the image information;
A driving state control method comprising: the computer specifying a driving mode for the occupant based on the attribute information.