US10706270B2 - Information provision device, and moving body - Google Patents

Information provision device, and moving body Download PDF

Info

Publication number
US10706270B2
US10706270B2 US16/317,144 US201716317144A US10706270B2 US 10706270 B2 US10706270 B2 US 10706270B2 US 201716317144 A US201716317144 A US 201716317144A US 10706270 B2 US10706270 B2 US 10706270B2
Authority
US
United States
Prior art keywords
user
moving body
unit
test
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US16/317,144
Other languages
English (en)
Other versions
US20190294867A1 (en
Inventor
Shinichiro Goto
Tomoko Shintani
Hiromitsu Yuhara
Eisuke Soma
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Honda Motor Co Ltd
Original Assignee
Honda Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honda Motor Co Ltd filed Critical Honda Motor Co Ltd
Assigned to HONDA MOTOR CO., LTD. reassignment HONDA MOTOR CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: YUHARA, HIROMITSU, SHINTANI, TOMOKO, SOMA, Eisuke, GOTO, SHINICHIRO
Publication of US20190294867A1 publication Critical patent/US20190294867A1/en
Application granted granted Critical
Publication of US10706270B2 publication Critical patent/US10706270B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G06K9/00302—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G06K9/00845—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/07—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail characterised by the inclusion of specific contents
    • H04L51/08—Annexed information, e.g. attachments
    • H04L51/22—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/42—Mailbox-related aspects, e.g. synchronisation of mailboxes

Definitions

  • the present invention relates to an information provision device which provides information on a moving body to a potential purchaser, a user, or the like, who is considering purchasing the moving body, and the moving body.
  • Patent Literature 1 transmits, to a mobile terminal of the salesperson, data on customer service conditions (response speed, explanation level, and requirements), which are desired by the potential purchaser, entered into a terminal installed at an entrance of the dealer, data indicating the customer information and recommended products based on the past purchase history and preference of the potential purchaser, and other data.
  • customer service conditions response speed, explanation level, and requirements
  • Patent Literature 1 discloses a configuration of transmitting an email to the customer who purchased a product for expressing appreciation and transmitting an email on product information to the customer who did not purchase the product.
  • Patent Literature 1 Japanese Patent Laid-Open No. 2005-128928
  • test drive increases the satisfaction level of the potential purchaser who is considering purchasing the moving body.
  • the satisfaction level of a user of the moving body (owner or the like of the moving body) other than the potential purchaser can be increased by causing the user of the moving body to recognize the use situation of the moving body in which the user has had a good impression.
  • An object of the present invention which has been made in view of such a background, is to provide an information provision device and a moving body capable of more effectively increasing the satisfaction level of the user of the moving body.
  • An information provision device of the present invention is
  • an information provision device which functions in a moving body, the information provision device comprising:
  • a user state detection unit which detects a state of a user of the moving body, the user being located in a predetermined area containing the moving body
  • an imaging unit which images inside the predetermined area
  • an emotion estimation unit which estimates an emotion of the user based on a detection result of the user state detection unit
  • an image transmission unit which causes the imaging unit to image the user when an emotion level of the user estimated by the emotion estimation unit increases, and transmits an image data of the imaged user to a communication terminal of the user when it is estimated or detected that the user has left the predetermined area containing the moving body.
  • the user state detection unit detects the state of the user, and the emotion estimation unit estimates the emotion of the user. Then, in response to an increase in the emotion level of the user estimated by the emotion estimation unit, the image transmission unit causes the imaging unit to image the user, and transmits the image data of the imaged user to the communication terminal of the user when it is estimated or detected that the user has left the predetermined area containing the moving body.
  • the user can view the image which was imaged when the user felt emotionally excited while the user was driving the moving body or operating a door of the moving body through the communication terminal of the user after leaving the moving body. This viewing causes the user to remember the emotionally excited scenes and thus can increase the satisfaction level of the user of the moving body.
  • the information provision device comprises an information provision unit which provides the user with information encouraging use of a specific function possessed by the moving body, wherein
  • the image transmission unit causes the imaging unit to image the user and transmits the image data of the imaged user to the communication terminal of the user.
  • This configuration reminds the user of the use situation of the specific function raising the emotion of the user thereby to re-recognize the usefulness of the specific function and to increase the satisfaction level of the user of the moving body.
  • the image transmission unit transmits the image data of the user together with data indicating the moving body as a sender to the communication terminal of the user.
  • This configuration allows the user to receive the image data directly transmitted from the moving body, and thus to feel close to the moving body.
  • the information provision device comprises a moving body image acquisition unit which acquires data of a moving body image showing the moving body, wherein
  • the image transmission unit transmits the image data of the user together with data of the moving body image to the communication terminal of the user.
  • This configuration allows the user to intuitively recognize, visually from the image of the moving body, that the image data of the user is directly transmitted from the moving body and thus to feel close to the moving body.
  • the information provision device comprises a test drive recognition unit which recognizes that the user is test-driving the moving body, wherein
  • the image transmission unit causes the imaging unit to image the user while the test drive recognition unit recognizes that the user is test-driving the moving body, and when a predetermined time has elapsed since a point of time when a state in which the test drive recognition unit recognizes that the user is test-driving the moving body is switched to a state in which the test drive recognition unit does not recognize that the user is test-driving the moving body, the image transmission unit transmits the image data of the user to the communication terminal of the user.
  • This configuration reminds the user of the raised emotions occurring to the user by transmitting the image data of the user after a certain interval, and thus can maintain the satisfaction level which the user feels to the moving body.
  • a moving body of the present invention comprises any one of the above described information provision devices.
  • the moving body comprising the information provision device can exert the same operational effects as the above described information provision devices and thereby can effectively increase the satisfaction level of the user to the moving body.
  • FIG. 1 is an explanatory view of a four-wheeled vehicle (moving body) having an information provision device mounted thereon.
  • FIG. 2 is a control block diagram of the information provision device.
  • FIG. 3 is an explanatory view of an emotion engine.
  • FIG. 4 is a first flowchart of the information provision device.
  • FIG. 5 is a second flowchart of the information provision device.
  • FIGS. 6A and 6B are a first explanatory view illustrating an operating situation of the information provision device during test drive.
  • FIG. 6A illustrates a situation where the vehicle is being stopped
  • FIG. 6B illustrates a situation where the vehicle is traveling.
  • FIGS. 7A and 7B are an explanatory view where the information provision device notifies the test driver of vehicle emotions during test drive.
  • FIG. 7A illustrates an example of an emotion notification while the vehicle is traveling
  • FIG. 7B illustrates an example of an emotion notification while the test drive is finished.
  • FIG. 8 is an explanatory view of a situation where a thank-you email is transmitted after the test drive is finished.
  • FIGS. 1 to 8 embodiments of the present invention will be described with reference to FIGS. 1 to 8 .
  • an information provision device 60 is mounted on a four-wheeled vehicle 1 (which corresponds to a moving body of the present invention and hereinafter referred to as a vehicle 1 ).
  • a vehicle 1 which corresponds to a moving body of the present invention and hereinafter referred to as a vehicle 1 .
  • the information provision device 60 of the present embodiment is a vehicle-mounted type, but the information provision device of the present invention may be a portable type which can be carried by a user and can be brought into a vehicle for use.
  • FIG. 1 illustrates a situation where a test driver P 1 (which corresponds to the user of the present invention), who is a potential purchaser and is considering purchasing the same type of the vehicle 1 , sits on a driver's seat 5 R of the vehicle 1 , and a salesperson P 2 in charge of sales of the vehicle sits on a passenger seat 5 L.
  • the salesperson P 2 carries a business terminal 9 (such as a tablet and a smartphone) and provides a customer service to the test driver P 1 through the business terminal 9 .
  • a business terminal 9 such as a tablet and a smartphone
  • the vehicle 1 comprises a front camera 11 , a front radar 12 , a rear camera 13 , and a rear radar 14 as the configuration for recognizing the surrounding situation of the vehicle 1 , and comprises a navigation device 40 having a global positioning system (GPS) unit 41 and a map database (DB) 42 to be detailed later as the configuration for acquiring route information to the destination.
  • GPS global positioning system
  • DB map database
  • the vehicle 1 comprises a driver's seat camera 20 (which corresponds an imaging unit and a user state detection unit of the present invention) which captures an image of at least a face portion of the driver and a driver's seat microphone 21 (which corresponds a user state detection unit of the present invention) which enters driver's voice.
  • a driver's seat camera 20 which corresponds an imaging unit and a user state detection unit of the present invention
  • a driver's seat microphone 21 which corresponds a user state detection unit of the present invention
  • the vehicle 1 comprises: a steering angle sensor 25 which detects a steering angle of a steering wheel; a torque sensor 26 which detects a steering torque applied to the steering wheel; an accelerator pedal sensor 24 which detects a depression amount of an accelerator pedal; a brake pedal sensor 27 which detects a depression amount of a brake pedal; and a sports mode switch 28 which accepts setting and releasing of a sports mode (a driving mode with improved response, such as control of great increase in driving force relative to accelerator operation).
  • a steering angle sensor 25 which detects a steering angle of a steering wheel
  • a torque sensor 26 which detects a steering torque applied to the steering wheel
  • an accelerator pedal sensor 24 which detects a depression amount of an accelerator pedal
  • a brake pedal sensor 27 which detects a depression amount of a brake pedal
  • a sports mode switch 28 which accepts setting and releasing of a sports mode (a driving mode with improved response, such as control of great increase in driving force relative to accelerator operation).
  • the vehicle 1 comprises door sensors 30 R, 30 L, 31 R, 31 L, and 32 which detect opening and closing of front doors 2 R, and 2 L, rear doors 3 R, and 3 L, and back door 4 ; and seat sensors 33 R, 33 L, 34 R, and 34 L which detect the presence or absence of a seated occupant on each of a driver's seat 5 R, a passenger seat 5 L, and a rear seat 6 .
  • the vehicle 1 comprises a speaker 22 (which corresponds a notification unit of the present invention) and a display unit 23 (which corresponds the notification unit of the present invention) for notifying a passenger.
  • the vehicle 1 comprises a control unit 50 which controls the overall operation of the vehicle 1 .
  • the control unit 50 is an electronic circuit unit including a CPU, a memory, various interface circuits, and the like, which are not illustrated.
  • the control unit 50 comprises a communication device 51 and an information provision device 60 .
  • the control unit 50 receives data imaged by the front camera 11 , the rear camera 13 , and the driver's seat camera 20 , and data of an object position detected by the front radar 12 and the rear radar 14 .
  • the control unit 50 receives signals detected by the steering angle sensor 25 , the torque sensor 26 , the accelerator pedal sensor 24 , the brake pedal sensor 27 , the door sensors 30 R, 30 L, 31 R, 31 L, and 32 , and the seat sensors 33 R, 33 L, 34 R, and 34 L.
  • control unit 50 receives an audio signal inputted into the driver's seat microphone 21 .
  • control unit 50 communicates with the navigation device 40 and communicates with the business terminal 9 and the like via the communication device 51 .
  • control unit 50 outputs control signals to control the operation of the speaker 22 , the display unit 23 , the communication device 51 , the navigation device 40 , and the like.
  • the information provision device 60 is formed of the CPU which executes the control programs for the information provision device 60 stored in the memory of the control unit 50 .
  • the information provision device 60 comprises a test drive recognition unit 61 , a current vehicle information acquisition unit 62 , a vehicle information acquisition unit 63 , an operation situation detection unit 64 , an action detection unit 65 , an emotion engine 66 (which corresponds the emotion estimation unit of the present invention), a function-of-interest estimation unit 67 , a surrounding situation recognition unit 68 , an information provision unit 69 , an image transmission unit 70 , and a vehicle emotion notification unit 71 .
  • the test drive recognition unit 61 recognizes that the test drive of the vehicle 1 is performed, by the communication with the business terminal 9 or the like.
  • the current vehicle information acquisition unit 62 acquires information on a vehicle (current vehicle) currently owned by the test driver P 1 , who is test-driving the vehicle 1 , by accessing a customer database (DB) 101 of a customer information server 100 via a communication network 110 .
  • DB customer database
  • the vehicle information acquisition unit 63 acquires information on the ON/OFF of an ignition switch (unillustrated) of the vehicle 1 , the operation of a safety device system (unillustrated) (operation of an advanced driver assistance systems (ADAS), an antilock brake system (ABS), an air bag, an emergency call, and the like), vehicle speed, acceleration/deceleration, and the like, through a control area network (CAN) or the like.
  • the operation situation detection unit 64 detects the operation situation of each operation unit of the vehicle 1 operated by the test driver P 1 based on the signal detected by each of the sensors 24 to 27 , 30 R, 30 L, 31 R, 31 L, 32 , 33 R, 33 L, 34 R, and 34 L.
  • the action detection unit 65 detects the action of the test driver P 1 based on the operation situation of each operation unit detected by the operation situation detection unit 64 , the situation of the test driver P 1 , who is driving or located around the vehicle 1 , and which is recognized from the images captured by the cameras 11 , 13 , and 20 , and the voice or the like of the test driver P 1 inputted into the driver's seat microphone 21 .
  • the emotion engine 66 includes an emotion engine A which estimates the emotion of a driver of the vehicle 1 (herein the test driver P 1 ) and an emotion engine B which generates an emotion of the vehicle 1 by personifying the vehicle 1 .
  • the emotion engine A quantifies the facial expression of the test driver P 1 recognized from the image captured by the driver's seat camera 20 ; the tone and intonation of voice of the test driver P 1 inputted into the driver's seat microphone 21 ; the operation situation of each operation unit operated by the test driver P 1 and detected by the operation situation detection unit 64 ; and the vehicle information acquired by the vehicle information acquisition unit 63 , all of which are plotted on the horizontal axis (axis of calmness) and the vertical axis (axis of steady state) as illustrated in FIG. 3 .
  • the degree of each emotion (emotion level) of the test driver P 1 is estimated and outputted.
  • the emotion engine 66 uses the facial expression of the test driver P 1 , the voice of the test driver P 1 , the operation situation of each operation unit of the vehicle 1 , and the vehicle information of the vehicle 1 to estimate the emotion level of the test driver P 1 , but at least the facial expression and the voice of the test driver P 1 may be used to estimate the emotion of the test driver P 1 .
  • the profile (preference, age, sex, and the like) of the test driver P 1 recognized from the customer data of the test driver P 1 , the biometric information (pulse rate, heart rate, blood pressure, and the like detected by biological sensors) of the test driver P 1 , the grip condition of the steering wheel gripped by the test driver P 1 may be used to estimate the emotion level of the test driver P 1 .
  • the emotion engine B quantifies the operation situation of each operation unit of the vehicle 1 detected by the operation situation detection unit 64 , the vehicle information of the vehicle 1 acquired by the vehicle information acquisition unit 63 , and other information, all of which are plotted on the horizontal axis (axis of calmness) and the vertical axis (axis of steady state) in the same manner as the emotion engine A as illustrated in FIG. 3 . Then, based on the quadrants to which the plotted points belong and the position of each point in the corresponding quadrant, the degree (emotion level) of each emotion (like, hate, tolerate, and calm) of the personified vehicle 1 is generated and outputted.
  • the function-of-interest estimation unit 67 estimates a specific function assumed to interest the test driver P 1 among the functions provided in the vehicle 1 .
  • the surrounding situation recognition unit 68 Based on the images around the vehicle 1 detected by the front camera 11 and the rear camera 13 ; the objects around the vehicle 1 detected by the front radar 12 and the rear radar 14 ; traffic information around the vehicle 1 received from the navigation device 40 ; and other information, the surrounding situation recognition unit 68 recognizes the situation of the road on which the vehicle 1 is travelling.
  • the vehicle emotion notification unit 71 sends a notification representing the emotion of the vehicle 1 by personifying the vehicle 1 to be detailed later.
  • STEP 1 is a process performed by the test drive recognition unit 61 .
  • the test drive recognition unit 61 recognizes that the test driver P 1 is test-driving the vehicle 1 .
  • the test driver P 1 who is a potential purchaser of a vehicle, visits a store S of a car dealer, the test driver P 1 is captured by a store camera 200 installed at an entrance of the store S, and the captured image data is transmitted to a personal computer 201 located in the store S.
  • the personal computer 201 analyzes the face image of the test driver P 1 and accesses a customer DB 101 held by the customer information server 100 to determine whether the test driver P 1 is a new customer (no customer data) or an existing customer (customer data being present). If the test driver P 1 is an existing customer, the customer data of the test driver P 1 acquired from the customer DB 101 (including the profile, preference, past transaction history, and the like of the test driver P 1 ) is transmitted to the business terminal 9 of the salesperson P 2 in charge of the test driver P 1 to notify that the test driver P 1 has come to the store.
  • the communication between the business terminal 9 and the personal computer 201 may be performed via the communication network 110 or may be performed by wireless LAN via a wireless router 202 connected to the personal computer 201 .
  • the personal computer 201 selects an appropriate salesperson P 2 , and transmits, to the business terminal 9 of the selected salesperson P 2 , data notifying that the test driver P 1 has come to the store.
  • the salesperson P 2 confirms the profile and the like of the test driver P 1 to newly generate customer data for the test driver P 1 .
  • the salesperson P 2 For the customer service of the test driver P 1 , the salesperson P 2 notifies that the vehicle 1 for test drive is ready and persuades the customer to have a test drive.
  • the salesperson P 2 transmits, to the information provision device 60 of the vehicle 1 , test drive start notification data notifying that the test driver P 1 will start test driving and customer data of the test driver P 1 .
  • the test drive recognition unit 61 recognizes that the test driver P 1 starts the test drive. Then, the process moves to STEP 2 .
  • the test drive recognition unit 61 may analyze a conversation voice between the test driver P 1 and the salesperson P 2 inputted into the driver's seat microphone 21 and from the analysis result, may recognize that test driver P 1 performs the test drive.
  • test drive recognition unit 61 When an ON (detection state) signal is outputted from the seat sensor 33 R of the driver's seat 5 R, the test drive recognition unit 61 recognizes that the test driver P 1 has got on the vehicle 1 , and when the seat sensor 33 R indicates an ON state, the test drive recognition unit 61 recognizes that the test driver P 1 is test-driving.
  • the test drive recognition unit 61 recognizes that the vehicle 1 starts test-driving, is test-driving, or ends test-driving.
  • the test drive recognition unit 61 Based on the change in the current position of the vehicle 1 acquired by the navigation device 40 , when it is detected that the vehicle 1 has returned to the store S, and when the output from the seat sensor 33 R is switched from the ON signal (detection state) to the OFF state (non-detection state) and the output from the door sensor 30 R of the front door 2 R on the driver's seat side is switched from the ON signal (detection state) to the OFF state (non-detection state), the test drive recognition unit 61 recognizes that the test drive has ended.
  • the test drive recognition unit 61 may analyze the conversation voice between the test driver P 1 and the salesperson P 2 inputted into the driver's seat microphone 21 , and from the analysis result (a voice analysis of the salesperson P 2 saying, for example, “Thank you very much for your test drive”, the test drive recognition unit 61 may recognize that the test drive of the test driver P 1 has ended.
  • the following STEP 2 is a process performed by the vehicle emotion notification unit 71 . While the test drive recognition unit 61 recognizes that the test drive is being performed, the vehicle emotion notification unit 71 sends a notification representing that the vehicle 1 is happy, in STEP 2 as illustrated in FIG. 7A . More specifically, the vehicle emotion notification unit 71 displays, on the display unit 23 , an image D 1 personifying the state in which the vehicle 1 is happy and at the same time outputs a voice V 5 saying, for example, “It's a perfect day for driving!” with a bright tone from the speaker 22 .
  • the higher the emotion level, the higher the degree of happiness expressed by the image D 1 and the voice V 5 may be.
  • the following STEP 3 is a process performed by the emotion engine A.
  • the emotion engine A performs the above described emotion estimation process at a predetermined sampling period and outputs the level of each emotion (like, hate, tolerate, and calm) of the test driver P 1 as needed.
  • the following STEP 10 branched from STEP 4 and STEP 5 is a process performed by the image transmission unit 70 .
  • the image transmission unit 70 determines whether or not the emotion level of “like” outputted from the emotion engine A is increased by a predetermined level or more.
  • the process is branched to STEP 10 , where the image transmission unit 70 captures an image of the test driver P 1 by the driver's seat camera 20 and holds data of the captured image in memory. Then, the process moves to STEP 5 .
  • the image may be a still image or a moving image. Meanwhile, if the emotion level of “like” is not increased by a predetermined level or more, the process moves to STEP 5 .
  • STEP 5 and STEP 20 to STEP 22 in FIG. 5 branched from STEP 5 are processes performed by the function-of-interest estimation unit 67 .
  • the function-of-interest estimation unit 67 determines whether or not the action detection unit 65 detects an action of the test driver P 1 . Then, if an action of the test driver P 1 is detected, the process is branched to STEP 20 in FIG. 5 . If no action of the test driver P 1 is detected, the process moves to STEP 6 .
  • the function-of-interest estimation unit 67 recognizes the emotion level of the test driver P 1 from the output of the emotion engine A. In the following STEP 21 , the function-of-interest estimation unit 67 determines whether or not the emotion level of “like” of the test driver P 1 is increased by a predetermined level or more. Then, if the emotion level of “like” of the test driver P 1 is increased by a predetermined level or more, the process moves to STEP 22 . If the emotion level of “like” of the test driver P 1 is not increased by a predetermined level or more, the process moves to STEP 6 in FIG. 4 .
  • the function-of-interest estimation unit 67 extracts a specific function of the vehicle 1 related to the action of the test driver P 1 detected in STEP 5 . For example, when an action that the test driver P 1 opens or closes the back door 4 of the vehicle 1 is detected (when the test driver P 1 opens or closes the back door 4 , the emotion of “like” of the test driver P 1 increases), the function-of-interest estimation unit 67 extracts “rear seat folding function” for enlarging the luggage compartment as the function related to this action.
  • the function-of-interest estimation unit 67 selects “sports mode function” as the function related to this action.
  • the following STEP 23 to STEP 25 , and STEP 30 are processes performed by the information provision unit 69 .
  • the information provision unit 69 determines the presence or absence of the current vehicle of the test driver P 1 from the customer data of the test driver P 1 . Then, if a current vehicle is found, the process moves to STEP 24 , where the information provision unit 69 acquires information on a specific function of the current vehicle and a vehicle (competing vehicle) competing with the vehicle 1 of the brand of the current vehicle (generally the manufacturer of the current vehicle).
  • the information provision unit 69 acquires information on “sports mode function” (presence or absence of the sports mode function, elements to be changed by the sports mode function, and the like) of the current vehicle of the test driver P 1 and “sports mode function” of a competing vehicle (which corresponds to the competing moving body of the present invention).
  • the information provision unit 69 notifies the test driver P 1 of the information on the specific function of the vehicle 1 , the current vehicle, and the competing vehicle by display on the display unit 23 and voice output from the speaker 22 . From this notification, the test driver P 1 can recognize the specific function of the vehicle 1 in terms of the difference between the current vehicle and the competing vehicle and the advantage thereof.
  • the process is branched from STEP 23 to STEP 30 , where the information provision unit 69 sends only the information on the specific function of the vehicle 1 .
  • the information on the specific function of the vehicle 1 includes information encouraging the use of the specific function.
  • FIG. 6A illustrates a notification sent when “rear seat folding function” is estimated as the specific function that interests the test driver P 1 , and the information provision unit 69 uses the speaker 22 to output a voice message V 1 saying “The rear seats can be easily folded. Please give it a try”. Further, the information provision unit 69 outputs, to the display unit 23 , a video explaining the rear seat folding procedure.
  • FIG. 6B illustrates a notification sent when “sports mode function” is estimated as the specific function that interests the test driver P 1 . More specifically, when the test driver P 1 performs an acceleration operation and says that “This acceleration is so nice!”, which voice V 2 is inputted into the driver's seat microphone 21 . In response to the voice V 2 , the information provision unit 69 uses the speaker 22 to output a voice V 3 saying “Thank you very much! This outstanding acceleration has been achieved by a combination of a high output motor and a low fuel consumption engine.” to appeal the driving performance of the vehicle 1 .
  • the information provision unit 69 uses the speaker 22 to output a voice V 4 saying that “The road seems to be empty, so why don't you switch to sports mode? Then, you can enjoy driving a lot more. Please try operating the sports mode switch”. Further, the information provision unit 69 outputs, to the display unit 23 , a video displaying the position (such as near a shift lever) in which the sports mode switch 28 is placed.
  • the following STEP 26 and STEP 40 branched from STEP 26 are processes performed by the image transmission unit 70 .
  • the image transmission unit 70 uses the operation situation detection unit 64 to determine whether or not the test driver P 1 performed the operation of the specific function.
  • the process is branched to STEP 40 , and if the test driver P 1 did not perform the operation of the specific function, the process moves to STEP 6 in FIG. 4 .
  • the image transmission unit 70 uses the driver's seat camera 20 to image the test driver P 1 and hold the imaged data in memory, and then the process moves to STEP 6 in FIG. 4 .
  • the image transmission unit 70 uses the driver's seat camera 20 to image the test driver P 1 and hold the imaged data in memory, but when the test driver P 1 performed the operation of the specific function and the emotion level of the test driver P 1 is increased by a predetermined level or more, in STEP 40 , the image transmission unit 70 may use the driver's seat camera 20 to image the test driver P 1 and hold the imaged data in memory.
  • STEP 6 and STEP 7 in FIG. 4 are processes performed by the vehicle emotion notification unit 71 .
  • the vehicle emotion notification unit 71 determines whether or not the test drive recognition unit 61 recognizes that the test driver P 1 ends the test drive of the vehicle 1 . Then, if the end of the test drive is recognized, the process moves to STEP 7 , and if the end of the test drive is not recognized, the process returns to STEP 2 , where the process following STEP 2 is executed again.
  • the vehicle emotion notification unit 71 notifies that the vehicle 1 feels sad as illustrated in FIG. 7B . More specifically, the vehicle emotion notification unit 71 displays, on the display unit 23 , an image D 2 personifying the state in which the vehicle 1 feels sad, and at the same time outputs a voice V 6 saying that “I'm very sorry to see you go. Hope to see you again.” from the speaker 22 with a heavy tone.
  • the following STEP 8 is a process performed by the image transmission unit 70 .
  • the image transmission unit 70 extracts the address of the test driver P 1 from the customer data of the test driver P 1 .
  • the image transmission unit 70 calculates the distance from the store S (test drive ending location) to a home H of the test driver P 1 to predict the time (expected time to return home) for the test driver P 1 to return to the home H. Note that the time elapsed from the time when the test drive ended to the expected time to return home corresponds the predetermined time of the present invention.
  • the image transmission unit 70 transmits an email 400 expressing appreciation for the test drive to a communication terminal 300 (a smartphone, a tablet, a mobile phone, and the like) of the test driver P 1 via a mail server 120 .
  • a communication terminal 300 a smartphone, a tablet, a mobile phone, and the like
  • the email 400 contains an image 401 of the test driver P 1 captured in STEP 10 of FIG. 4 or in STEP 40 of FIG. 5 ; a thank-you document 402 ; and an image 403 of the vehicle 1 or a vehicle of the same type and the same color as the vehicle 1 .
  • the image 403 corresponds the moving body image of the present invention and the configuration where the image transmission unit 70 acquires data of the image 403 from the customer information server 100 or the like corresponds to the moving body image acquisition unit of the present invention.
  • the image 401 of the test driver P 1 is an image captured when the emotion level of “like” of the test driver P 1 is increased by a predetermined level or more, the image 401 is basically a smile image. If a plurality of images are captured, smile detection is performed on each image, and an image with the highest smile level may be selected from among the detected smile images and may be used as the image 401 of the test driver P 1 to be attached to the email 400 .
  • the addresser of the email 400 is a vehicle type name (illustrated by “HT-SPORT” in FIG. 8 ) of the vehicle 1 , which can enhance the impression of the test driver P 1 to this type of vehicle by including the image 403 of the vehicle 1 or a vehicle of the same type and the same color as the vehicle 1 .
  • test driver P 1 performs test drive of the vehicle 1 .
  • the user can be made to recognize the usefulness of the current vehicle by providing information on the specific function assumed to interest the user in the same manner as described above.
  • a still image (image 401 ) of the test driver P 1 is attached to the email 400 , but a moving image of the user P 1 may be attached thereto. Further, when the image of the test driver P 1 is captured, data of the voice emitted by the test driver P 1 may be recorded and attached to the email together with the image. Furthermore, when the test rider P 1 has contracted the purchase of the vehicle, the contract date, the expected delivery date, and the like may be included in the email and the image information.
  • the image transmission unit 70 sends an email to the test driver P 1 by assuming the time for the test driver P 1 to return home, but the email may be sent to the test driver P 1 when it is estimated or detected that the user P 1 has left the predetermined area including the vehicle 1 .
  • the image transmission unit 70 may recognize that the user P 1 has left the predetermined area including the vehicle 1 , for example, (1) when a predetermined time has elapsed from when it was recognized from the images and the like captured by the cameras 11 , 13 , and 20 that the user P 1 left the vehicle 1 , or (2) when a predetermined time has elapsed from when the user P 1 ended the test drive of the vehicle 1 or the user performed the operation such as folding the seat.
  • the above embodiment comprises the action detection unit 65 and the emotion engine 66 and the function-of-interest estimation unit 67 estimates the specific function that interests the user based on the action of the user detected by the action detection unit 65 and the emotion of the user estimated by the emotion engine A when the action is detected.
  • another embodiment may comprise only the action detection unit 65 and the function-of-interest estimation unit 67 may estimate the specific function based on the action of the user detected by the action detection unit 65 .
  • another embodiment may comprise only the emotion engine 66 and the function-of-interest estimation unit 67 may estimate the specific function that interests the user based on the emotion of the user P 1 detected by the emotion engine 66 .
  • the above embodiment has described the four-wheeled vehicle 1 as the moving body of the present invention, but the present invention may be applied to various types of moving bodies including other types of vehicles such as two-wheeled vehicles, ships, airplanes, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Finance (AREA)
  • Development Economics (AREA)
  • Strategic Management (AREA)
  • Accounting & Taxation (AREA)
  • Multimedia (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Economics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Traffic Control Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
US16/317,144 2016-09-30 2017-06-22 Information provision device, and moving body Active US10706270B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2016193788 2016-09-30
JP2016-193788 2016-09-30
PCT/JP2017/023046 WO2018061353A1 (ja) 2016-09-30 2017-06-22 情報提供装置、及び移動体

Publications (2)

Publication Number Publication Date
US20190294867A1 US20190294867A1 (en) 2019-09-26
US10706270B2 true US10706270B2 (en) 2020-07-07

Family

ID=61762592

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/317,144 Active US10706270B2 (en) 2016-09-30 2017-06-22 Information provision device, and moving body

Country Status (5)

Country Link
US (1) US10706270B2 (zh)
JP (1) JP6657415B2 (zh)
CN (1) CN109690601B (zh)
DE (1) DE112017004942T5 (zh)
WO (1) WO2018061353A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220282681A1 (en) * 2021-03-02 2022-09-08 Mazda Motor Corporation Vehicle control system
US20220332326A1 (en) * 2021-04-14 2022-10-20 Toyota Jidosha Kabushiki Kaisha Information processing apparatus, non-transitory storage medium, and information processing method

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6991452B2 (ja) * 2017-06-27 2022-01-12 カワサキモータース株式会社 疑似感情生成方法並びに走行評価方法および走行評価システム
CN112307816B (zh) * 2019-07-29 2024-08-20 北京地平线机器人技术研发有限公司 车内图像获取方法、装置以及电子设备、存储介质
US20230298092A1 (en) * 2022-03-16 2023-09-21 Tekion Corp Vehicle availability user interface elements for an online system

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004341719A (ja) 2003-05-14 2004-12-02 Nec System Technologies Ltd 客観的顧客満足度評価システム及びその方法
JP2005128928A (ja) 2003-10-27 2005-05-19 Kureo:Kk 接客支援プログラム、接客支援方法、接客支援装置、接客支援バッチ発行プログラム、接客支援バッチ発行方法および接客支援バッチ発行装置
JP2010015451A (ja) 2008-07-04 2010-01-21 Toyota Motor Corp 運転者状態記録装置
US20110095908A1 (en) * 2009-10-22 2011-04-28 Nadeem Tamer M Mobile sensing for road safety, traffic management, and road maintenance
US20120215640A1 (en) * 2005-09-14 2012-08-23 Jorey Ramer System for Targeting Advertising to Mobile Communication Facilities Using Third Party Data
US20130106993A1 (en) * 1995-05-22 2013-05-02 Donnelly Corporation Driver assistance system for a vehicle
US20130343613A1 (en) * 2010-12-08 2013-12-26 Thomas Heger Camera-based method for determining distance in the case of a vehicle at standstill
US20140039723A1 (en) * 2012-08-03 2014-02-06 Ford Global Technologies, Llc Apparatus and method for transmitting static and dynamic information to a personal communication device in a vehicle
US20140135598A1 (en) * 2011-08-05 2014-05-15 Daimler Ag Method and Device to Monitor at Least One Vehicle Passenger and Method to Control at Least One Assistance Device
US20140171752A1 (en) * 2012-12-14 2014-06-19 Electronics And Telecommunications Research Institute Apparatus and method for controlling emotion of driver
US20140300739A1 (en) * 2009-09-20 2014-10-09 Tibet MIMAR Vehicle security with accident notification and embedded driver analytics
US20150039365A1 (en) * 2013-07-31 2015-02-05 Mashhur Zarif Haque Digital vehicle tag and method of integration in vehicle allocation system
US20150352955A1 (en) * 2013-04-18 2015-12-10 Mitsubishi Electric Corporation Moving body state informing apparatus, server system and moving body state informing method
US20160001781A1 (en) * 2013-03-15 2016-01-07 Honda Motor Co., Ltd. System and method for responding to driver state
US20160150070A1 (en) * 2013-07-18 2016-05-26 Secure4Drive Communication Ltd. Method and device for assisting in safe driving of a vehicle

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002245224A (ja) * 2001-02-19 2002-08-30 Hitachi Ltd 潜在顧客の情報収集システム及び方法
JP2003331176A (ja) * 2002-05-09 2003-11-21 Toyota Motor Corp 試乗車提供システム及び方法
JP3926254B2 (ja) * 2002-11-25 2007-06-06 Ｎｅｃシステムテクノロジー株式会社 車両に対する顧客満足度評価システム及びその方法
US9418368B2 (en) * 2007-12-20 2016-08-16 Invention Science Fund I, Llc Methods and systems for determining interest in a cohort-linked avatar
JP2010277583A (ja) * 2009-04-28 2010-12-09 Everica:Kk 試乗車両又は広告車両の提供システム
US9348492B1 (en) * 2011-04-22 2016-05-24 Angel A. Penilla Methods and systems for providing access to specific vehicle controls, functions, environment and applications to guests/passengers via personal mobile devices
JP2015169980A (ja) * 2014-03-05 2015-09-28 シャープ株式会社 運転監視装置

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130106993A1 (en) * 1995-05-22 2013-05-02 Donnelly Corporation Driver assistance system for a vehicle
JP2004341719A (ja) 2003-05-14 2004-12-02 Nec System Technologies Ltd 客観的顧客満足度評価システム及びその方法
JP2005128928A (ja) 2003-10-27 2005-05-19 Kureo:Kk 接客支援プログラム、接客支援方法、接客支援装置、接客支援バッチ発行プログラム、接客支援バッチ発行方法および接客支援バッチ発行装置
US20120215640A1 (en) * 2005-09-14 2012-08-23 Jorey Ramer System for Targeting Advertising to Mobile Communication Facilities Using Third Party Data
JP2010015451A (ja) 2008-07-04 2010-01-21 Toyota Motor Corp 運転者状態記録装置
US20140300739A1 (en) * 2009-09-20 2014-10-09 Tibet MIMAR Vehicle security with accident notification and embedded driver analytics
US20110095908A1 (en) * 2009-10-22 2011-04-28 Nadeem Tamer M Mobile sensing for road safety, traffic management, and road maintenance
US20130343613A1 (en) * 2010-12-08 2013-12-26 Thomas Heger Camera-based method for determining distance in the case of a vehicle at standstill
US20140135598A1 (en) * 2011-08-05 2014-05-15 Daimler Ag Method and Device to Monitor at Least One Vehicle Passenger and Method to Control at Least One Assistance Device
US20140039723A1 (en) * 2012-08-03 2014-02-06 Ford Global Technologies, Llc Apparatus and method for transmitting static and dynamic information to a personal communication device in a vehicle
US20140171752A1 (en) * 2012-12-14 2014-06-19 Electronics And Telecommunications Research Institute Apparatus and method for controlling emotion of driver
US20160001781A1 (en) * 2013-03-15 2016-01-07 Honda Motor Co., Ltd. System and method for responding to driver state
US20150352955A1 (en) * 2013-04-18 2015-12-10 Mitsubishi Electric Corporation Moving body state informing apparatus, server system and moving body state informing method
US20160150070A1 (en) * 2013-07-18 2016-05-26 Secure4Drive Communication Ltd. Method and device for assisting in safe driving of a vehicle
US20150039365A1 (en) * 2013-07-31 2015-02-05 Mashhur Zarif Haque Digital vehicle tag and method of integration in vehicle allocation system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
International Search Report dated Sep. 12, 2017, issued in counterpart International Application No. PCT/JP2017/023046 (1 page).

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220282681A1 (en) * 2021-03-02 2022-09-08 Mazda Motor Corporation Vehicle control system
US11739703B2 (en) * 2021-03-02 2023-08-29 Mazda Motor Corporation Vehicle control system
US20220332326A1 (en) * 2021-04-14 2022-10-20 Toyota Jidosha Kabushiki Kaisha Information processing apparatus, non-transitory storage medium, and information processing method
US11993271B2 (en) * 2021-04-14 2024-05-28 Toyota Jidosha Kabushiki Kaisha Information processing apparatus, non-transitory storage medium, and information processing method

Also Published As

Publication number Publication date
WO2018061353A1 (ja) 2018-04-05
DE112017004942T5 (de) 2019-06-19
US20190294867A1 (en) 2019-09-26
JP6657415B2 (ja) 2020-03-04
CN109690601B (zh) 2022-11-18
CN109690601A (zh) 2019-04-26
JPWO2018061353A1 (ja) 2019-03-22

Similar Documents

Publication Publication Date Title
US10773726B2 (en) Information provision device, and moving body
US10706270B2 (en) Information provision device, and moving body
JP4980737B2 (ja) 車両用情報提示装置
CN107886045B (zh) 设施满意度计算装置
CN112026790B (zh) 车载机器人的控制方法及装置、车辆、电子设备和介质
CN111717083B (zh) 一种车辆交互方法和一种车辆
CN111261154A (zh) 智能体装置、智能体提示方法及存储介质
US11184586B2 (en) Server, vehicle image capturing system, and vehicle image capturing method
CN111007968A (zh) 智能体装置、智能体提示方法及存储介质
JP2020166524A (ja) 監視システム、監視方法およびコンピュータプログラム
CN112918381B (zh) 一种车载机器人迎送宾方法、装置及系统
US20210082395A1 (en) Method for operating a sound output device of a motor vehicle, voice-analysis and control device, motor vehicle and motor-vehicle-external server device
JP2017072874A (ja) 情報提供装置、情報提供システム及び情報提供方法
JP2020103704A (ja) 情報処理装置及びプログラム
JP7151400B2 (ja) 情報処理システム、プログラム、及び制御方法
JP6332072B2 (ja) 対話装置
CN110281857B (zh) 显示控制装置以及显示控制方法
CN114291008B (zh) 车辆用智能体装置、车辆用智能体系统以及计算机可读存储介质
JP2020060623A (ja) エージェントシステム、エージェント方法、およびプログラム
JP7543082B2 (ja) 情報提示装置、情報提示方法及び情報提示プログラム
JP7347344B2 (ja) 情報処理装置、情報処理システム、プログラム、及び車両
CN115534807A (zh) 一种车辆内遗落物品的提醒方法和提醒装置
JP2024104607A (ja) 情報処理装置、情報処理方法、情報処理プログラム及び記憶媒体
JP2024067341A (ja) 車両の情報提示方法及び情報提示装置

Legal Events

Date Code Title Description
AS Assignment

Owner name: HONDA MOTOR CO., LTD., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GOTO, SHINICHIRO;SHINTANI, TOMOKO;YUHARA, HIROMITSU;AND OTHERS;SIGNING DATES FROM 20181204 TO 20181211;REEL/FRAME:048782/0578

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4