KR20210117619A - Proactive digital assistant - Google Patents

Abstract

The present invention relates to a proactive digital assistant. The present invention may provide a method comprising the following steps of: detecting a suggestion triggering condition in a vehicle; providing a suggestion to a user based on the detected suggestion triggering condition; receiving a user's answer to the suggestion; and performing a function of the vehicle based on the user's answer and vehicle configuration information of the vehicle. The method may be provided, in which the vehicle configuration information is stored in an external device.

Description

PROACTIVE DIGITAL ASSISTANT

The present disclosure relates to a proactive digital assistant, and more particularly, to a proactive digital assistant in a vehicle.

An artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike the existing rule-based smart system, the machine learns, judges, and becomes smarter by itself. As artificial intelligence systems are used, the recognition rate improves and users can understand user preferences more accurately.

Digital assistants have become popular through smartphones and smart speakers, and digital assistants have developed rapidly through artificial intelligence systems. The digital assistant may interpret a user's natural language command, for example, a voice command, and perform a function corresponding to the user's intention. On the other hand, various technologies for using the digital assistant usefully in a vehicle are being developed.

The above information is presented only as background information to help understanding of the present disclosure. No determination or claim is made as to which of the above is applicable as prior art to the present disclosure.

According to the present disclosure, detecting a suggested triggering condition in a vehicle; providing a suggestion to a user based on the detected suggestion triggering condition; receiving the user's answer to the suggestion; and performing a function of the vehicle based on the answer of the user and vehicle configuration information of the vehicle.

A method may be provided in which the vehicle configuration information is stored in an external device.

The method may further include transmitting identification information of the vehicle to the external device, wherein the vehicle configuration information of the vehicle is identified based on the identification information of the vehicle.

The method may further include receiving, from the external device, indication information determined based on the vehicle configuration information, wherein the function of the vehicle is performed based on the indication information.

The method further includes transmitting information on the proposal triggering condition to an external device, wherein the providing the proposal includes receiving a proposal generated from the external device based on the detected proposal triggering condition to the user. A method may be provided comprising the step of providing said proposal.

There may be provided a method including receiving, from the external device, the indication information determined by the external device based on the vehicle configuration information, and the proposal generated in the external device together.

A method may be provided in which the vehicle configuration information is stored in the vehicle.

transmitting the suggestion and the user's answer to an external device; receiving a request for factor information from the external device; identifying corresponding factor information corresponding to the factor information in the vehicle configuration information and transmitting the corresponding factor information to the external device; and receiving, from the external device, the indication information determined based on the correspondence factor information, wherein the function of the vehicle is performed based on the indication information.

The vehicle configuration information may provide a method including information on at least one of an adjustable temperature range, an adjustable air volume range, whether each seat supports cooling or not, and a heating support for each seat.

The method further comprising the step of detecting a proposal triggering disturbance condition in the vehicle, wherein the providing the proposal comprises providing the proposal based on the detected proposal triggering condition and the proposal triggering disturbance condition may be provided.

The detecting of the suggested triggering condition may include detecting the suggested triggering condition based on an environment of the vehicle.

A method may be provided wherein the environment of the vehicle includes an internal environment and an external environment of the vehicle.

The environment of the vehicle may include at least one of a temperature, a state of consumables of the vehicle, a state of a power source of the vehicle, a location of the vehicle, and whether the vehicle is parked.

The detecting of the suggestion triggering condition may include detecting the suggestion triggering condition based on user profile information.

The user profile information may include user profile information of each of a plurality of users including the user.

A method may be provided, wherein the detecting of the suggested triggering condition includes detecting the suggested triggering condition based on a routine of the user in the vehicle.

A method may be provided in which the proposal triggering condition and the proposal are learned based on the user's answer to the proposal.

Further, according to the present disclosure, a memory for storing instructions; and executing the instructions: detect a suggestion triggering condition in the vehicle, provide a suggestion to a user based on the detected suggestion triggering condition, receive the user's answer to the suggestion, and A device including at least one processor configured to perform a function of the vehicle may be provided based on the vehicle configuration information of the vehicle.

A computer-readable recording medium recording a program for executing the method in a computer may be provided.

Furthermore, according to the present disclosure, there is provided a method comprising: receiving a proposal provided to a user based on a proposal triggering condition detected in the vehicle, and a response of the user to the proposal; determining instruction information for causing the vehicle to perform a function of the vehicle, based on the answer of the user and vehicle configuration information of the vehicle; and transmitting the indication information to the vehicle.

1 is a diagram for describing a digital assistant according to an exemplary embodiment.
2 is a block diagram illustrating units of a device according to an embodiment.
3 is a flowchart of a method according to an embodiment.
4 is a diagram for explaining a condition for triggering a proposal according to an embodiment.
5 is a diagram for explaining a proposed triggering condition and a proposed triggering interference condition according to an embodiment.
6 is a diagram for describing an interaction between a vehicle and a server according to an exemplary embodiment.
7 is a diagram for describing an interaction between a vehicle and a server according to an exemplary embodiment.
8 is a block diagram of a device according to an embodiment.

Hereinafter, with reference to the accompanying drawings, embodiments will be described in detail so that those of ordinary skill in the art can easily implement them. However, the embodiments may be implemented in various forms and are not limited to the embodiments described herein. And, in order to clearly describe the embodiments in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts.

The singular expression includes the plural expression unless the context clearly dictates otherwise. Terms such as “comprise” or “have” are intended to designate the presence of a feature, number, step, action, component, part, or combination thereof, but one or more other features or number, step, action, configuration It should be understood that it does not preclude the possibility of the presence or addition of elements, parts or combinations thereof. In particular, the numbers are examples for helping understanding, and it should not be understood that the embodiments are limited by the numbers described.

1 is a diagram for describing a digital assistant according to an exemplary embodiment.

In embodiments, a digital assistant installed in the vehicle 100 is described, but in addition to the vehicle 100 , mobile devices such as smartphones, tablets, laptops, smart watches, smart bands, and wearables such as smart glasses It may be mounted on various end-user devices, such as appliances, refrigerators, televisions, home appliances such as speakers, and the like. A digital assistant may also be referred to as a voice assistant.

FIG. 1 shows only the steering wheel 110 , the display 120 , and the air outlet 130 as components of the vehicle 100 or the dashboard of the vehicle 100 for convenience of explanation. The display 120 may be included in a center fascia, but is not limited thereto. For example, the display 120 is a head up display 120 (HUD), and may be reflected by the windshield of the vehicle 100 and shown to the user. The display 120 may be provided on the dashboard separately from the center fascia, or may be implemented as a transparent display 120 .

Referring to FIG. 1 , when a suggestion triggering condition is detected in the vehicle 100 , a suggestion may be provided to a user. The proposal of the digital assistant may include a proposal of a function to be performed in the vehicle 100 by the digital assistant. The digital assistant's proposal may be a user's request for permission to perform a corresponding function in the vehicle 100 . For example, when the temperature difference between the inside and outside of the vehicle 100 is out of a predetermined range, for example, 5 degrees, the digital assistant may provide the user with a suggestion for operating the air conditioner. That is, the digital assistant may request the user's permission to operate the air conditioner. The suggestion of the digital assistant may be provided by voice so as to be easily recognized by a user driving the vehicle 100 , but is not limited thereto. For example, the suggestion may be provided to the user via the display 120 of the vehicle 100 .

The digital assistant may perform a corresponding function based on the user's answer to the digital assistant's suggestion. The user's answer may be received at the digital assistant as a voice, but is not limited thereto. For example, the user's answer may be received via physical buttons 112 , 114 , and 116 . Referring to FIG. 1 , physical buttons 112 , 114 , and 116 may be provided on the steering wheel 110 of the vehicle 100 . Physical buttons 112 , 114 , and 116 may include an agree button 112 and a decline button 114 to allow the user to respond to the digital assistant's offer. The consent button 112 and the reject button 114 may perform different functions together. For example, the consent button 112 may function as a selection button, an input button, a digital assistant call button, a call answer button, a message reply button, and the like. The reject button 114 may function as a back button, a cancel button, a call rejection button, a mute button, and the like. The physical buttons 112 , 114 , and 116 may be provided at the edge of the handle 110 , but may also be provided at the center of the handle 110 .

When the user answers “yes” to the air conditioner operation suggestion or presses the consent button 112 , the digital assistant may operate the air conditioner of the vehicle 100 . When the user answers "no" to the air conditioner operation suggestion or presses the reject button 114 , the digital assistant may not operate the air conditioner of the vehicle 100 . The user's silence can be treated as "yes" or "no", and if the user is silent, the digital assistant may provide the same suggestion to the user again, or a different suggestion. The digital assistant may adjust the air volume of the air conditioner based on the internal temperature, internal humidity, and the like.

According to an embodiment, even if the user does not call the digital assistant first, the digital assistant proactively speaks to the user, thereby shortening the procedure the user must go through to perform a specific function. That is, the vehicle 100 performs an appropriate function currently required by the user in the vehicle 100 only by answering the preemptive suggestion of the digital assistant, without the user creating a sentence and delivering it to the digital assistant or complicatedly pressing several buttons or adjusting the dial. can be

In an embodiment, whether the plurality of air outlets including the air outlet 130 operates may vary based on the occupant of the vehicle. For example, when there is no passenger in the passenger seat, cooling efficiency may be increased by closing the air outlet 130 close to the passenger seat. Furthermore, the digital assistant may control the blowing direction of the tuyeres.

In one embodiment, the digital assistant may provide a suggestion to the user based on the user's user profile, user routine, environment information of vehicle 100, etc., which will be described below with reference to FIG. 4 .

2 is a block diagram illustrating units of a device according to an embodiment.

A digital assistant may be used in the device 200 . The digital assistant may be included in the operating system of the device 200 , but is not limited thereto. For example, the digital assistant may be installed in the device 200 through an application store, an application market, or the like. The device 200 may be a vehicle, but is not limited thereto. For example, the device 200 is not only a vehicle, but also various end users such as mobile devices such as smartphones, tablets, and laptops, smart watches, smart bands, wearable devices such as smart glasses, refrigerators, televisions, and home appliances such as speakers. It may be a device. The device 200 may be an electronic device connectable to a vehicle.

Referring to FIG. 2 , the device 200 may include a condition detecting unit 210 , a proposal providing unit 220 , an answer receiving unit 230 , and a function performing unit 240 . The units of device 200 may be units of a digital assistant. According to embodiments, device 200 may include more or fewer units than the number of units described above. Since the naming of units of the device 200 is for distinguishingly describing operations performed in the device 200 , a specific operation is not necessarily performed in a specific unit. For example, an operation depicted as being performed in a particular unit of the device 200 may be performed in another unit, and an operation depicted as being performed in one unit of the device 200 may be performed in a plurality of units. and even if an operation described as being performed by interactive processing between a plurality of units of the device 200 is performed by one unit, substantially the same functionality may be provided. Furthermore, even if an operation depicted as being performed in the device 200 is performed in another device or performed with the help of another device, substantially the same functionality may be provided.

The units of the device 200 may be implemented as software modules, and the software modules may be executed by a processor to perform functions of the units. Meanwhile, the software modules may be stored in the memory as a set of instructions, and the instructions stored in the memory may be executed by the processor. The units of the device 200 may be implemented as a hardware module or a combination of a hardware module and a software module. The device 200 may include a memory and a processor.

Further reference is made to FIG. 3 to describe the function of each unit of the device 200 .

3 is a flowchart of a method according to an embodiment.

In operation S310, a suggested triggering condition may be detected in the vehicle. The device 200 , for example, the condition detection unit 210 of the device 200 may detect a suggested triggering condition in the vehicle.

In one embodiment, the suggested triggering condition in the vehicle may be different according to the environment of the vehicle, the user profile of the user, the user routine, etc., which will be described later with reference to FIG. 4 .

In an embodiment, when a suggested triggering condition is detected in the vehicle, the device 200 may transmit information about the condition to the server. The server may be a digital assistant server, or a digital assistant cloud. The device 200 may notify the server that a predetermined suggested triggering condition has been detected. In one embodiment, the proposal triggering condition may be predetermined. The suggested triggering condition may be set as a condition suitable for the user through the artificial neural network. For example, the suggested triggering condition may be set to an appropriate condition based on the user's user routine. Accordingly, a proposal triggering condition suitable for each user may be set.

In operation S320, a suggestion may be provided to the user. The device 200 , for example, the proposal providing unit 220 of the device 200 may provide a proposal to the user. A suggestion may be provided to the user based on the suggestion triggering condition detected in operation S310.

In an embodiment, the device 200 may receive a proposal generated by the server based on the detected proposal triggering condition. In an embodiment, a proposal generated based on a proposal triggering condition may be learned through an artificial neural network, and accordingly, a proposal suitable for each user may be generated. The device 200 may generate a proposal based on the detected proposal triggering condition.

In one embodiment, when the life of the consumables of the vehicle reaches a predetermined threshold, the digital assistant may provide the user with an offer of replacement of the consumables. For example, when the vehicle is driven more than a predetermined distance after changing the engine oil of the vehicle, the digital assistant may provide a suggestion of replacing the engine oil to the user. The digital assistant may automatically set a deadline for changing the engine oil based on the user's average daily driving distance. For example, a digital assistant can offer a user a suggestion to change engine oil within a week. The digital assistant may recommend to the user a repair shop where engine oil can be changed, or may suggest navigation guidance to the repair shop to the user. Similarly, if the vehicle's power source falls below a predetermined threshold, the digital assistant may provide the user with a suggestion to replenish that power source. For example, when the power source falls below 10% of the total power source, an appropriate gas station may be recommended to the user by calculating the drivable distance and time with the remaining power source, or guidance to the gas station may be suggested to the user.

In operation S330, a user's answer may be received. The device 200 , for example, the answer receiving unit 230 of the device 200 may receive the user's answer. The user's answer may include, but is not limited to, agreeing to or rejecting the digital assistant's offer. The user's response to the suggestion may be a command related to the digital assistant's suggestion. For example, if the digital assistant provides the user with a suggestion of changing the engine oil of the vehicle, and the user instructs the registration of a schedule for the suggestion, the digital assistant enters the "engine oil change" schedule on the date specified by the user can do. The digital assistant can recommend to the user a workshop where the engine oil can be changed on that date. The digital assistant may recommend a repair shop to the user based on the vehicle's current location, replacement cost, business hours of the repair shop, and the like. If the user agrees, the digital assistant may proceed with a reservation for the corresponding repair shop.

In operation S340 , a function of the vehicle may be performed based on the user's answer and vehicle configuration information of the vehicle. The device 200 , for example, the function performing unit 240 of the device 200 may perform a vehicle function based on a user's answer and vehicle configuration information of the vehicle. In one embodiment, the function of the vehicle may include operation of a heating, ventilation, & air conditioning (HVAC) system. The functions of the vehicle may include the operation of the vehicle's infotainment system, eg, navigation, video player, audio player, phone, message, mail, etc., operation of various applications installed in the infotainment system. The function of the vehicle may include an operation of a user's mobile device connected to the vehicle and an operation of various applications installed in the mobile device.

In one embodiment, if the user was watching media before starting the vehicle or boarding the vehicle, the digital assistant may request the user's permission to continue playing the media. If the user responds to the request, the digital assistant can continue playing the media. The user can continue to watch the media without going through a complicated procedure to go through to watch the media continuously. In one embodiment, after the user responds to the digital assistant's suggestion, the digital assistant may provide further suggestions to the user. For example, if the user's answer is judged to be inactive, an entertainment element may be recommended to the user. For example, the digital assistant may further suggest a conversation, music playback, video playback, etc. to the user.

4 is a diagram for explaining a condition for triggering a proposal according to an embodiment.

In an embodiment, the digital assistant may provide the user with a suggestion of the operation of the air conditioning system of the vehicle based on the temperature difference between the interior and exterior of the vehicle. The digital assistant can provide suggestions of various operations of the HVAC system, for heating, ventilation, and air-conditioning of the vehicle. For example, when the internal temperature of the vehicle exceeds 40 degrees, the digital assistant provides the user with suggestions to turn on the air conditioner, set the target temperature to the lowest, set the air volume to the maximum, and operate in the air circulation mode. and if the user agrees to the proposal, the digital assistant may perform the function according to the proposal in the vehicle.

Referring to FIG. 4 , in one embodiment, the digital assistant may refer to environmental information associated with the vehicle, a user profile, a user routine, and the like to detect a suggested triggering condition.

In an embodiment, the environment information may include internal environment information and external environment information of the vehicle. The environment information may include information on a difference between an environment inside and outside the vehicle. For example, the environmental information may include information on weather, temperature, humidity, air quality, fine dust concentration, state of consumables of the vehicle, the state of the vehicle's power source, whether the vehicle is parked, the current location of the vehicle, the environment of the current location of the vehicle, etc. information may include, but is not limited to. The environmental information may be collected through a sensor installed in the vehicle or may be collected through the Internet.

In one embodiment, when the vehicle is parked, the digital assistant may use the vehicle's camera to photograph the surroundings of the vehicle and request permission from the user to transmit the captured image to the user's mobile device. When the user responds to the request, the image and information on the current location of the vehicle may be transmitted to the user's mobile device together.

In one embodiment, when the vehicle passes through a place with low air quality or a high concentration of fine dust, for example, a tunnel, a construction site, or a digital assistant closes the window of the vehicle, A suggestion may be suggested to the user, and accordingly, the interior of the vehicle may be comfortably maintained even in a situation in which the user is difficult to recognize.

In one embodiment, the digital assistant may adjust the opening and closing speed of the vehicle window. For example, if there is a tunnel in front of the vehicle and it is calculated that the tunnel will be entered 30 seconds after the user's consent reply is received, the digital assistant can adjust the opening and closing speed of the window so that the window closes within 30 seconds. have. Accordingly, fresh air can be introduced into the vehicle for as long as possible. The maximum speed of the opening/closing speed may be limited to a predetermined speed.

In one embodiment, the user profile may comprise a logged in user profile in the vehicle. The user profile may include a user profile linked to a user account of the vehicle. The user profile may include a user profile of a plurality of users including the user. For example, a user profile may include user profiles of family members of that user. The user profile may include user profiles of the user's friends or colleagues. The user profile may include user profiles of users who agree to inquire or use their own user profile among passengers.

In one embodiment, the digital assistant may play media based on the preferences of a plurality of users. For example, the digital assistant may play music in the vehicle by combining the music recommendation lists generated based on the user profiles of the passengers. Accordingly, an environment that can satisfy all occupants of the vehicle may be created.

In one embodiment, a user routine may be linked to a user profile. A user routine is a routine that a user repeats in a vehicle, and information on the user routine may be collected and analyzed over a predetermined time period. For example, when there is a radio broadcast that the user enjoys listening to in a specific time zone, the digital assistant may request the user's permission to play the corresponding radio broadcast in the corresponding time zone.

In an embodiment, the suggestion triggering condition may be learned through an artificial neural network, and accordingly, a suggestion triggering condition suitable for each user may be set.

Referring to FIG. 4 , the digital assistant may refer to vehicle configuration information. The digital assistant may refer to the vehicle configuration information to provide suggestions to the user. The digital assistant may refer to vehicle configuration information to perform functions in the vehicle. The vehicle configuration information will be described later with reference to FIGS. 6 and 7 .

5 is a diagram for explaining a proposed triggering condition and a proposed triggering interference condition according to an embodiment.

In one embodiment, the digital assistant may include an event queue 510 , a block event receiver 520 , and an event dispatcher 530 . The event queue 510 and the block event receiver 520 may be included in the condition detection unit 210 described above with reference to FIG. 2 . The event dispatcher 530 may be included in the proposal providing unit 220 described above with reference to FIG. 2 .

Various events occurring in the vehicle are queued in the event queue 510 . Events can be queued according to the order of occurrence. Referring to FIG. 5 , a call event, a message event, an HVAC event, and the like may occur in the vehicle. A phone event may be recorded based on a telephony manager, a call log DB, etc., a message event may be recorded through an SMS (short-message service) Received Intent, and an HVAC event, the vehicle's may be recorded through HVAC and vehicle services, but is not limited thereto. The condition detection unit 210 may detect a suggested triggering condition based on events stored in the event queue 510 .

The condition detection unit 210 may detect a suggested triggering interference condition through the block event receiver 520 . For example, in the block event receiver 520 , when an event satisfying the proposal triggering interference condition is received, the proposal may not be provided to the user based on the detected proposal triggering condition. For example, when the preemptive suggestion of the digital assistant is deactivated, the digital assistant may perform a function in response to the user's instruction without providing the suggestion to the user. For example, when a user's instruction is received even if a suggested triggering condition is detected, a vehicle function may be first performed according to the user's instruction so that the user's instruction can be quickly processed. After the function according to the user's instruction is performed, the digital assistant may provide a suggestion to the user based on the detected suggestion triggering condition.

The proposal providing unit 220 may provide a proposal to the user through the event dispatcher 530 . The suggestion may include a suggestion of an event or function to be called in the vehicle by the event dispatcher 530 . The proposal may be a request for permission of the user to call a corresponding event in the vehicle or perform a corresponding function. When the user responds to the suggestion, a corresponding event may be called in the vehicle or a corresponding function may be performed.

6 is a diagram for describing an interaction between a vehicle and a server according to an exemplary embodiment.

Referring to FIG. 6 , the vehicle 1000 may provide a suggestion to the user through interaction with the server 2000 . The server 2000 may be composed of a plurality of servers. The operation S610 of detecting the suggestion triggering condition in the vehicle, the operation S620 of providing a suggestion to the user based on the detected suggestion triggering condition, and the operation S630 of receiving the user's answer to the suggestion are the operations described above with reference to FIG. 2 . Since it is substantially the same as , redundant description is omitted. For example, based on detecting that the temperature difference between the inside and outside of the vehicle is out of a predetermined range, the vehicle provides the user with an air conditioner operation suggestion, and the user may respond to the suggestion.

In operation S612 , the vehicle 1000 may transmit information on the detected triggering condition to the server 2000 . For example, the vehicle 1000 may notify the server that the corresponding suggested triggering condition has been detected. When the vehicle 1000 is implemented to generate a proposal based on the detected proposal triggering condition, operation S612 may be omitted.

In operation S614 , the server may generate a proposal to be provided to the user based on the detected proposal triggering condition and transmit it to the vehicle 1000 . Meanwhile, the vehicle 1000 may be implemented to generate a proposal based on the detected proposal triggering condition.

In one embodiment, the vehicle configuration information is information representing characteristics of the vehicle 1000 . For example, the vehicle configuration information may include information on the hardware configuration of the vehicle 1000 . The vehicle configuration information may include information limited in performing a function in the vehicle 1000 . For example, the vehicle configuration information may include information on an adjustable temperature range, an adjustable fan speed range, whether or not a phone service exists, and the like. The vehicle configuration information may include information on manufacturing the vehicle 1000 . For example, the vehicle configuration information may include information about a vehicle model name, a vehicle manufacturer, a vehicle sales country, a year, a trim, a vehicle selection option, and the like. Vehicle configuration information includes the driver's seat location, engine type, engine structure, engine power, engine torque, power source type, power source capacity, phone presence, camera presence, number, and location, sunroof presence, It may include information on the number and location of the seats, whether each seat supports heating, whether or not each seat supports cooling.

The following is an example code of vehicle configuration information.

<configuration>

<limitations>

<temperatureRange>59,85</temperatureRange>

<fanSpeedRange>0,7</fanSpeedRange>

<telephonySupport>true</telephonySupport>

</limitations>

<oem>

<maker>XXX</maker>

<model>YYY</model>

<model_year>2019</model_year>

<trim>ZZZ</trim>

<geography>Korea</geography>

</oem>

<automotive>

<driver_pos>left</driver_pos>

<engine>

<displacement>1995</displacement>

<cylinder>4</cylinder>

<horse_power>190</horse_power>

<toque>40.8</toque>

</engine>

<fuel>

<type>gasoline</type>

<tank_capa>60</tank_capa>

</fuel>

<exterior>

<camera>

<type>left_side,right_side,rear,front</type>

</camera>

<sunroof exist="none" />

</exterior>

<interior>

<seat>

<count>5</count>

<row_1_left>

<exist>true</exist>

<heating>true</heating>

<cooling>true</cooling>

</row_1_left>

<row_1_right>

<exist>true</exist>

<heating>true</heating>

<cooling>true</cooling>

</row_1_right>

<row_2_left>

<exist>true</exist>

<heating>true</heating>

<cooling>true</cooling>

</row_2_left>

<row_2_center>

<exist>true</exist>

<heating>true</heating>

<cooling>true</cooling>

</row_2_center>

<row_2_right>

<exist>true</exist>

<heating>true</heating>

<cooling>true</cooling>

</row_2_right>

</seat>

</interior>

</automotive>

</configuration>

In one embodiment, the digital assistant may perform a function in the vehicle based on the vehicle configuration information. Referring to FIG. 6 , vehicle configuration information may be stored in the server 2000 .

In operation S632 , the vehicle 1000 may transmit vehicle identification information to the server 2000 . Transmission of vehicle identification information for the proposal may be performed only when the user responds to the proposal of the digital assistant, but is not limited thereto. The vehicle identification information may include a vehicle identification number of the vehicle. Operation S632 may be performed together with operation S612.

It should be understood that operations S632 and S634 may be performed before the above-described operations are performed, and are not limited to being performed in the order of the flowchart.

In operation S634 , the server 2000 may identify vehicle configuration information of the corresponding vehicle based on the vehicle identification information received from the vehicle 1000 .

In operation S636, the server 2000 may determine instruction information for causing the vehicle to perform a predetermined function. The server 2000 may determine instruction information indicating which function the vehicle will perform based on the user's answer received from the vehicle and the identified vehicle configuration information. In the indication information determined by the server 2000, settable ranges such as temperature, air volume, air conditioning, heating, heating wire, and cooling sheet may be limited by vehicle configuration information.

In operation S638 , the server 2000 may transmit the determined indication information to the vehicle 1000 . Operation S636 may be performed between operations S612 and S614, and operation S638 may be performed together with operation S614.

In operation S640 , the vehicle 1000 may perform a function based on the instruction information. That is, the function indicated by the indication information may be performed.

In an embodiment, the vehicle 1000 may refer to the vehicle configuration information to provide a suggestion to the user based on the detected suggestion triggering condition. That is, the vehicle 1000 may acquire vehicle configuration information in advance by transmitting vehicle identification information to the server 2000 . The vehicle 1000 may provide a proposal generated based on the vehicle configuration information to the user.

According to an embodiment, the digital assistant may be smoothly operated even in vehicles manufactured in various ways by various manufacturers. The digital assistant can be learned through an artificial neural network, and in order to improve learning efficiency and response accuracy, the terminal (or vehicle 1000) transmits a query to the server 2000, and the server 2000 responds to the query. A response may be transmitted to the terminal (or the vehicle 1000 ). However, when the vehicle configuration information is hard-coded, the digital assistant cannot fine-tune the environment of the vehicle 1000 . According to an embodiment, since vehicle configuration information is managed in the server 2000, the digital assistant operated in the server 2000 can provide a more precise suggestion to the user by learning using data collected from various vehicles. have. Furthermore, even if vehicle-related information is not stored in the vehicle at the time of initial manufacturing or shipment of the vehicle, according to an embodiment, since vehicle configuration information is managed in the server 2000, information on the vehicle is maintained and utilized as the latest information. can be

7 is a diagram for describing an interaction between a vehicle and a server according to an exemplary embodiment.

Referring to FIG. 7 , vehicle configuration information may be stored in the vehicle 1000 .

Referring to FIG. 7 , the vehicle 1000 may provide a suggestion to the user through interaction with the server 2000 . In operation S710 of detecting a suggested triggering condition in the vehicle, transmitting information on the detected condition to the server 2000 (S712), receiving a proposal from the server 2000 (S714), in response to the detected suggestion triggering condition An operation S720 of providing a suggestion to the user based on the operation S720 and an operation S730 of receiving the user's answer to the proposal are substantially the same as the operations described above with reference to FIGS. 2 and 6 , and thus a redundant description thereof will be omitted.

In an embodiment, the vehicle 1000 may transmit the user's answer to the server 2000 . Transmission of the user's answer to the suggestion may be performed only when the user responds to the digital assistant's suggestion, but is not limited thereto.

In operation S733 , the server 2000 may request factor information for determining the indication information from the vehicle 1000 . The factor information may include information on factors required by the server 2000 to determine a function to be performed in the vehicle 1000 . For example, the factor information necessary to determine the function to be performed in the vehicle 1000 is, for each vehicle, such as an adjustable temperature range, an adjustable air volume range, whether to support cooling of each seat, and support for heating of each seat, etc. Alternatively, it may be information on various hardware configurations that are different for each option of the vehicle. In the indication information determined by the server 2000, settable ranges such as temperature, air volume, cooling, heating, heating wire, and cooling sheet may be limited by the printing information.

In operation S735 , the vehicle 1000 may transmit a response to the request for factor information from the server 2000 to the server 2000 . That is, the vehicle 1000 may obtain the corresponding factor information corresponding to the factor information requested from the vehicle configuration information and transmit it to the server 2000 .

In operation S736 , the server 2000 may determine instruction information indicating a function to be performed in the vehicle 1000 based on the response transmitted from the vehicle 1000 .

In operation S738 , the server 2000 may transmit instruction information to the vehicle 1000 . In one embodiment, operation S738 may be performed together with operation S714, and operations S733, S735, and S736 may be performed before operation S714 so that operations S714 and S738 may be performed together. In an embodiment, operation S733 may be performed together with operation S714, or may be performed before operation S714.

In operation S740 , the vehicle 1000 may perform a function based on the indication information.

According to an embodiment, vehicle configuration information is stored in the vehicle 1000, and when the server 2000 needs to refer to the vehicle configuration information, it only requests a part necessary for the vehicle 1000, so the user's privacy is not violated. can be suppressed.

8 is a block diagram of a device according to an embodiment.

Device 800 or software running on device 800 may perform the acts or steps of methods described or depicted herein, and may provide functionality described or depicted herein. Reference to device 800 may include one or more devices 800 as appropriate. One or more devices 800 may perform an operation or step of a method described or depicted herein without substantial spatial or temporal constraints. One or more devices 800 may perform an operation or step of the method in real-time or batch mode.

The device 800 may be a vehicle, mounted on the vehicle, or connected to the vehicle. The device 800 may be mounted on various end user devices, such as mobile devices such as smartphones, tablets, and laptops, wearable devices such as smart watches, smart bands, and smart glasses, refrigerators, televisions, and home appliances such as speakers, as well as vehicles. can A digital assistant may also be referred to as a voice assistant.

The device 800 may include a processor 830 and a memory 870 . Software modules, eg, program modules, of the device 800 may be stored in the memory 870 as a set of instructions, and the instructions may be executed by the processor 830 to perform corresponding functions. Device 800 may include more or fewer components than the number of components described above. For example, referring to FIG. 8 , the device 800 may include an input unit 810 , an output unit 820 , a processor 830 , a communication interface 850 , a microphone 860 , and a memory 870 . can

The processor 830 of the device 800 may include an AI processor 830 for generating a learning network model, but is not limited thereto. According to an embodiment, the AI processor 830 may be implemented as a chip separate from the processor 830 . The processor 830 may perform each operation of the method of the above-described embodiments.

The input unit 810 of the device 800 may receive a user input. The input unit 810 may include a unit for interacting with a user. For example, the input unit 810 may include a key pad, a dome switch, and a touch pad (contact capacitive method, pressure resistance film method, infrared sensing method, surface ultrasonic conduction method, integral type). tension measurement method, piezo effect method, etc.), a touch screen, a jog wheel, a jog switch, etc., but is not limited thereto.

The output unit 820 of the device 800 may output information to be provided to the user, such as an audio signal, a video signal, or a vibration signal. The output unit 820 may include a display unit, a sound output unit, a vibration motor, and the like. The device 800 may provide a suggestion to the user through the output unit 820 . The offer may provide the offer to the user in response to detecting the various conditions described above, regardless of receiving the user's call instruction. The suggestion may be provided to the user as a voice through the audio output unit. The suggestion may be provided to the user through the display unit. Suggestions may be presented to the user via voice and display. The offer may provide the offer to the user in response to detecting the various conditions described above, regardless of receiving the user's call command.

The sound output unit outputs audio data received from the communication interface 850 or stored in the memory 870 . Also, the sound output unit outputs a sound signal related to a function (eg, a call signal reception sound, a message reception sound, and a notification sound) performed by the device 800 . The sound output unit may include a speaker, a buzzer, and the like.

The vibration motor may output a vibration signal. For example, the vibration motor may output a vibration signal corresponding to the output of audio data or video data (eg, a call signal reception sound, a message reception sound, etc.). Also, the vibration motor may output a vibration signal when a touch is input to the touch screen.

The output unit 820 may provide an interactive interface. The interactive interface may be provided in the form of a message window or a chatting window, or may be provided in the form of input/output of voice, but is not limited thereto.

The communication interface 850 of the device 800 may include one or more components that allow communication between the device 800 and other devices, such as a server. For example, the communication interface 850 may include a short-range communication unit, a mobile communication unit, and a broadcast receiving unit.

Short-range wireless communication unit, Bluetooth communication unit, BLE (Bluetooth Low Energy) communication unit, near field communication unit (Near Field Communication unit), WLAN (Wi-Fi) communication unit, Zigbee communication unit, infrared (IrDA) It may include a data association) communication unit, a Wi-Fi Direct (WFD) communication unit, an ultra wideband (UWB) communication unit, an Ant+ communication unit, and the like, but is not limited thereto.

The mobile communication unit transmits/receives a radio signal to and from at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to transmission/reception of a voice call signal, a video call signal, or a text/multimedia message.

The broadcast receiver receives a broadcast signal and/or broadcast-related information from the outside through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel.

The microphone 860 receives an external sound signal and processes it as electrical voice data. For example, the microphone 860 may receive an acoustic signal from an external device or a speaker. The microphone 860 may use various noise removal algorithms for removing noise generated in the process of receiving an external acoustic signal. The device 800 may receive a user's natural language command and a natural language answer through the microphone 860 .

The memory 870 may store a program for processing and control of the processor 830 or may store input/output data. Programs stored in the memory 870 may be classified into a plurality of modules according to their functions.

Functions related to artificial intelligence according to the present disclosure are operated through the processor 830 and the memory 870 . The processor 830 may include one or a plurality of processors 830 . At this time, one or a plurality of processors 830 are general-purpose processors 830 such as CPUs, APs, Digital Signal Processors (DSPs), etc., GPUs, graphics-only processors 830 such as VPUs (Vision Processing Units), or artificial ones such as NPUs. It may be an intelligence-only processor 830 . One or more processors 830 control to process input data according to a predefined operation rule or artificial intelligence model stored in the memory 870 . Alternatively, when one or more processors 830 are the AI-only processors 830, the AI-only processor 830 may be designed with a hardware structure specialized for processing a specific AI model. For example, the AI-only processor 830 may be designed as a hardware structure specialized for a digital assistant.

The predefined action rule or artificial intelligence model is characterized in that it is created through learning. Here, being made through learning means that the basic artificial intelligence model is learned using a plurality of learning data by a learning algorithm, so that a predefined action rule or artificial intelligence model set to perform a desired characteristic (or purpose) is means to be made Such learning may be performed in the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and/or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

Methods according to the embodiments may be implemented in the form of hardware, software, or a combination of hardware and software. When the embodiment is implemented using software, one or more programs (software, software module) may be used. The program may be included in a computer-readable recording medium, but is not limited thereto. The program may be included in a computer program product. A computer-readable recording medium storing the program may be included in the computer program product. The program is configured to be executable by one or more processors in the device. The one or more programs may include instructions, which may be executed by one or more processors included in the device to cause the device to perform methods according to embodiments.

Here, computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and non-volatile media, removable and non-removable media. Also, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media. Also, some embodiments may be implemented as a computer program or computer program product comprising instructions executable by a computer, such as a computer program executed by a computer. A plurality of computer-readable recording media may be distributed in network-connected computer systems, and data stored in the distributed recording media, for example, program instructions and code may be executed by at least one computer. have.

Functions of the various elements shown in the drawings may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, such functionality may be provided by a single dedicated processor, a single shared processor, or a plurality of separate processors, some of which may be shared.

Here, terms such as “unit” and “module” mean a unit that processes at least one function or operation, which is a hardware component such as a processor or circuit, or a software component, or a combination of a hardware component and a software component. It can be implemented as a combination.

As used herein, terms such as "processor", "controller", or "controller" should not be construed to exclusively refer to hardware capable of executing software, and without limitation, digital signal processor (DSP) hardware, storage software, and the like. read only memory (ROM), random access memory (RAM), and non-volatile storage devices for

Here, an expression such as “at least one” modifies the entire list of components, and does not individually modify components of the list. For example, "at least one of A, B, and C" means only A, only B, only C, both A and B, both B and C, both A and C, all of A and B and C, or a combination thereof. points to

Here, the expression that a certain element is "connected" with another element may indicate a case in which they are "directly connected" to each other, as well as a case in which they are "electrically connected" through another element.

Here, the operations or steps constituting the method according to the embodiment may be performed in an appropriate order unless the order thereof is described or there is no description to the contrary, and the order in which they are described is not limited to the embodiment. The use of all examples or exemplary terminology (eg, etc.) is merely used to describe the embodiments in detail, and the scope of the present disclosure is not limited by the examples or exemplary terminology unless defined by the claims. . For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present disclosure is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be interpreted as being included in the scope of the present disclosure. do.

Claims (20)

detecting a suggested triggering condition in the vehicle;
providing a suggestion to a user based on the detected suggestion triggering condition;
receiving the user's answer to the suggestion; and
and performing a function of the vehicle based on the answer of the user and vehicle configuration information of the vehicle. The method of claim 1,
The vehicle configuration information is stored in an external device. 3. The method of claim 2,
Further comprising the step of transmitting the identification information of the vehicle to the external device,
The vehicle configuration information of the vehicle is identified based on the identification information of the vehicle. 3. The method of claim 2,
Further comprising the step of receiving the indication information determined based on the vehicle configuration information from the external device,
and the function of the vehicle is performed based on the indication information. The method of claim 1,
Further comprising the step of transmitting information on the proposed triggering condition to an external device,
The providing of the proposal includes providing the proposal to the user by receiving the proposal generated in the external device based on the detected proposal triggering condition. The method of claim 1,
and receiving, from the external device, the indication information determined by an external device based on the vehicle configuration information, and the proposal generated in the external device together. The method of claim 1,
wherein the vehicle configuration information is stored in the vehicle. 8. The method of claim 7,
receiving a request for factor information from an external device;
identifying corresponding factor information corresponding to the factor information in the vehicle configuration information and transmitting the corresponding factor information to the external device; and
Further comprising the step of receiving the indication information determined based on the correspondence factor information from the external device,
and the function of the vehicle is performed based on the indication information. The method of claim 1,
The vehicle configuration information includes information on at least one of an adjustable temperature range, an adjustable air volume range, whether each seat supports cooling, and a heating support for each seat. The method of claim 1,
Further comprising the step of detecting a suggested triggering disturbance condition in the vehicle,
The step of providing the proposal includes providing the proposal based on the detected proposal triggering condition and the proposal triggering interruption condition. The method of claim 1,
The detecting of the suggested triggering condition includes detecting the suggested triggering condition based on an environment of the vehicle. 12. The method of claim 11,
wherein the environment of the vehicle includes an internal environment of the vehicle and an external environment of the vehicle. 12. The method of claim 11,
The environment of the vehicle includes at least one of a temperature, a state of consumables of the vehicle, a state of a power source of the vehicle, a location of the vehicle, and whether the vehicle is parked. The method of claim 1,
The detecting of the suggestion triggering condition includes detecting the suggestion triggering condition based on user profile information. 15. The method of claim 14,
The user profile information includes user profile information of each of a plurality of users including the user. The method of claim 1,
The detecting of the suggestion triggering condition includes detecting the suggestion triggering condition based on a routine of the user in the vehicle. The method of claim 1,
The suggestion triggering condition and the suggestion are learned based on the user's answer to the suggestion. a memory for storing instructions; and
By executing the above instructions:
detecting a suggested triggering condition in the vehicle,
provide a suggestion to the user based on the detected suggestion triggering condition;
receiving the user's reply to the proposal;
and at least one processor configured to perform a function of the vehicle based on the answer of the user and vehicle configuration information of the vehicle. 18. The method according to any one of claims 1 to 17,
A computer-readable recording medium recording a program for executing the method in a computer. receiving a suggestion provided to a user based on a suggestion triggering condition detected in the vehicle, and a response of the user to the suggestion;
determining instruction information for causing the vehicle to perform a function of the vehicle based on the answer of the user and vehicle configuration information of the vehicle; and
transmitting the indication information to the vehicle.

Images