KR102322863B1 - Apparatus and method for providing information based on a current task of a user - Google Patents

Abstract

An information providing apparatus and method for providing information based on a user's current performance are provided. Determine at least one sensory channel to use for providing information based on at least one active sense of the user. This improves the user's reaction time.

Description

Apparatus and method for providing information based on the user's current task

The present invention relates to an apparatus and method for providing information based on a user's current task, and more particularly, to a Human Machine Interface (HMI) device that provides information in a way that a user such as a driver of an autonomous vehicle can recognize well; it's about how

According to Society of Automotive Engineers (SAE) J3016, Automatic Driving System (ADS) is divided into five levels, from level 0, which is fully manual driving, to level 5, which is fully autonomous driving, depending on the role of the driver. At this time, in Level 2 (Partially Autonomous Driving) and Level 3 (Conditional Autonomous Driving) ADS, which are levels where the driver and the ADS alternately control the vehicle, it is important to safely transfer control of the vehicle between the driver and the ADS. If it fails, it can lead to an accident.

During autonomous driving, drivers can perform non-driving related tasks (NDRTs). Accordingly, the transfer of control from the ADS to the driver performing a task other than driving may be more problematic than the transfer of control from the driver to the ADS. This is because it takes time for the driver to recognize and respond to a Take Over Request (TOR) from ADS, and it takes time until the driver can drive stably even after gaining control.

The United Nations Economic Commission for Europe (UNECE) Automatically Commanded Steering Function (ACSF) recommends that, when an ADS attempts to hand over control of a vehicle to the driver, the ADS must maintain lateral control of the vehicle for 4 seconds after the occurrence of the TOR. . The driver should recognize the TOR and start controlling the vehicle within 4 seconds, but in some cases 4 seconds may not be enough.

For example, during autonomous driving, the driver can listen to music, enjoy the scenery, talk, eat, put on makeup, and sleep. It can be difficult to perceive and respond to that information within a given threshold of seconds. This delay in response depends on age, the type of NDRT the driver is performing, and the level of NDRT immersion.

Therefore, there is a need for a technology that can effectively provide urgent or important information, such as a request to switch control, to a user in a manner that minimizes response delay in any situation.

In order to solve the above-mentioned problem, the present invention is to provide an apparatus and method for providing information based on a user's current task.

Another object of the present invention is to provide an apparatus and method for providing information through an active sense with a high degree of user's attention.

The above and other objects, advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings.

The information providing apparatus according to the present invention includes a recognition unit that acquires driver state information of an autonomous vehicle, a task determiner that determines a current task of a driver based on the driver state information, and at least a and an information providing unit that determines one sensory channel and provides the autonomous driving-related information to the driver through the at least one sensory channel.

The information providing method according to the present invention comprises the steps of determining a current performance task of a driver based on driver state information of an autonomous vehicle, determining at least one active sense of the driver according to the current performance task, and the at least one and determining at least one sensory channel to provide autonomous driving-related information based on the active sense of the .

In addition, according to the present invention, determining at least one active sense of the user according to the current performing task of the user to be provided with information according to the present invention, at least one sensory channel to be used for providing information based on the at least one active sense determining; and providing information to the user through the at least one sensory channel.

According to the present invention, information can be provided through the senses activated according to the currently performed task of the information user.

In addition, it is possible to provide information quickly and effectively by determining the target of the current user's attention.

Finally, the present invention provides an information providing apparatus and method capable of delivering information by reducing or minimizing a user's response delay.

1 shows a configuration diagram of an information providing interface system 100 according to an embodiment of the present invention.
2 is a block diagram illustrating an information providing apparatus 120 according to an embodiment of the present invention.
3 is a flowchart illustrating a process for determining an operation to be performed according to an embodiment of the present invention.
4 is a flowchart illustrating a process of determining a sensory channel according to an embodiment of the present invention.
5 is a diagram showing an active sensory model according to an embodiment of the present invention.
6 is a flowchart illustrating an information providing method according to an embodiment of the present invention.

Hereinafter, the configuration of the present invention will be described in detail through preferred embodiments with reference to the accompanying drawings, but the idea of the present invention will be schematically described first for the purpose of enhancing understanding and convenience of explanation.

When humans pay attention to and immerse themselves in a specific task, the sensitivity of the sensory organs necessary to perform the task increases, while the sensitivity of the sensory organs that are not used (inactivated) decreases. For example, if you are talking to someone who is reading a book with concentration, you may not be able to recognize the content of the story well. In the case of concentrated reading, sight is activated, while other senses of hearing, touch, and smell are relatively inactive. On the other hand, human attention tends to be concentrated on the object currently being used or related to the use of the surrounding objects.

Therefore, it is effective to transmit information according to the current task, the situation or the moment, using the object occupying the attention through the sensory channel corresponding to the activated sense.

As a result, determining the active sensory, high-attention-focused sensory channel and/or attention-occupying object for the current performance task can provide information through this, so that no matter what performance the human being performs, i.e., any action or action. It can effectively convey information even when it is in a state or in a state. Accordingly, a delay in a user's response to information provision can be minimized.

The aspect in which the present invention is implemented will be described with reference to each preferred embodiment below. It is obvious that the present invention is not limited to the embodiments disclosed below, but may be implemented in other various forms within the scope of the technical spirit of the present invention. The terminology used herein is also for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprise” and/or “comprising” means that the stated component, step, action and/or element is the presence of one or more other components, steps, actions and/or elements. or added.

1 shows a configuration diagram of an information providing interface system 100 according to an embodiment of the present invention.

The information providing apparatus 120 according to an embodiment of the present invention recognizes the status information of the user 110, determines the current operation of the user 110 according to the recognized status information of the user 110, and A method suitable for delivering information to the user 110 is determined according to the result of analyzing the performed task, and information required to be delivered is provided to the user 110 using the method. In one example, determining a suitable manner to convey information to the user 110 may correspond to determining at least one of a device 130 to use to convey the information and a sensory channel to use to convey the information. have.

Specifically, the information providing apparatus 120 according to an embodiment of the present invention recognizes the state of the user 110, and based on the user 110's currently performed task, the user (110) in a manner suitable for the user (110). 110) to provide information that needs to be transmitted.

For example, when the user 110 is performing an e-book reading operation using the portable terminal 130 , the information providing device 110 selects the portable terminal 130 , which is a target that occupies the attention of the user 110 . It can be used to provide information that needs to be transmitted. That is, the information providing device 120 may display a pop-up on the screen of the portable terminal 130 to visually provide information that needs to be delivered. In one example, the information providing apparatus 110 may provide information that needs to be transmitted aurally using a speaker of the portable terminal 130 .

In one example, user 110 may be a driver or occupant of an autonomous vehicle.

The information providing device 120 may provide the user 110 with information that the user 110 needs to recognize. As an example, the information providing device 120 autonomously gives the driver 110 of the autonomous vehicle. It can provide driving-related information. Here, the autonomous driving-related information may include a control right switching request (TOR).

In one example, the information providing device 120 includes an HMI device that transmits autonomous driving-related information from the autonomous driving vehicle to the driver 110 .

The device 130 to be used to transmit information includes a device capable of transmitting sensory stimuli, including visual, auditory, olfactory, and tactile, to the user 110 , and may be, for example, a vehicle-mounted device or a portable device.

Hereinafter, an exemplary detailed configuration of the information providing apparatus 120 will be described with emphasis on the aspect of providing autonomous driving-related information to the driver of the autonomous driving vehicle with reference to FIG. 2 .

2 is a block diagram illustrating an information providing apparatus 120 according to an embodiment of the present invention.

The information providing apparatus 120 includes a recognition unit 210 .

The recognition unit 210 recognizes ( 310 ) the state of the driver 110 by collecting current state information of the driver 110 with reference to FIG. 3 .

The current state information of the driver 110 may include an image, an audio, and a biosignal. For example, the current state information may include image information of the vehicle and the driver 110 , sound information inside the vehicle, voice information of the driver 110 , and biosignals of the driver 110 .

The recognition unit 210 may include a biosignal collecting unit, a portable device information collecting unit, a vehicle device information collecting unit, an image collecting unit, and a voice collecting unit. For example, the recognition unit 210 may include a portable human machine interface (HMI) device such as a smartphone, smart watch, or laptop that the driver can carry or move and use, an audio video system mounted on the vehicle center fascia, and a front windshield. It may include a vehicle-mounted HMI device such as a HUD (Head Up Display) device installed on the shield glass, an image sensor mounted on the vehicle to collect an image of the driver 110 , and a biometric sensor that collects a biosignal of the driver 110 . have. The image sensor may be at least one of a visible light region image sensor for general information collection and an infrared image sensor for gaze detection, and the biometric sensor includes ECG, EEG, PPG, GSR, and respiratory rate sensors, and voice The collection unit may be a voice sensor, but is not limited thereto, and of course includes various devices that perform the same or similar function in the present or future.

The recognition unit 210 analyzes the physiological load analyzer that analyzes the physiological load of the driver 110 by analyzing the biological signal collected by the biological signal collection unit, analyzes the image collected by the image collection unit, and analyzes the image collected by the driver ( 110) the driver's body recognition unit for recognizing the head position, head direction, gaze, mouth shape, hand position, arm position, trunk position, foot position, etc., to detect the portable device of the driver 110 from the collected images A target point that detects the gaze of the driver 110 from the image collected by the beta information recognition unit and the image collection unit that recognizes the beta information of the driver 110 and occupies the gaze using information such as the gaze direction and location. It includes a gaze recognition unit for recognizing , a portable device information collection unit for collecting work information being performed from the portable HMI device, and a vehicle device information collecting unit for collecting work information being performed from the vehicle-mounted HMI device.

The information providing device 120 includes a task determination unit 220 . The performing task determining unit 220 determines the current performing task of the driver 110 based on the current state information of the driver 110 recognized by the recognition unit 210 .

The currently performed task is a generic term for the current operation, behavior, and state of the driver 110 . For example, the currently performed task may be a driving action of the driver 110 of the autonomous vehicle, or a task other than driving (NDRT) being performed during autonomous driving, such as listening to music, watching a movie, watching a scene, or playing a game. , talking, video calling, reading, writing, using a smartphone, eating, drinking, putting on makeup, dressing, cleaning, washing, sleeping, doing nothing. In other words, the performance task encompasses not only a certain intentional action or action, but also a person's present state (including the state of doing nothing), unconscious action, and reflex action.

A process in which the to-be-executed task determining unit 220 determines the current performing task of the driver 110 will be described in more detail with reference to FIG. 3 .

The information providing device 120 includes an information providing unit 230 .

The information providing unit 230 provides information such as autonomous driving related information to the driver 110 based on the current performed task determined by the performing task determining unit 220 . Here, the information providing unit 230 determines at least one sensory channel based on the currently performed task determined by the performing task determining unit 220 , and provides information to the driver 110 through the determined at least one sensory channel. .

The information providing unit 230 includes an active sensory determining unit 232 .

The active sensory determination unit 232 determines at least one active sense of the driver 110 by analyzing the current performed task determined by the performing task determining unit 220 .

The active sense may correspond to a sense used by the driver 110 to perform a currently performed task or a sense activated while the driver 110 is currently performing a task. For example, active senses may include sight, hearing, bodily (tactile), cognition, and the like.

The active sensory determining unit 232 may determine the sensitivity or activity of the activated sense for each active sense.

In one example, referring to FIG. 1 , when the driver 110 of the autonomous vehicle is reading an e-book with the portable terminal 130, the senses of the driver 110 are activated, so that the visual activity is high, Hearing not used for reading is inactivated, so the activity of hearing is relatively low. In this case, the performance determination unit 220 may determine to read the operation performed by the driver 110 , and the active sensory determination unit 232 may determine that the visual activity is high and the auditory activity is low.

The information providing unit 230 includes a sensory channel determining unit 234 . The sensory channel determiner 234 determines at least one sensory channel to provide information to the driver 110 based on the at least one active sense determined by the active sensory determiner 232 .

At least one sensory channel corresponds to a sensation to use to provide information to the driver 110 . In one example, the at least one sensory channel may correspond to the at least one active sensation determined by the active sensory determiner 232 . For example, sensory channels may include visual, auditory, bodily (tactile), and cognitive channels, and the like.

The information providing unit 230 includes an attention occupancy determining unit 236 .

The attention occupancy determining unit 236 determines an object occupying the driver's attention based on the driver state information recognized by the recognition unit 210 . For example, the attention occupation determination unit 236 may determine the portable terminal 130 as an object occupying the attention of the driver 110 of the autonomous driving vehicle with reference to FIG. 1 based on the driver state information. .

A detailed operation process of the information providing unit 230 according to an example will be described below with reference to FIG. 4 .

3 is a flowchart illustrating a process for determining an operation to be performed according to an embodiment of the present invention.

In step 310 , the recognition unit 210 recognizes state information of the driver 110 with reference to FIG. 2 .

In step 320 , with reference to FIG. 2 , the task determining unit 220 analyzes the state information of the driver 110 to determine the expected task of the driver 110 .

In step 320 , the to-be-executed task determining unit 220 determines the expected task of the driver 110 by analyzing driver state information based on the performing task model 222 with reference to FIG. 2 . In one example, the to-do model 222 may be a preset model. In one example, the performance model 222 may be built by learning.

The performing task model 222 is a model constructed by analyzing state information of the driver 110 when the driver 110 performs a specific task. For example, the to-do model 222 may include portable HMI device activation information, vehicle-mounted HMI device activation information, driver's gaze direction, gaze position, driver's head position, head direction, mouth shape, hand position, arm position, and torso. It is possible to observe the position, the position of the foot, and the driver's physiological load, and to respond to the model constructed by classifying such information by task. For example, the performing task model 222 performs the eating task in a state in which the mouth shape is repeatedly opened or closed, the position of the hand is repeatedly moved around the mouth, and the position of the head is repeatedly moved up and down. can be defined

The task determining unit 220 compares the result of analyzing the driver state information recognized by the recognition unit 210 with the state information for each task of the performing task model 222 to determine the matching task of the driver 110 . Decide on the expected work to be performed.

For example, the driver state information recognized by the recognition unit 210 indicates that the shape of the mouth is repeatedly opened or closed, the position of the hand is repeatedly moved around the mouth, and the position of the head is repeatedly moved up and down. In the case of including, the performing task determining unit 220 may determine the expected performing task of the driver 110 as the eating performing task. As another example, in the driver state information recognized by the recognition unit 210 , the driver 110's head position has little change, the driver 110's gaze direction is toward the smart phone, and the hand position is around the smart phone. In the case of including a state located in , the to-be-performed task determining unit 220 may determine to use the smart phone for the expected performing task of the driver 110 . As another example, when the driver 110's gaze cannot be detected, the body movement is low, and the physiological signal load is particularly low, the to-be-executed task determining unit 220 sets the driver's expected task to sleep. can be decided with

In step 330 , the task determining unit 220 repeats steps 310 and 320 until N predicted tasks are determined. In one example, N may be an integer greater than zero. For example, N may be a predetermined value. In one example, N may be determined by learning.

In step 330, N predicted tasks are stored in a queue. The to-be-performed task determining unit 220 may store N expected to be performed in order in the queue, and when there is no more space to store the expected to be performed in the queue, it deletes the oldest expected to be performed and stores a new expected to be performed. . For example, the queue may be a circular queue.

In step 340 , the to-be-performed task determining unit 220 _{calculates a probability P i that each to-be-performed task i} is the currently performed task of the driver 110 with respect to each of the N predicted tasks determined in step 330 (i is integer less than or equal to N). The probability P _i may correspond to a ratio in which the to-be- _{performed task i} is determined as the expected to-be-performed task with respect to the recent N predicted tasks. In determining the task to be performed, since the result of determining the expected task N times is used, it is possible to filter out cases of momentary erroneous judgment and to increase the accuracy of the determination of the current task.

In step 350, if the probability P _i determined in step 340 is equal to or greater than the threshold P _Threshold , in step 360, the driver 110 determines that the currently performed task _i is being performed. . In one example, in step 360 , the to-be-performed task determining unit 220 determines at least one currently performed task. _{In one example, if the probability P i} determined in step 340 is less than P _Threshold , the to-be-performed task determining unit 220 may determine that the driver 110 does not currently perform the to-be- _{performed task i in step 370 .} In one example, P _Threshold may be a predetermined value. In one example, N may be determined by learning.

4 is a flowchart illustrating a process of determining a sensory channel according to an embodiment of the present invention.

In step 410 , with reference to FIG. 2 , the active sensory determining unit 232 , based on the current performing task determined by the performing task determining unit 220 in step 360 , at least one active sense of the driver 110 . to decide In step 410 , the active sensory determination unit 232 determines at least one sense used for performing the currently performed task as at least one active sense for the currently performed task by analyzing the currently performed task.

In step 420 , the active sensory determining unit 232 analyzes the currently performed task based on a preset active sensory model, and determines the activity of at least one active sensory determined in step 410 . In one example, the active sensory model 238 may be a preset model. In one example, the active sensory model 238 may be built by learning.

Hereinafter, the active sensory model 238 will be described with reference to FIG. 5 .

5 shows an exemplary active sensory model 238 . Active sensory model 238 includes a list of tasks to be performed. The active sensory model 238 includes information on at least one active sense used to perform a corresponding performance task for each performance task. In one example, the active senses may include sight, hearing, bodily (tactile) sense, and cognition.

Referring to FIG. 5 , the active sensory model 238 includes information on the activity level of each active sensory. The active sensory model 238 classifies the activity levels of each active sensory step by step or quantifies them in various ways and provides them. For example, the active sensory model 238 of FIG. 5 divides the activity of each active sense into four levels of very low, low, medium, and high according to the type of task to be performed. According to the active sensory model 238 of FIG. 5 , when music appreciation is performed, visual and cognitive activity are low, auditory activity is medium, and body sensory activity is very low. In one example, the activity of each active sensory of the active sensory model 238 may be a predetermined value. In one example, the activity of each active sense of the active sensory model 238 may be determined by learning.

4, in step 430, the sensory channel determiner 234 with reference to FIG. 2, based on the activity of at least one active sensory determined by the active sensory determiner 232 in step 420, , determine the attention-occupying activity of the corresponding at least one active sense.

The attention-occupying activity corresponds to the driver's 110's concentration of attention for each active sensation. The active sensory determining unit 232 determines the attention-occupying activity for each active sense of at least one active sense. The attention-occupying activity of the at least one active sense varies according to the type of the currently performed task, the number of the currently performed task, and the age of the driver 110 . In one example, the attention-occupying activity of at least one active sense may be increased in proportion to the demand for cognitive power required to perform the performance task.

In one example, the attention-occupying activity of the at least one active sense may be determined according to Equation 1 below.

[Equation 1]

Visual attention-occupying activity = α ₁ V+β ₁ C+V ₀

Attention-occupied activity of hearing = α ₂ A+β ₂ C+A ₀

Tactile attention-occupying activity = α ₃ P+β ₃ C+P ₀

V: Visual activity presented by the active sensory model

A: Hearing activity presented by the active sensory model

P: Body (tactile) activity suggested by the active sensory model

C: Cognitive activity presented by the active sensory model

V ₀ : Visual activity when no action is taken

A ₀ : Hearing activity level when no operation is performed

P ₀ : Physical activity when no work is done

α ₁ : Visual attention occupancy factor

α ₂ : auditory attention occupancy factor

α ₃ : body attention occupancy factor

β _x : cognitive attention occupancy factor

In one example, visual activity, auditory activity, and physical activity when no work is performed may have a relationship _{of A 0} >V ₀ >P _{0 .} When information is provided through the auditory channel, information can be provided in all directions, but information provided through the visual channel can be provided only when the direction of the driver's 110 gaze and the direction of the visual channel coincide. This corresponds to a case that reflects the fact that information on content that can be provided as a channel is limited.

The visual attention occupation factor α ₁ corresponds to a factor indicating how much the driver 110 paid attention to the visual channel. In one example, the visual attention occupancy factor α ₁ may be proportional to the gaze occupancy time per unit time for the corresponding visual channel. In one example, the visual attention occupation factor α ₁ may be inversely proportional to the number of tasks that the driver 110 simultaneously performs.

The auditory attention occupation factor α ₂ corresponds to a factor indicating how much the driver 110 paid attention to the auditory channel. In one example, the auditory attention occupation factor α ₂ may be inversely proportional to the number of tasks that the driver 110 simultaneously performs.

The body attention occupation factor α ₃ corresponds to a factor indicating how much attention the driver 110 paid attention to the body channel. In one example, the body attention occupation factor α ₃ may be inversely proportional to the number of tasks that the driver 110 simultaneously performs.

The cognitive attention occupancy factor β _x corresponds to a factor indicating how high cognitive power the driver 110 used to process information provided by each channel. In an example, the cognitive attention occupancy factor β _x may be inversely proportional to the number of tasks that the driver 110 simultaneously performs. In an example, the cognitive attention occupation factor β _x may have a different value according to the age of the driver 110 .

In one example, the attention grabbing activity of the at least one active sense may be associated with a delay in response of the driver 110 to each active sense. In other words, the attention-occupying activity may be associated with a delay in response of the driver 110 to the provision of information through the at least one sensory channel. For example, as the attention-occupying activity of the sensory channel providing information is higher, the response delay of the driver 110 may be lower. This is because the sense of high attention-occupying activity corresponds to the sense of high concentration of attention of the driver 110 , and thus the driver 110 receiving the information through the corresponding channel can respond immediately.

In operation 440 , the sensory channel determiner 234 may determine at least one sensory channel based on the attention occupancy activity determined in operation 430 .

In step 440 , the sensory channel determiner 234 may determine, as at least one sensory channel, a sense corresponding to the occupancy activity of the corresponding attention when the attention occupancy activity determined in step 430 exceeds the maximum reference activity. .

In operation 440 , the sensory channel determiner 234 may determine all active sensations as at least one sensory channel when the attention occupancy activity determined in operation 430 is equal to or less than the minimum reference activity.

Referring to FIG. 2 , the information providing unit 230 provides autonomous driving related information to the driver 110 through at least one sensory channel determined by the sensory channel determiner 234 in step 440 .

6 is a flowchart illustrating an information providing method according to an embodiment of the present invention.

In step 610 , the recognition unit 210 recognizes driver state information with reference to FIG. 2 . Step 610 may correspond to step 310 with reference to FIG. 3 .

In step 620 , referring to FIG. 2 , the to-be-performed task determining unit 220 determines the current performing task of the driver 110 based on driver state information. Step 620 may correspond to steps 320 to 370 with reference to FIG. 3 .

Referring to FIG. 2 in step 630 , the information providing unit 230 determines at least one sensory channel based on the currently performed task. Step 630 may correspond to steps 410 to 440 with reference to FIG. 4 .

Referring to FIG. 2 in step 640 , the attention occupancy determining unit 236 determines an object occupying the attention of the driver 110 based on the driver state information recognized by the recognition unit 210 in step 610 . do.

In one example, in operation 640 , the attention occupation determining unit 236 may select a device occupying the driver's 110's gaze attention as the target occupying the driver's 110 attention. In one example, in operation 640 , the attention occupancy determiner 236 may determine, as an attention occupancy target of the driver 110 , a device having a ratio of occupying the driver's 110's gaze for a unit time equal to or greater than a threshold value.

In step 650 , the information providing unit 230 selects a device and a sensory channel to provide autonomous driving-related information to the driver 110 .

In one example, in step 650 , the information providing unit 230 is related to autonomous driving through a sensory channel that can be provided by the object occupying the attention determined in step 640 among the at least one sensory channel determined in step 630 . information can be provided.

In one example, when the at least one sensory channel determined in step 630 includes a visual channel in step 650 , the information providing unit 230 provides a visual interface to the attention occupant determined in step 640 . If so, autonomous driving-related information can be provided through the visual interface of the subject occupying the corresponding state. If the target of attention determined in step 640 does not provide a visual interface, it may be determined to provide autonomous driving-related information through a representative visual device.

In one example, when the at least one sensory channel determined in step 630 includes an auditory channel in step 650 , the information providing unit 230 provides an auditory interface to the attention occupant determined in step 640 . If so, it may decide to provide autonomous driving-related information through the auditory interface of the occupant of the state. If the target of attention determined in step 640 does not provide an auditory interface, it may be determined to provide autonomous driving-related information through the representative auditory device.

In one example, when the at least one sensory channel determined in step 630 includes a body channel in step 650 , the information providing unit 230 is autonomously in contact with the driver 110 through a device in physical contact. You may decide to provide driving-related information.

In one example, when it is determined in step 640 that there is no target of attention occupied in step 650 , the information providing unit 230 determines to provide autonomous driving-related information through at least one of representative visual, auditory, and bodily devices. can

In one example, in step 650 , the information providing unit 230 determines to provide autonomous driving-related information through at least one of representative visual, auditory, and bodily devices in addition to the attention-occupying target determined in step 640 . can

In step 660 , the information providing unit 230 checks whether there is information to be provided to the driver 110 by the autonomous driving system, and if there is, provides the corresponding information in step 670 .

For example, if it is determined in step 620 that the driver 110 is performing a movie watching operation with a smart phone, at least one sensory channel is determined as a visual channel and an auditory channel in step 630 , and in step 640 . ), the target of attention is determined by a smart phone, and in steps 650 to 670 , the information providing unit 230 provides visual and auditory information to the driver 110 through the visual and auditory channels of the smart phone. can provide Therefore, if the driver 110 is currently performing NDRT such as watching a movie with a smart phone, when an urgent request (eg, TOR, etc.) to be sent to the driver 110 with reduced sensitivity to driving tasks occurs, the By providing information to the driver 110 through the visual channel and the auditory channel, the driver 110 reaction delay can be minimized, and the driver 110 can recognize and respond to the request within a given threshold time. Accordingly, the safety of autonomous driving may be improved.

In one example, in step 670 , the information providing unit 230 may temporarily block information other than the information to be provided.

In one example, the information providing process shown in FIG. 6 may be repeatedly performed.

In one example, step 640 may be performed prior to steps 620 and 630 or performed concurrently with steps 620 and 630 in parallel.

In one example, the task to be performed may be determined in operation 630 based on the target of attention determined in operation 640 .

As described above, the description has been made on the assumption that autonomous driving-related information is provided to the driver of the autonomous driving vehicle, but the technical spirit of the present invention is not limited to the information providing method according to each of the above-described embodiments.

An embodiment provides an information providing method for providing information in an appropriate manner according to the task currently being performed by the user 110 in an environment requiring information provision to the user 110 . determining at least one active sense of the user according to the current performance task of the user 110 to be provided with information, determining at least one sensory channel to be used to provide information based on the at least one active sense; and providing information to the user 110 through at least one sensory channel.

In one example, the user 110 may include a subject capable of recognizing information. For example, the user 110 may be a person. In another example, the user 110 may be a robot.

In one example, the information providing device 120 includes a general information providing device. For example, the information providing device 120 includes an electronic billboard, a kiosk, or a smart home controller.

Meanwhile, the information providing apparatus and method according to an embodiment of the present invention may be implemented in a computer system or recorded in a recording medium. The computer system may include at least one processor, a memory, a user input device, a data communication bus, a user output device, and storage. Each of the above-described components performs data communication through a data communication bus.

The computer system may further include a network interface coupled to the network. The processor may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in a memory and/or storage.

The memory and storage may include various types of volatile or non-volatile storage media. For example, memory may include ROM and RAM.

Accordingly, the information providing method according to the embodiment of the present invention may be implemented as a computer-executable method. When the information providing method according to an embodiment of the present invention is performed in a computer device, computer readable instructions may perform the information providing method according to the present invention.

Meanwhile, the information providing method according to the present invention described above may be implemented as a computer readable code on a computer readable recording medium. The computer-readable recording medium includes any type of recording medium in which data that can be read by a computer system is stored. For example, there may be a read only memory (ROM), a random access memory (RAM), a magnetic tape, a magnetic disk, a flash memory, an optical data storage device, and the like. In addition, the computer-readable recording medium may be distributed in computer systems connected through a computer communication network, and stored and executed as readable codes in a distributed manner.

So far, the present invention has been looked at mainly in the embodiments. Those of ordinary skill in the art to which the present invention pertains will understand that the present invention may be implemented in variously changed or modified forms without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated by the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

120: information providing device
210: recognition unit
220: work determination unit
230: information provider
232: active sensory determining unit
234: sensory channel determining unit
236: state occupation decision unit

Claims (16)

a recognition unit for acquiring driver state information of the autonomous vehicle;
a to-be-performed task determining unit that analyzes the driver state information based on a preset to-be-performed work model to determine an expected to be performed of the driver, and to determine the current to be performed based on the expected to be performed; and
An information providing unit that determines at least one sensory channel based on the currently performed task and provides the autonomous driving-related information to the driver through the at least one sensory channel,
The driver state information includes image information of the vehicle and the driver, sound information inside the vehicle, voice information of the driver, and bio-signals of the driver,
The information providing apparatus, characterized in that the currently performed task includes a driver's driving of the autonomous vehicle and a task other than driving (NDRT) performed during autonomous driving.
delete The method of claim 1,
The task determination unit,
and determining the current performance of the driver by analyzing a set of the expected to-be-performed tasks.
The method of claim 1,
The information providing unit,
an active sense determining unit configured to determine at least one active sense of the driver based on the currently performed task; and
a sensory channel determining unit configured to determine the at least one sensory channel through which the driving-related information is to be provided based on the at least one active sense;
An information providing device comprising a.
5. The method of claim 4,
The active sensory determining unit determines the activity level of the at least one active sensory by analyzing the currently performed task based on a preset active sensory model;
The sensory channel determining unit,
and determining the at least one sensory channel based on the attention-occupying activity of the at least one active sense determined based on the activity level.
6. The method of claim 5,
The sensory channel determining unit,
and when the attention-occupying activity exceeds a maximum reference activity, determining a sensation corresponding to the attention-occupying activity as the at least one sensory channel.
6. The method of claim 5,
The sensory channel determining unit,
and when the attention-occupying activity is equal to or less than a minimum reference activity, determining all of the active senses as the at least one sensory channel.
6. The method of claim 5,
and the attention-occupying activity is associated with a delay in response of the driver to the provision of the driving-related information.
The method of claim 1,
The information providing unit,
an attention occupation determining unit configured to determine an object occupying the driver's attention based on the driver state information;
An information providing device comprising a.
10. The method of claim 9,
The information providing unit,
An information providing apparatus for providing the driving-related information through a sensory channel provided by an object occupying the attention among the at least one sensory channel.
determining a currently performed task of a driver based on driver state information of the autonomous vehicle;
determining at least one active sense of the driver according to the currently performed task; and
determining at least one sensory channel to provide autonomous driving-related information based on the at least one active sensory;
The driver state information includes image information of the vehicle and the driver, sound information inside the vehicle, voice information of the driver, and bio-signals of the driver,
The method for providing information, characterized in that the currently performed task includes a driver's driving behavior of the autonomous vehicle and a task other than driving (NDRT) performed during autonomous driving.
12. The method of claim 11,
determining an attention occupant target of the driver based on the driver state information; and
providing autonomous driving-related information to the driver through a sensory channel provided by the attention-occupying target among the at least one sensory channel
A method of providing information further comprising a.
12. The method of claim 11,
Determining the at least one active sense comprises:
determining the activity of the at least one active sense;
including,
Determining the at least one sensory channel comprises:
determining an attention-occupying activity of the at least one active sense based on the activity of the at least one active sense;
A method of providing information, including
14. The method of claim 13,
Determining the at least one sensory channel comprises:
When the attention-occupying activity exceeds a maximum reference activity, determining the sensation corresponding to the attention-occupying activity as the at least one sensory channel.
determining at least one active sense of the user according to a current task determined based on status information of the user to be provided with information of the autonomous vehicle;
determining at least one sensory channel to use for providing information based on the at least one active sensory; and
providing information to the user through the at least one sensory channel;
The user status information includes image information of the vehicle and the user, sound information inside the vehicle, voice information of the user, and biosignals of the user,
The method for providing information, characterized in that the currently performed task includes a user's driving of the autonomous vehicle and a task other than driving (NDRT) performed during autonomous driving.
16. The method of claim 15,
Determining the target of the user's attention based on the current state information
further comprising,
Providing the information includes:
The method of providing information to the user through a sensory channel provided by the attention-occupying object among the at least one sensory channel.

Images