KR102355423B1 - Apparatus and method for controlling multi-media system equipped in vehicle - Google Patents

Abstract

The present invention relates to the steps of determining the control right of the autonomous vehicle, when the control right is transferred from the autonomous vehicle to the driver, storing the operation state of a multimedia device mounted in the autonomous vehicle and limiting the use, and the control right is transferred from the driver to the autonomous vehicle when switched to, determining whether to restore the multimedia device to an operational state in response to the location information and time information of the autonomous vehicle, and, according to the determination, to the driver according to the decision to provide the driver with an updated state corresponding to the operational state or location information or time information Provided is a method of controlling a multimedia device in an autonomous driving vehicle, comprising displaying one of the multimedia devices through a multimedia device.

Description

Apparatus and method for controlling a multimedia system mounted on an autonomous vehicle {APPARATUS AND METHOD FOR CONTROLLING MULTI-MEDIA SYSTEM EQUIPPED IN VEHICLE}

The present invention relates to an apparatus and method for controlling a multimedia system mounted on an autonomous vehicle, and more particularly, to a user who uses an autonomous vehicle when limiting or canceling the restricted use of the multimedia system according to a driving mode for a user using the autonomous vehicle. , to a method and apparatus capable of enhancing user convenience.

The driving of the vehicle may refer to a process from the time the driver or the user starts turning on the engine to operate the vehicle until the engine is turned off by the driver or the user. The definition of an autonomous vehicle can be defined as a vehicle that autonomously drives to a given destination by recognizing the surrounding environment, judging the driving situation, and controlling the vehicle without driver intervention. These autonomous vehicles are expected to become a future personal transportation means that can increase convenience by reducing traffic accidents, increasing traffic efficiency, saving fuel, and replacing driving.

Autonomous driving can provide drivers with a sense of stability and safety, while at the same time not placing any restrictions on the driver's activities. However, in the case of autonomous driving, it should be clearly defined who is responsible for what and when. In the case of partially autonomous driving, misunderstandings can have tragic consequences. Unless it is a fully autonomous vehicle, the driver needs to be able to drive at a certain point in time, so it is necessary to clearly recognize that this can actually happen, and to provide visual and auditory support so that the driver can feel responsible for driving. Obligations and sanctions are being created that allow for an immediate reaction through at least two senses. For example, even if the driver is doing work that is not related to driving, communication between the vehicle and the driver can prepare the driver to drive again.

US 2015/0094897 A1

According to the present invention, when the control right for the vehicle is switched from the driver to the vehicle according to the execution of the autonomous driving mode in a driving vehicle in a state in which the function limitation of the multimedia device according to the vehicle driving is activated, the driving regulation of the multimedia device is temporarily deactivated. It is possible to provide a control method and apparatus for partially or wholly releasing function restrictions for a user's multimedia device.

In addition, the present invention stores the current state of the multimedia device in the memory when the driving regulation of the multimedia device is reactivated and the user's function use restriction starts, and when the control right is switched to the driver again, the state before being stored in the temporary storage device A method and apparatus for automatically recovering may be provided.

In addition, the present invention provides a device capable of providing information or a screen by updating the state of a multimedia device previously used by the driver in response to new location and time information when the control right of the autonomous vehicle is switched from the driver to the vehicle; method can be provided.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able

A method of controlling a multimedia device in an autonomous vehicle according to an embodiment of the present invention includes determining a control right of the autonomous vehicle; when the control right is transferred from the autonomous vehicle to the driver, storing an operation state of a multimedia device mounted on the autonomous vehicle and limiting use; determining whether to restore the multimedia device to the operating state in response to location information and time information of the autonomous vehicle when the control right is transferred from the driver to the autonomous vehicle; and displaying one of the operation state or an updated state corresponding to the location information or the time information to the driver through the multimedia device according to the determination.

In addition, the method of controlling the multimedia device in the autonomous driving vehicle may further include permitting the use of the multimedia device after confirming a driving risk factor when the control right is transferred from the driver to the autonomous driving vehicle.

In addition, the control method of the multimedia device in the autonomous vehicle may further include the step of notifying the driver of the change of the control right by at least one of a sound, a text, a symbol/mark, and a lamp when the change of the control right is determined can

In addition, the step of storing the operating state of the multimedia device and limiting the use may include: checking whether the manipulation of the multimedia device is performed by the driver; limiting the use if the manipulation is by the driver; and permitting the use if the manipulation is not performed by the driver.

Also, whether the manipulation is performed by the driver may be determined by recognizing a motion for the manipulation by the multimedia apparatus.

In addition, when there are a plurality of multimedia devices in the step of storing the operation state of the multimedia apparatus and limiting the use, each operation state may be stored in the storage device and different use restrictions may be applied according to the operation characteristics.

In addition, the method of controlling a multimedia device in an autonomous vehicle may further include, when the storage device cannot store the operating state due to insufficient storage capacity, deleting the oldest information to secure the storage capacity.

In addition, the step of determining whether to restore the multimedia device to the operating state is a first step of checking whether there is a difference between the first position information and the current second position information at the time of storing the operation state by more than a preset range. Step 1; a second step of confirming whether the operation state is linked with the time information; and recovering the multimedia device according to the results of the first step and the second step.

In addition, the step of determining whether to restore the multimedia device to the operating state may include determining that the first location information and the second location information are within the preset range, and the operating state does not interlock with the time information. in this case, determining whether the manipulation of the multimedia device is performed by the driver; returning to the operating state when the operation is by the driver; and if the manipulation is not performed by the driver, maintaining the current operating state.

In addition, the determination of the control right and the limitation of use of the multimedia device may be determined by the location of the autonomous vehicle.

In the computer-readable recording medium according to another embodiment of the present invention, an application program characterized by realizing the above-described method for controlling a multimedia device in an autonomous driving vehicle through being executed by a processor may be recorded.

According to another embodiment of the present invention, a vehicle capable of autonomous driving in which a multimedia device is mounted includes an autonomous driving control device that determines a control right of the autonomous driving vehicle; When the control right is switched from the autonomous driving vehicle to the driver, the operation state of the multimedia device mounted on the autonomous driving vehicle is stored and use is restricted, and when the control right is switched from the driver to the autonomous driving vehicle, the autonomous driving vehicle a multimedia control device for determining whether to restore the multimedia device to the operating state in response to location information and time information; and a display device for displaying, through the multimedia device, one of the operation state or an updated state corresponding to the location information or the time information to the driver according to the determination.

In addition, when the control right is transferred from the driver to the autonomous driving vehicle in an autonomous driving capable vehicle equipped with a multimedia device, the autonomous driving control device checks a driving risk factor, and the multimedia control device controls the autonomous driving The use of the multimedia device may be permitted in response to the verification result of the device.

In addition, when the autonomous driving control device determines to switch the control right, the display device may notify the driver of the change of the control right through at least one of a sound, a text, a symbol/mark, and a lamp.

In addition, the display device may include at least one of a display or speaker linked to the multimedia device, a plurality of symbols or lamps constituting a dashboard, and a window or a windshield capable of displaying information on the autonomous vehicle. have.

In addition, in order to store the operating state of the multimedia device and limit the use of the multimedia device in a vehicle capable of autonomous driving in which the multimedia device is mounted, the multimedia control device checks whether the manipulation of the multimedia device is by the driver, and If by the driver, the use may be restricted, and if the operation is not by the driver, the use may be permitted.

Also, whether the manipulation is performed by the driver may be determined by recognizing a motion for the manipulation by the multimedia apparatus.

In addition, the vehicle capable of autonomous driving in which the multimedia device is mounted further includes a storage device for storing the operation state of the multimedia apparatus, and when there are a plurality of multimedia apparatuses, each operation state is stored in the storage device, Depending on the operating characteristics of the device, different usage restrictions may be applied.

Also, when the storage device cannot store the operating state due to insufficient storage capacity, the oldest information may be deleted to secure the storage capacity.

In addition, in order to determine whether to restore the multimedia device to the operating state in the autonomous driving-capable vehicle in which the multimedia device is mounted, the multimedia control device includes the first location information at the time of storing the operating state and the current a first step of confirming whether there is a difference between the second location information and the preset range; and performing a second step of checking whether the operation state is linked with the time information, and recovering the multimedia device according to the results of the first step and the second step.

In addition, in order to determine whether to restore the multimedia device to the operating state in the autonomous driving-capable vehicle in which the multimedia device is mounted, the multimedia control device is configured to include the first location information and the second location information If it is within a set range and the operation state is not linked with the time information, it is checked whether the operation of the multimedia device is by the driver, and if the operation is by the driver, the operation is returned to the operation state, If this is not by the driver, the current operating state may be maintained.

In addition, the determination of the control right and the limitation of use of the multimedia device may be determined by the location of the autonomous vehicle.

A multimedia system for a vehicle according to another embodiment of the present invention includes: a multimedia device in which at least one function executed according to a control right of an autonomous vehicle is restricted or switched; and a control device configured to perform an automatic recovery procedure in response to location information of the autonomous vehicle, time information, and user information operating the multimedia device for the restriction or the switching in the multimedia device.

In addition, the user information may be determined by recognizing a movement for manipulating the multimedia device.

In addition, the vehicle multimedia system may further include a storage device capable of storing each operation state when the plurality of multimedia devices or the plurality of functions are provided.

Also, the control device may update the function in response to the location information and the time information.

Aspects of the present invention are only some of the preferred embodiments of the present invention, and various embodiments in which the technical features of the present invention are reflected are detailed descriptions of the present invention that will be described below by those of ordinary skill in the art can be derived and understood based on

The effect on the device according to the present invention will be described as follows.

The present invention can provide a method and apparatus for efficiently utilizing a multimedia device mounted on a vehicle in a process in which the control right between the driver and the vehicle is switched in an autonomous vehicle.

In addition, according to the present invention, it is possible to set the use of a multimedia device mounted on a vehicle in response to laws and restrictions required for an autonomous vehicle, thereby improving safety and ease of use of the vehicle.

In addition, the present invention can provide information and a screen according to location and time to a user or driver by linking a multimedia device mounted on an autonomous vehicle and a navigation device.

In addition, the present invention may recognize a motion of a driver or a passenger in an autonomous vehicle and provide information and a screen corresponding to the motion through a multimedia device.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings are provided to help understanding of the present invention, and provide embodiments of the present invention together with detailed description. However, the technical features of the present invention are not limited to specific drawings, and features disclosed in each drawing may be combined with each other to form a new embodiment.
1 illustrates a method of controlling a multimedia device in an autonomous vehicle.
Fig. 2 explains an example of driving regulation according to the driving mode.
Fig. 3 explains an example of travel regulation according to the running state.
4 illustrates an example of driving regulation while driving.
5 illustrates an example of travel regulation.
6 illustrates a first control method of a multimedia device.
7 illustrates a second control method of the multimedia apparatus.
8 illustrates an example of a method for detecting manipulation of a multimedia device.
9 illustrates an example of an automatic recovery procedure of the multimedia device.
10 illustrates an example of an interface for an automatic recovery procedure of a multimedia device.
11 illustrates a first example of a multimedia system in an autonomous vehicle.
12 illustrates a second example of a multimedia system in an autonomous vehicle.

Hereinafter, an apparatus and various methods to which embodiments of the present invention are applied will be described in more detail with reference to the drawings. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves.

In the description of the embodiment, in the case where it is described as being formed on "up (above) or under (below)" of each component, the upper (upper) or lower (lower) means that the two components are in direct contact with each other or One or more other components are all formed by being disposed between two components. In addition, when expressed as “upper (upper) or lower (lower)”, a meaning of not only an upper direction but also a lower direction based on one component may be included.

The level of autonomous driving in autonomous vehicles is divided into several categories. For example, the level of autonomous driving is the first stage as a selective active control stage, which is the stage of automation of specific functions, and the integrated active control stage in which the existing autonomous driving technologies are integrated and functioning (the drivers keep their gaze forward, do not use the steering wheel and pedals) In the second stage, the vehicle recognizes traffic signals and road flow so that the driver can engage in other activities such as reading, and the third stage as a limited autonomous driving stage that requires driver intervention only in certain situations, driver intervention in all situations It can be classified into the fourth stage as a fully autonomous driving stage that does not require this.

In the case of Level 3 or Level 4 of the autonomous driving level, a multimedia system including a navigation function for setting a route may be essential for the vehicle. As the role played by the driver or the autonomous vehicle is determined according to the driving situation or driving state, there is a need to place restrictions on the driver's right to use the multimedia system while the vehicle is driving for driving safety. For example, in the North American NTHSA driving regulation guideline, there are driving regulations that link vehicle speed signal/PGS lever position/brake information for the purpose of preventing accidents caused by manipulation of multimedia devices while driving a vehicle.

In autonomous driving mode, the control authority for the vehicle is transferred from the driver to the vehicle system, so the driver's freedom to manipulate multimedia devices is secured. There is a problem that can feel and can not operate the multimedia system, so improvement measures are needed. For example, driving regulation conditions for general vehicles are set only according to stop/driving conditions, but in the case of autonomous vehicles, ON/OFF switching of driving restrictions can be performed even under driving conditions, so a method and device to control this are required can be

1 illustrates a method of controlling a multimedia device in an autonomous vehicle.

As shown, the method for controlling a multimedia device in an autonomous vehicle includes the steps of determining the control right of the autonomous vehicle (2), and when the control right is switched from the autonomous vehicle to the driver, the operation state of the multimedia device mounted in the autonomous vehicle storing and limiting use (4); when control is transferred from the driver to the autonomous vehicle, determining whether to restore the multimedia device to an operating state in response to the location information and time information of the autonomous vehicle ( 4), and according to the determination, displaying (6) to the driver one of the operating status or the updated status corresponding to the location information or the time information via the multimedia device. In particular, in one embodiment of the present invention, the control right of the autonomous vehicle is switched from the driver to the vehicle, thereby removing the restriction on the use of the multimedia device for the driver (or passenger) and selectively enabling the driver to use the multimedia device efficiently. You can switch and restore the operating mode of the device. To this end, when the control right of the autonomous vehicle is switched from the vehicle to the driver and the driver stops using the multimedia device, it is necessary to store the operation state of the multimedia device at the time.

Meanwhile, although not shown, the method of controlling the multimedia device in the autonomous driving vehicle may further include permitting the use of the multimedia device after checking for driving risk factors when the control right of the autonomous driving vehicle is switched from the driver to the autonomous driving vehicle. can Even when the control right of the autonomous vehicle lies with the vehicle, if a factor threatening driving safety is found, the vehicle needs to notify the driver and induce the driver's attention. Accordingly, even if the autonomous vehicle has the control right for driving, when it is determined that there is no driving risk factor after checking the driving risk factor, the driver can freely use the multimedia device installed in the vehicle.

In addition, the control method of the multimedia device in the autonomous vehicle is determined to change the control right for driving of the autonomous vehicle (from vehicle to driver or from driver to vehicle, etc.) The method may further include the step of notifying the change of the control right by at least one method of a ramp. The control right of an autonomous vehicle is a very important factor, and the driver or passenger in the autonomous vehicle needs to check information about it.

For example, the step (4) of storing the operation state of the multimedia device and limiting the use may include the steps of confirming whether the manipulation of the multimedia device is by the driver, limiting the use if the manipulation is by the driver, and the manipulation of the multimedia device by the driver It may include a step of permitting use if it is not by In some cases, if the multimedia device mounted on the autonomous driving vehicle is being operated by a passenger other than the driver, there is no need to restrict the use of the multimedia device regardless of whether the control right for driving of the autonomous driving vehicle is transferred. An autonomous vehicle may include a plurality of multimedia devices, some of which may be operated independently of the driver. There is no need to restrict the use of a multimedia device that is operated regardless of the driver in response to the control right for the autonomous vehicle driving. However, in the case of a multimedia device that can be operated by both a driver and a passenger, it is necessary to determine who operates the multimedia device. For example, whether the manipulation is performed by the driver may be determined by the multimedia apparatus recognizing a motion (motion) for the manipulation.

In addition, if there are a plurality of multimedia devices mounted on the vehicle in the step (4) of storing the operation state of the multimedia apparatus and limiting the use, each operation state is stored in the storage device, and different usage restrictions according to the operation characteristics are stored in the storage device. can do. Meanwhile, the method for controlling a multimedia device in an autonomous vehicle may further include, when the storage device cannot store the operating state due to insufficient storage capacity, deleting the oldest information to secure the storage capacity. Since it is more difficult to expand the storage device of an autonomous vehicle as needed than a portable computing device, a situation in which the capacity of the magnetic field device is insufficient may occur. However, by deleting the oldest information in chronological order from the storage device, it is possible to minimize the inconvenience of the driver or passenger.

Meanwhile, although not shown, the step 4 of determining whether to restore the multimedia device to the operating state in the control method of the multimedia device in the autonomous vehicle includes the first location information at the time of storing the operating state and the current second A first step of checking whether the location information has a difference of more than a preset range, a second step of checking whether the operation state is interlocked with time information, and recovering the multimedia device according to the results of the first step and the second step may include steps.

For example, while the autonomous vehicle is driving, control may be transferred from the vehicle to the driver or from the driver to the vehicle, before, after, or even during the transition, the autonomous vehicle may move. It may be assumed that if the autonomous vehicle has control over driving, and the driver is collecting interest information adjacent to the vehicle's location through a multimedia device. In this case, for some reason, the control right for driving may be transferred to the driver, and after the driver directly drives the vehicle for a certain period of time, the control right may be transferred to the autonomous vehicle again. Since the vehicle has already traveled a certain distance, the information may be useless if control is transferred to the autonomous vehicle and the multimedia device the driver wants to use is restored to the state before the driver directly drives. Accordingly, in this case, when the first location information and the second location information are compared and there is a difference of more than a preset range, the more efficient use of the multimedia device is to provide the driver with interest information corresponding to the second location information, which is the current location. It is possible. Here, the preset range may be determined by specific numerical values (eg, 10 m, 500 m, 1 km, etc.) or map information (eg, administrative district, etc.), or may be determined by the setting of a driver or a user.

Although the case where the position of the vehicle is moved has been described in the above example, not only the position of the vehicle but also time passes. If the driver is collecting or receiving time-linked information through the multimedia device, it may be more efficient to use the multimedia device when the multimedia device delivers information corresponding to the current time instead of information corresponding to the previous time.

Furthermore, although not shown, in the step (4) of determining whether to restore the multimedia device to the operating state, the first location information and the second location information are within a preset range, and the operating state does not interlock with the time information. In this case, the method may further include the steps of determining whether the manipulation of the multimedia device is by the driver, returning to the operating state if the manipulation is by the driver, and maintaining the current operating state if the manipulation is not by the driver . If the multimedia device is not operated by the driver (for example, when a passenger uses the multimedia device mounted on the vehicle), even if the multimedia device becomes available again, a separate operation or recovery procedure may not be necessary.

In addition, the location and layout of the multimedia device mounted on the vehicle may influence whether to limit the use of the vehicle or to determine whether to return. The vehicle may be provided with a seat in which not only the driver but also at least one passenger can ride, and a multimedia device that can be used and used in each seat may be mounted. In the case of a multimedia device other than a multimedia device that the driver can operate or that is placed close to the driver's seat, it may be free from the determination of the right to control the driving of the autonomous vehicle.

Meanwhile, the determination of the right to control the driving of the autonomous driving vehicle and the restrictions on the use of the multimedia device mounted in the autonomous driving vehicle may be determined by the location of the autonomous driving vehicle. Autonomous vehicles may have to comply with different laws or rules depending on where they drive and where they operate. The autonomous vehicle may collect regulatory information corresponding to the driving position through a telematics server or the like or store it in the vehicle. Based on such information, the autonomous vehicle can perform driving and control rights and driving regulations according to laws required by the driving region.

Fig. 2 explains an example of driving regulation according to the driving mode. More specifically, FIG. 2 will explain vehicle control rights and driving regulations according to conditions (driving mode) while the autonomous driving vehicle is driving by way of example.

As shown, the control right of the autonomous vehicle can be divided into three types. For example, a manual driving mode in which the driver has all control over vehicle driving, an autonomous driving mode in which the vehicle has all control, or a partial autonomous driving mode in which the driver and the vehicle partially share driving control.

In the manual driving mode, driving-related control of the vehicle is performed by the driver. In the case of a vehicle equipped with an automatic transmission (A/T), if the position of the transmission is not the parking (P), it may be determined that the vehicle is driving. On the other hand, in the case of a vehicle equipped with a manual transmission (M/T), it may be determined that the vehicle is driving when the parking brake is released, the gear is not in a neutral state, or the vehicle speed is higher than or equal to a certain speed.

In the manual driving mode, since the driver is fully responsible for driving safety, driving regulations need to be applied. Here, the driving regulation may include, for example, the driver restricting the use of multimedia devices installed in the vehicle. For example, prohibit scrolling of text in multimedia devices, prohibit display of some information or modes (such as detailed help), prohibit Bluetooth pairing mode, play or broadcast video ( DMB) functions or restricting the use of apps or some functions of connectivity with a portable terminal may be examples of driving regulations. In addition to this, various regulatory items may be set in the driving regulation. In addition, the content of these regulations may vary according to regulations, laws, etc. applied according to the location of the vehicle.

In the partially autonomous driving mode, the driving-related control of the vehicle is performed by the driver, but may be partially assisted by a driving assistance device or system mounted on the vehicle. Here, examples of the driving assistance device or system may include a highway driving assistance system, a careless driving warning system, a front/rear/side collision avoidance system, a lane departure prevention system, and the like.

Even in partially autonomous driving mode, since the driver has duties and responsibilities for driving-related control of the vehicle, driving regulations need to be applied. The driving regulation in the partially autonomous driving mode may be the same as the driving regulation in the manual driving mode. According to an embodiment, driving restrictions in the partially autonomous driving mode are more relaxed than driving restrictions in the manual driving mode, so that some functions or use of some multimedia devices may be more possible than in the manual driving mode.

Unlike manual driving mode or partially autonomous driving mode, driving regulations need to be relaxed in autonomous driving mode or when the vehicle is stopped. In autonomous driving mode, since the vehicle has driving-related control and is not in a driving state when the vehicle is stopped, the need to consider the driver's role in driving safety is reduced. In the autonomous driving mode or when the vehicle is stopped, it is necessary to increase the convenience of the driver in the vehicle by lifting the driving restrictions.

Meanwhile, in the autonomous driving mode, the vehicle may control the vehicle by itself, but when the driver wants to intervene, it may be possible to switch to the manual driving mode or the partially autonomous driving mode at any time. If the transition from the autonomous driving mode to the manual driving mode or the partially autonomous driving mode is made, it is necessary for the driver to partially restrict the use of multimedia devices installed in the vehicle for driving safety by activating driving regulations.

Fig. 3 explains an example of travel regulation according to the running state. Specifically, FIG. 3 describes whether or not the driving regulation according to the driving state of the vehicle is applied, not the control right related to the driving of the vehicle.

As shown, it is possible to determine whether the vehicle is stopped or driving, and whether to apply the driving regulation.

For example, when the position of the transmission is parking (P), when the parking brake is set, when the transmission is in a neutral state and there is no movement of the vehicle (below a preset speed), it may be determined that the vehicle is stopped. When the vehicle is stopped, the need for the driver to consider the impact on driving safety is reduced, thereby reducing the need for regulations and restrictions on the driver's use of multimedia devices. Accordingly, it is possible to release the driving regulation when the vehicle is stopped or to lower the driving regulation when the vehicle is divided into a plurality of stages.

Conversely, when the vehicle engine is turned on and the vehicle is not being stopped, it may be determined that the vehicle is running. When the vehicle is driving, the burden on driving safety is placed on the driver, and accordingly, driving regulation can be activated or the level of driving regulation can be increased.

4 illustrates an example of driving regulation while driving. Specifically, FIG. 4 explains whether the control right for driving in the autonomous driving vehicle is switched over time and activation and deactivation of driving regulations by reflecting various situations occurring while the vehicle is driving.

As shown, initially the vehicle may be stopped in a manual driving mode. In this case, the right to control the vehicle may not be an important consideration, and the driving regulation is inactive.

Thereafter, when the vehicle starts to travel, the driver may have a control right for driving the vehicle in the manual driving mode, and driving regulation may be activated.

While driving in the manual driving mode, it may be desired to switch the control right of the vehicle from the driver to the vehicle, and the vehicle may be switched to the autonomous driving mode. After the control right of the vehicle is completely transferred to the autonomous driving mode, the driving regulation may be deactivated.

When the vehicle is driving in autonomous driving mode, the driver can intervene in driving. When the driver intervenes in driving, the control right for driving of the vehicle is transferred from the vehicle to the driver, and driving regulation may be activated along with the transition to the manual driving mode (or partially autonomous driving mode). In this case, the device for controlling the in-vehicle multimedia device may store the operating state of the multimedia device used or progressed by the driver immediately before switching to the manual driving mode in the storage device.

After the driver intervenes in driving, the vehicle may be driven in a manual driving mode (or partially autonomous driving mode), and driving regulation may remain activated. Thereafter, when the driver attempts to transfer the control right of the vehicle to the vehicle, the driving regulation may be deactivated after the control right is completely transferred.

When the control right of the vehicle is completely switched and the vehicle takes control, the driving regulation is deactivated, and the driver can use the multimedia device again. Since the operation state of the multimedia device used before the driver intervened in driving is stored in the storage device, it is possible to inquire whether the driver wants to return the operation of the multimedia device based on the information (eg, a pop-up message). When the driver wants the multimedia device to operate in a previous state, the multimedia device may be operated based on information stored in the storage device.

5 illustrates an example of travel regulation.

As shown, through the vehicle's multimedia device, the driver may search for map information 68 . For example, the driver may search for information of interest in the vicinity of the location where the vehicle is driving. However, in a state in which the driving regulation is activated, it is possible to limit the driver's ability to search for information of interest through the multimedia device. On the other hand, it is possible to induce the driver to pay attention to driving safety through a pop-up message 98 or the like in response to driving regulations.

Although the pop-up message 98 is described as an example for driving regulation in FIG. 5, it is also possible to completely switch the screen as well as the pop-up message 98. FIG. By stopping the operation of the multimedia device used by the driver and switching to a screen corresponding to the driving regulations, it is possible to block in advance actions that are harmful to driving safety, including manipulation of the multimedia device that the driver can do.

6 illustrates a first control method of a multimedia device. More specifically, the first control method may describe a specific process in which the driving regulation is switched from activation to deactivation.

As shown, the first control method may check whether the engine of the vehicle is turned on ( 10 ). It may be determined whether the vehicle is driven in the manual driving mode (12). At this time, since the vehicle travels in the manual driving mode, driving regulation may be activated ( 12 ).

It is possible to monitor whether an event in which the control right for driving of the vehicle is switched occurs ( 14 ). When the control right for driving of the vehicle is switched from the driver to the vehicle, it can be recognized that the vehicle is driven in the autonomous driving mode (16). Even if the vehicle is driven in the autonomous driving mode, it may be determined whether there is a dangerous condition in the driving of the vehicle ( 18 ). If there are factors, conditions, etc. that are dangerous to the driving of the vehicle, the driving regulation may be maintained in an activated state ( 22 ). On the other hand, if it is determined that there is no element or condition that is dangerous to the driving of the vehicle, the driving regulation may be deactivated ( 20 ).

7 illustrates a second control method of the multimedia apparatus. Specifically, the second method may specifically describe a process in which the driving regulation is activated from deactivation or deactivated again.

As illustrated, in the second control method, driving restrictions may be deactivated in a situation in which the vehicle is driving in the autonomous driving mode ( 24 ). In the autonomous driving mode, a switching event for the control right of the vehicle may occur ( 26 ). In this case, the event includes a case in which the control right of the vehicle is switched from the vehicle to the driver. When the control right of the vehicle is transferred to the driver, the vehicle may inform the driver that the driving regulation is activated (28). For example, “Driving regulation has been activated. A message containing text such as “Please look ahead” can be delivered to the driver through the screen of the multimedia device. Whether the driving regulation is activated may be transmitted to the driver through various methods such as a sound, a symbol, and a lamp, as well as in the form of a message including text. After notifying the driver that the driving regulation has been activated (28), the operating state of the multimedia device may be stored in the storage device (30).

During driving in the manual driving mode or the partially autonomous driving mode, the right to control the driving of the vehicle may be determined ( 32 ). If the vehicle has the control right (34), it is possible to determine a situation or condition that is harmful to driving safety (36). If a situation or condition endangering driving safety exists (YES in 36), driving regulation may be activated (38). On the other hand, when it is determined that there is no situation or condition that may be harmful to driving safety (NO in 36), the driving regulation may be deactivated (46).

If the driver continues to control the driving of the vehicle while driving in the manual driving mode or the partially autonomous driving mode ( 40 ), it may be checked whether the manipulation of the multimedia device is performed by the driver ( 42 ). If the manipulation of the multimedia device is by the driver, the driving regulation may be activated ( 44 ). If the operation of the multimedia device is performed by a passenger, driving restrictions may be deactivated (46).

If the driving restrictions are deactivated (46), the driver may be asked whether the multimedia device will return to the previous operating state (eg, return to the previous screen) (48). If the driver opposes returning the multimedia device to the previous state, a current screen, a new screen, or a menu screen of the multimedia device may be displayed ( 52 ).

Conversely, if the driver wants the multimedia device to return to the previous state, an automatic return procedure may be performed ( 50 ). When a return screen and an operation state to be returned are determined according to the automatic return procedure, the corresponding information or the corresponding screen may be displayed (54). In this case, the displayed screen may include a screen in a previous state or a screen updated according to location or time information.

8 illustrates an example of a method for detecting manipulation of a multimedia device.

As illustrated, it is possible to determine whether the person operating the multimedia device is a driver or a passenger according to whether the person operating the multimedia device is located on the left side or the right side of the multimedia device.

For example, the screen of the multimedia device may be divided into left and right based on the center (R1, R2). If it is desired to operate menus, buttons, etc. located in the left area R1, the person located on the left side of the multimedia device performs a movement 64 from the left to the left area R1 without moving to the right area R2. You can operate menus, buttons, etc. Unlike a person located on the left, a person located on the right side of the multimedia device may move 66 through the right area R2 to the left area R1 to operate menus, buttons, etc. located in the left area R1. have.

Accordingly, when motion occurring in front of the screen of the multimedia device is detected in the process of manipulating menus and buttons located on the screen of the multimedia device, it can be determined whether the person operating the menus or buttons is located on the right or left side of the multimedia device. can judge The location of the multimedia device in the vehicle, the driver or the passenger on the left or right side are the design specifications of the vehicle. can detect

If the menu and button on the screen of the multimedia device are arranged in the right region R2, a phenomenon opposite to the above-described case occurs. A person located on the left side of the multimedia device passes through the left area R1 and arrives at the right area R2, whereas a person located on the right arrives at the right area R2 without having to go to the left area R1. do. In this way, it can be determined whether the person operating the multimedia apparatus is located on the left or right side with respect to the screen of the multimedia apparatus.

9 illustrates an example of an automatic recovery procedure of the multimedia device. 9 illustrates the automatic recovery procedure 50 described in FIG. 7 in more detail.

As shown, when the automatic return procedure is started according to the driver's selection ( 70 ), the current location of the vehicle may be checked ( 72 ).

It may be checked whether there is a difference between the current location of the vehicle and the location of the vehicle at the point in time when the operation state of the multimedia device is stored by more than a preset range ( S74 ). Here, the preset range may be numerically determined such as 5m, 10m, 50m, 100m, 500m, etc., may be set corresponding to an error range of a location tracking device (eg, GPS) mounted on the vehicle, and the driver of the vehicle may be decided arbitrarily.

If a change in the location of the vehicle occurs, it may be determined whether the operating state of the multimedia device stored in the storage device to which the driver wants to return is linked to the time information ( S82 ). If the operation state of the multimedia device is not linked to the time information, an updated screen may be displayed based on the change in the location information ( 84 ). For example, in the case of location information of a gas station close to the location of the vehicle, it is information that corresponds only to the location information of the vehicle regardless of time. can

Conversely, when the operation state of the multimedia apparatus is linked to time information, an updated screen may be displayed based on a change in location information and time information ( 86 ). For example, in the case of gasoline price information of a gas station close to the location of the vehicle, the location of the nearby gas station may vary according to the location of the vehicle, and the price of gasoline may also vary over time. In this case, when the screen updated based on the current location information and time information of the vehicle is transmitted to the driver, the driver's use of the multimedia device may be more efficient.

On the other hand, if there is no difference between the current location of the vehicle and the vehicle location at the point in time when the operating state of the multimedia device is stored (NO in 74), the operating state of the multimedia device stored in the storage device to which the driver wants to return is time information It can be determined whether it is linked to (78). If the operation state of the multimedia device is linked to the time information, an updated screen may be displayed based on the time information ( 88 ).

If the operation state of the multimedia apparatus is not linked to the time information, it may be checked whether the operation of the multimedia apparatus is performed by the passenger (80). If the multimedia device is being operated by a passenger, the current screen may be maintained ( 90 ). If the multimedia device is being operated by a driver rather than a passenger, a screen stored in the storage device may be displayed ( 92 ).

Meanwhile, some of the processes related to the control method of the multimedia apparatus or the automatic return procedure may include a background process that can be executed behind the scenes without driver's intervention.

10 illustrates an example of an interface for an automatic recovery procedure of a multimedia device.

As shown, a pop-up message 96 may be exemplified as an interface for performing an automatic return procedure. A pop-up message 96 may be output through the screen 94 of the multimedia device, and the intention of the driver may be confirmed. If the driver wants to return to the previous screen, the screen of the multimedia device can be restored through the automatic return procedure. If the driver does not want to return to the previous screen, the automatic return procedure may not be executed.

11 illustrates a first example of a multimedia system in an autonomous vehicle.

As shown, a vehicle capable of autonomous driving equipped with a multimedia device is capable of controlling the power required for driving the vehicle and controlling a sensor capable of recognizing the surrounding environment, a driving control device, a transmission, an engine/motor, etc. It may include a vehicle control unit 62 . The autonomous driving controller 60 may determine the control right of the autonomous driving vehicle in response to information checked and collected by the vehicle controller 62 , a user input, and the like.

The multimedia device 100 mounted on the vehicle includes an input device 106 for receiving a user's input, a display device 104 for outputting information/data, etc., an operation state of the multimedia device, information requested by the user, and the like. It is possible to control the storage device 108 , the input device 106 , the display device 104 , and the storage device 108 , and regulate driving in response to the control right for driving of the vehicle transmitted from the autonomous driving control device 60 . It may include a control device 102 capable of determining and controlling.

For example, when the control right for driving of the vehicle is transferred from the autonomous driving vehicle to the driver, the control device 102 may store the operation state of the multimedia device mounted in the autonomous driving vehicle and limit the use thereof. Also, when the control right is transferred from the driver to the autonomous vehicle, the control device 102 may determine whether to restore the multimedia device to an operating state in response to location information and time information of the autonomous vehicle. In response to the determination of the control device 102 , the display device 104 in the multimedia apparatus 100 may display one of an operation state or an updated state corresponding to location information or time information to the driver.

When the control right is transferred from the driver to the autonomous driving vehicle in the autonomous driving capable vehicle equipped with the multimedia device, the autonomous driving control device 60 may check driving risk factors. The control device 102 may permit the use of the multimedia device 100 in response to the confirmation result of the autonomous driving control device 60 .

When the autonomous driving control device 60 decides to switch the control right in a vehicle capable of autonomous driving equipped with a multimedia device, the control device 102 provides the display device 104 to the driver with a sound, text, symbol/mark, or lamp. At least one method may be used to notify the change of control right. On the other hand, the display device 104 includes at least one of a display or speaker linked to the multimedia device, a plurality of symbols or lamps constituting a dashboard, a window or a windshield capable of displaying information on an autonomous vehicle, or At least one may be linked.

In order to store the operating state of the multimedia device and limit the use of the multimedia device in an autonomous driving capable vehicle equipped with the multimedia device, the control device 102 may check whether the multimedia device is operated by the driver. If the operation is by the driver, use of the multimedia device may be restricted. Conversely, if the operation is not by the driver, use of the multimedia device may be permitted. Here, whether the manipulation of the multimedia device is performed by the driver may be determined by recognizing the motion for the manipulation of the multimedia device.

A vehicle capable of autonomous driving in which the multimedia device is mounted may include a storage device 108 for storing an operation state of the multimedia device. If there are a plurality of multimedia devices, the control device 102 may store each operation state in the storage device 108 and set different usage restrictions according to each operation characteristic. Meanwhile, when the storage device 108 cannot store the operating state of the multimedia device due to insufficient storage capacity, the oldest information may be deleted to secure the storage capacity.

In order to determine whether to restore the multimedia device to the operating state in the autonomous driving capable vehicle in which the multimedia device is mounted, the control device 102 includes the first location information at the time of storing the operating state and the current second location information. A first step of confirming whether there is a difference of more than a preset range in the location information and a second step of checking whether the operation state is linked with the time information may be performed. The control device 102 may restore the operating state of the multimedia device according to the results of the first and second steps.

In order to determine whether to restore the multimedia device to the stored operating state in the autonomous driving capable vehicle equipped with the multimedia device, the control device 102 determines whether the first location information and the second location information are within a preset range, If the status is not linked with the time information, it can be checked whether the operation of the multimedia device is by the driver. If the manipulation is by the driver, the control device 102 may return the multimedia device to the stored operating state. If the operation is not by the driver, the control device 102 may keep the multimedia device in its current operating state.

In a vehicle capable of autonomous driving in which a multimedia device is mounted, driving regulations including the determination of control rights and restrictions on the use of the multimedia device 100 may be determined by the location of the autonomous driving vehicle. Information such as rules, restrictions, and policies that may vary depending on the location of the autonomous vehicle may be stored in the vehicle or may be transmitted through a telematics server or the like.

12 illustrates a second example of a multimedia system in an autonomous vehicle. The embodiment described with reference to FIG. 12 has parts similar to those of the embodiment described with reference to FIG. 11 , and descriptions of similar parts may be omitted.

As shown, the autonomous vehicle includes a vehicle control unit 62 capable of controlling the power required for driving the vehicle, and controlling a sensor capable of recognizing the surrounding environment, a driving control device, a transmission, an engine/motor, and the like. can do. The autonomous driving control device 60 capable of determining the control right of the autonomous driving vehicle in response to the information checked and collected by the vehicle controller 62, input from a user, etc. may be included in the multimedia device 100 rather than the vehicle. Although this embodiment does not include the autonomous driving controller 60 in the vehicle, it may be used in a vehicle to which the autonomous driving controller 60 can be added by upgrading the multimedia device 100 .

Meanwhile, according to an embodiment, the multimedia system for a vehicle includes a multimedia device in which at least one function executed according to the control right of the autonomous driving vehicle is restricted or switched, and the location information of the autonomous driving vehicle, time information and The control device may include a control device that performs an automatic return procedure in response to user information for manipulating the multimedia device. The in-vehicle multimedia system may determine user information by recognizing a movement for manipulating the multimedia device.

Also, when there are a plurality of multimedia devices or a plurality of functions, the in-vehicle multimedia system may further include a storage device capable of storing each operation state.

Also, the control device in the vehicle multimedia system may update the function of the multimedia device in response to location information and time information.

The method according to the above-described embodiment may be produced as a program to be executed by a computer and stored in a computer-readable recording medium. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape. , floppy disks, optical data storage devices, and the like.

The computer-readable recording medium is distributed in a network-connected computer system, so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the above-described method can be easily inferred by programmers in the technical field to which the embodiment belongs.

It is apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit and essential characteristics of the present invention.

Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

100: multimedia system
62: vehicle control unit
60: autonomous driving control device
50: automatic return procedure

Claims (26)

determining the right to control the autonomous vehicle;
when the control right is transferred from the autonomous driving vehicle to the driver, storing an operation state of a multimedia device mounted on the autonomous driving vehicle prior to activation of the driving regulation according to activation of the driving regulation and limiting use;
When the control right is transferred from the driver to the autonomous vehicle, at least one of current location information and current time information of the autonomous vehicle is reflected in restoring the multimedia device to the stored operation state according to the deactivation of the driving regulation determining whether or not to; and
According to the determination, any one of a first operation state corresponding to the stored operation state or a second operation state corresponding to an operation state updated by reflecting at least one of the current location information and the current time information is set to the multimedia apparatus Steps to show through
A method of controlling a multimedia device in an autonomous vehicle, comprising: According to claim 1,
when the control right is transferred from the driver to the autonomous vehicle, confirming a driving risk factor and permitting use of the multimedia device
A method for controlling a multimedia device in an autonomous vehicle, further comprising: According to claim 1,
when it is decided to switch the control right, notifying the driver of the change of the right of control by at least one method of sound, text, symbol/mark, and a lamp;
A method for controlling a multimedia device in an autonomous vehicle, further comprising: According to claim 1,
The step of storing the operating state of the multimedia device and limiting its use includes:
checking whether the manipulation of the multimedia device is performed by the driver;
limiting the use if the manipulation is by the driver; and
permitting the use if the operation is not by the driver
A method of controlling a multimedia device in an autonomous vehicle, comprising: 5. The method of claim 4,
The method of controlling a multimedia apparatus in an autonomous vehicle, wherein the multimedia apparatus recognizes a movement for the operation and determines whether the operation is performed by the driver. According to claim 1,
If there are a plurality of multimedia devices in the step of storing and limiting the use of the operation state before the activation of the driving regulation of the multimedia apparatus, each operation state is stored in the storage device, and different use restrictions according to the operation characteristics A method of controlling a multimedia device in an autonomous vehicle capable of 7. The method of claim 6,
When the storage device cannot store the operating state prior to activation of the driving regulation due to insufficient storage capacity, deleting the oldest information to secure the storage capacity
A method for controlling a multimedia device in an autonomous vehicle, further comprising: According to claim 1,
The determining step is
a first step of confirming whether there is a difference of more than a preset range between the location information at the time of storing the stored operation state and the current location information;
a second step of confirming whether the stored operation state is linked with time information; and
recovering the multimedia device according to the results of the first step and the second step;
A method of controlling a multimedia device in an autonomous vehicle, comprising: 9. The method of claim 8,
The determining step is
When the location information at the time of storage and the current location information are within the preset range, and the stored operation state does not work with the time information,
checking whether the manipulation of the multimedia device is performed by the driver;
returning to the first operating state when the manipulation is by the driver; and
If the manipulation is not by the driver, maintaining the current operating state
A method for controlling a multimedia device in an autonomous vehicle, further comprising: According to claim 1,
The method of controlling a multimedia device in an autonomous driving vehicle, wherein the determination of the control right and the limitation of use of the multimedia device are determined by a location of the autonomous driving vehicle. A computer-readable recording medium in which an application program is recorded, characterized in that the method for controlling a multimedia device in an autonomous vehicle according to any one of claims 1 to 10 is realized through being executed by a processor. an autonomous driving control device that determines a control right of the autonomous driving vehicle;
When the control right is switched from the autonomous driving vehicle to the driver, according to the activation of the driving regulation, the operation state of the multimedia device before the activation of the driving regulation is stored and use is restricted, and the control right is transferred from the driver to the autonomous driving vehicle a multimedia control device configured to determine whether to reflect at least one of current location information and current time information of the autonomous vehicle in restoring the multimedia device to the stored operating state according to deactivation of the driving regulation, if switched; and
According to the determination, any one of a first operation state corresponding to the stored operation state or a second operation state corresponding to an operation state updated by reflecting at least one of the current location information and the current time information is set to the multimedia apparatus display device through
A vehicle capable of autonomous driving equipped with a multimedia device, including a. 13. The method of claim 12,
When the control right is transferred from the driver to the autonomous vehicle,
The autonomous driving control device checks for driving risk factors,
The multimedia control device permits the use of the multimedia device in response to the confirmation result of the autonomous driving control device,
Autonomous vehicle equipped with multimedia device. 13. The method of claim 12,
When the autonomous driving control device determines to switch the control right,
The display device notifies the driver of the change of the control right by at least one method of sound, text, symbol/mark, and lamp,
Autonomous vehicle equipped with multimedia device. 15. The method of claim 14,
The display device includes at least one of a display or speaker interlocked with the multimedia device, a plurality of symbols or lamps constituting a dashboard, and a window or a windshield capable of displaying information on the autonomous vehicle,
Autonomous vehicle equipped with multimedia device. 13. The method of claim 12,
To store the operating state prior to the activation of the driving regulation of the multimedia device and limit the use,
The multimedia control device is
Check whether the operation of the multimedia device is by the driver,
If the manipulation is by the driver, limit the use;
permitting the use if the operation is not by the driver,
Autonomous vehicle equipped with multimedia device. 17. The method of claim 16,
Whether the operation is performed by the driver is determined by the multimedia device recognizing the movement for the operation,
Autonomous vehicle equipped with multimedia device. 13. The method of claim 12,
Further comprising a storage device for storing the operating state before the activation of the driving regulation of the multimedia device,
When there are a plurality of multimedia devices, the operation state prior to activation of each of the driving restrictions may be stored in the storage device and different usage restrictions may be applied according to each operation characteristic,
Autonomous vehicle equipped with multimedia device. 19. The method of claim 18,
When the storage device cannot store the operation state due to insufficient storage capacity, the oldest information is deleted to secure the storage capacity,
Autonomous vehicle equipped with multimedia device. 13. The method of claim 12,
For this determination,
The multimedia control device is
a first step of confirming whether there is a difference of more than a preset range between the location information at the time of storing the stored operation state and the current location information; and
performing a second step of confirming whether the stored operation state is linked with time information;
recovering the multimedia device according to the results of the first step and the second step;
Autonomous vehicle equipped with multimedia device. 21. The method of claim 20,
For this determination,
The multimedia control device is
When the location information at the time of storage and the current location information are within the preset range, and the stored operation state does not work with the time information,
Check whether the operation of the multimedia device is by the driver,
If the operation is by the driver, it returns to the first operating state,
If the operation is not by the driver, maintaining the current operating state,
Autonomous vehicle equipped with multimedia device. 13. The method of claim 12,
the determination of the right of control and the restriction of the use of the multimedia device are determined by the location of the autonomous vehicle;
Autonomous vehicle equipped with multimedia device. a multimedia device in which at least one function executed according to the control right of the autonomous vehicle is restricted or switched; and
A control device for performing an automatic recovery procedure in response to at least one of location information and time information of the autonomous vehicle in response to the restriction or the transition in the multimedia device
including,
The multimedia device comprises:
When the control right is switched from the autonomous vehicle to the driver, according to the activation of the driving regulation, the operation state of the multimedia device before the activation of the driving regulation is stored and use is restricted;
When the control right is transferred from the driver to the autonomous vehicle, at least one of current location information and current time information of the autonomous vehicle is reflected in restoring the multimedia device to the stored operation state according to the deactivation of the driving regulation decide whether to
According to the determination, any one of a first operation state corresponding to the stored operation state or a second operation state corresponding to an operation state updated by reflecting at least one of the current location information and the current time information is set to the multimedia apparatus Displayed through, in-vehicle multimedia system. 24. The method of claim 23,
The control device is
Performing the automatic return procedure in consideration of user information for operating the multimedia device,
The user information is determined by recognizing a movement for manipulating the multimedia device, the vehicle multimedia system. 24. The method of claim 23,
If there are a plurality of multimedia devices or a plurality of functions, a storage device capable of storing each operation state before activation of the driving regulation
Further comprising a, vehicle multimedia system.
delete

Images