KR20230076171A - Extended Reality Virtual Content Provision System for Interworking with Autonomous Vehicles and Metaverse - Google Patents

Abstract

The present invention relates to an extended reality virtual content provision system for interworking with an autonomous vehicle and metaverse, which, when driving autonomously, creates metaverse space using extended reality technology and provides a driver and a passenger with the created metaverse space. Through this, during autonomous driving, the driver and the passenger can easily check various information such as biometric information, weather information, and driving information visually. When necessary, the driver and the passenger can use games, SNS, video conferences, video calls, internet searches, and internet shopping and thus can utilize the free time generated through autonomous driving in various ways. To this end, the extended reality virtual content provision system for interworking with an autonomous vehicle and metaverse, according to the present invention, comprises: an autonomous driving unit mounted on a vehicle and automatically controlling the driving of the vehicle according to a selection of a driver; a biometric data collection unit mounted on the vehicle and collecting one or more pieces of biometric data selected from blood pressure, electrocardiogram, and body temperature of the driver; an immersive output unit which outputs extended reality data through glass equipment worn by the driver or a display device installed in the vehicle; a metaverse server unit which provides extended reality data of a predetermined size, in which an avatar of the driver participates while interworking with the immersive output unit through wireless Internet; and a motion detection unit which detects the motion of the driver and matches the motion with the avatar provided by the metaverse server unit.

Description

Extended Reality Virtual Content Provision System for Interworking with Autonomous Vehicles and Metaverse}

The present invention relates to a system for providing extended reality virtual content for linking an autonomous vehicle with a metabus, and more particularly, provides a metaverse space using an extended reality technology to drivers and passengers during autonomous driving to provide biometric information, weather information and driving It relates to an extended reality virtual content providing system for linking a self-driving vehicle and a metabus that visually provides various information such as information.

Extended reality technology (XR) refers to a technology that encompasses mixed reality (MR), which includes virtual reality technology (VR) and augmented reality technology (AR). Research on technologies for constructing a 3D virtual world like the world (metaverse) and servicing it in various fields is being actively conducted.

As an example of this, an augmented information simulator (hereinafter referred to as 'patent document') for providing improved UI/UX of a realistic HUD in Publication No. 10-2019-0063601 is disclosed.

The augmented information simulator disclosed in the patent document includes a visible light projector; screen; windshield; a first processing device providing image contents for the visible light projector to project a virtual driving image on the screen to the visible light projector; a second processing device generating ADAS information for the virtual driving image; a laser projector that projects a corresponding laser onto the windshield to display the ADAS information as an augmented form of a HUD; and a third processing device that is equipped with a camera and references the coordinates of the camera as the coordinates of the driver's point of view to generate matching information for displaying the distortion of the ADAS information displayed on the windshield in a corrected coordinate system. consists of including

However, the augmented information simulator disclosed in the above patent document is intended to verify a realistic HUD to be applied to an actual vehicle and check problems that may occur during driving through this, and for this purpose, ADAS (Advanced Driver Assistance System) information is generated and displayed on the HUD, but in this case, only the information of adjacent vehicles and the current vehicle driving information are provided in the driving image of the vehicle, and the driver and passengers are provided with I have a problem with not making full use of my free time.

In this case, the driver and passengers easily become drowsy driving due to spare time generated during autonomous driving, and as a result, there is a problem in that they cannot promptly respond to unexpected situations.

Therefore, it is necessary to develop a system for providing virtual content for autonomous vehicles that can provide various information by utilizing spare time during autonomous driving, reduce driver and passenger drowsiness, and consequently reduce accidents caused by autonomous driving. do.

KR 10-2019-0063601 A (2019. 06. 10.) KR 10-2317223 B1 (2021. 10. 19.) KR 10-2020-0083310 A (2020. 07. 08.) KR 10-2021-0047084 A (2021. 04. 29.) KR 10-2019-0095183 A (2019. 08. 14.) KR 10-2179661 B1 (2020. 11. 11.)

The present invention has been devised to solve the problems of the conventional self-driving system as described above, and the problem to be solved by the present invention is to create a metaverse space using extended reality technology during autonomous driving and provide it to drivers and passengers, Through this, various information such as biometric information, weather information, and driving information can be visually and easily checked during autonomous driving, and games, SNS, video conferences, video calls, Internet searches, and Internet shopping can be performed when necessary. It is to provide an extended reality virtual content provision system for interlocking self-driving vehicles and metabuses that allows the free time generated through autonomous driving to be utilized in various ways.

An extended reality virtual content providing system for linking an autonomous vehicle and a metabus according to the present invention for solving the above problems includes an autonomous driving unit mounted on a vehicle and automatically controlling driving of the vehicle by a driver's selection; a biometric data collection unit installed in the vehicle and collecting at least one biometric data selected from the driver's blood pressure, electrocardiogram, and body temperature; a tangible output unit configured to output augmented reality data through glasses worn by the driver or a display device provided in the vehicle; a metaverse server unit that provides extended reality data of a predetermined size in which the driver's avatar is active while being interlocked with the realistic output unit through wireless Internet; A motion sensor for detecting a driver's motion and matching the motion with an avatar provided by the metaverse server unit, wherein the metaverse server unit determines whether or not the autonomous driving unit is running when there is a request for the augmented reality data. and transmits the augmented reality data toward the immersive output unit only when the vehicle is controlled for autonomous driving.

In addition, when the biometric data collected by the biometric data collection unit of the present invention exceeds or falls below a set reference value, the self-driving unit of the present invention issues a warning to alert the driver, and if the abnormal condition continues for more than a preset time, the vehicle Another feature is to control the emergency mode.

In addition, in the present invention, the emergency mode of the vehicle searches for a hospital adjacent to the vehicle and then automatically changes the destination route for autonomous driving, or moves the vehicle to a shoulder and then ends the autonomous driving of the vehicle, and at the same time Another feature is that an emergency text message is automatically generated and transmitted to a contact set in an emergency contact network.

In addition, another feature of the present invention is that when the emergency text is automatically generated and transmitted, one or more pieces of information selected from GPS location information of the vehicle, abnormal type of biometric data, and emergency occurrence time are provided together.

And the present invention includes each of the extended reality data that allows the metaverse server to perform games, SNS, video conferences, video calls, Internet searches and Internet shopping, and the games, SNS, video conferences, video calls, Internet search and Internet shopping are configured so that the avatar moves in the virtual reality space on the expanded reality data or is switched through a set selection operation, and the game is configured so that a plurality of users share and use the virtual reality space. with different characteristics.

According to the present invention, there is an advantage in that various activities can be performed in the metaverse space during the spare time that occurs when the driver controls the vehicle by autonomous driving.

In addition, it is possible to prevent the driver from easily falling asleep during autonomous driving for a long time due to activities in the metabus space, and as a result, it is possible to prevent the driver from not being able to respond quickly due to not recognizing unexpected situations that may occur during autonomous driving.

In addition, since the driver's biometric data is collected in real time while the vehicle is driving, it is possible to quickly detect the driver's health problem, thereby preventing accidents caused by the driver's health problem in advance.

1 is a configuration diagram showing an example of an extended reality virtual content providing system for linking an autonomous vehicle and a metaverse according to the present invention.

Hereinafter, it will be described in detail according to the accompanying drawings showing a preferred embodiment of the present invention.

The present invention creates a metabus space using extended reality technology during autonomous driving and provides it to the driver and passengers, so that various information such as biometric information, weather information, and driving information can be visually and easily checked during autonomous driving. Extended reality virtual content for interworking with self-driving vehicles and metabus that allows users to use free time generated through autonomous driving in various ways by enabling city games, SNS, video conferences, video calls, Internet searches, and Internet shopping. It is intended to provide a provision system, and the present invention, as shown in FIG. A motion sensor 50 is included.

The autonomous driving unit 10 is a component that fully automatically controls the vehicle 1 to a destination without driver's intervention or automatically controls the vehicle 1 to a destination with minimal driver's intervention.

The autonomous driving unit 10 includes a speed maintenance function, an adaptive speed maintenance function, a lane maintenance function, a lane change function, and the like.

Here, the speed maintenance function is a function of maintaining the speed of the vehicle 1 in accordance with the speed set by the driver or the speed limit of the road, and the adaptive speed maintenance function is a function of decelerating the speed set according to the speed of the vehicle in front (preceding vehicle) to clear the road. It is a function to keep the distance, and the lane keeping function refers to a function to drive to the destination while maintaining the driving lane.

In addition, the lane change function is a function that automatically changes the driving lane when necessary while the vehicle 1 autonomously travels to the destination. Comprehensive judgment must be made based on data, and for this reason, sufficient review of safety according to various irregular situations occurring during autonomous driving must be made. It is preferable to configure so that the driver can separately activate or deactivate the lane change function through a setting option.

In addition, the autonomous driving unit 10 may further include an automatic parking function, and through this automatic parking function, the driver can automatically and safely park the vehicle 1 at a destination without manually parking.

When the driver sets a destination through the configuration of the autonomous driving unit 10 as described above, the vehicle 1 is automatically controlled to the destination using a speed maintenance function, an adaptive speed maintenance function, a lane maintenance function, and a lane change function. At this time, a state where the driver's hand is placed on the steering wheel (hereinafter referred to as 'hands-on state') is continuously sensed for the driver's quick manual operation in response to an unexpected situation.

Here, when the vehicle 1 is driven autonomously through the autonomous driving unit 10 and it is determined that the driver is using the augmented reality technology, the determination of the hands-on state is suspended.

At this time, the driver's hands-on state may be detected by various sensors such as a human body sensor installed along the steering wheel (steering handle) of the vehicle 1, a pressure sensor, and a temperature sensor. .

Alternatively, the positive and negative wires along the handle are installed at a certain interval so as not to form a closed loop, and then a closed loop connecting the positive and negative wires is formed by the driver gripping the handle to form a closed circuit. A current of may be configured to flow to determine whether the driver's hands are on.

The biometric data collection unit 20 is mounted on the vehicle 1 and is a component that collects biometric data such as blood pressure, electrocardiogram, and body temperature of the driver in real time.

The biometric data collection unit 20 may be configured as a separate wristband worn on the driver's wrist, etc., and determines the driver's health condition and stress levels through the collected biometric data, and such biometric data collection unit ( 20) may be configured to wirelessly communicate with the vehicle 1 via Bluetooth or the like.

In addition, the biometric data collection unit 20 detects the angle of the driver's head, eyelid movement and speed, etc., detects and predicts that the driver falls asleep during autonomous driving through this, and when it is determined that the driver is asleep or dozing off, outside air is detected. is introduced into the room to ventilate or generate a warning sound to induce the driver not to fall asleep or doze off.

As described above, in order to prevent the driver from falling asleep during autonomous driving, there may be an unexpected situation during autonomous driving or a situation requiring manual input from the driver. This is due to the fact that problems such as deviation from the route may occur, and for this reason, the present invention is configured to monitor the driver during autonomous driving in order to prevent accidents and route deviation that may occur during autonomous driving.

When the driver selects to autonomously drive the vehicle 1 to the destination through the autonomous driving unit 10, the realistic output unit 30 connects to the metaverse server unit 40 to be described later and provides expanded reality data through the It is a device that enables various experiences in a virtual reality space.

The realistic output unit 30 may be configured as a glass device manufactured in the form of glasses to be worn by a driver, or may be configured as a window of the vehicle 1 or a separate display device.

In addition, when AR data is output through the glass equipment or the window of the vehicle 1, stereoscopic data is output to the transparent glass and window, and when VR data is output, the glass and window become opaque, and then the VR data will be output

At this time, if the glass window of the vehicle 1 is completely opaque to output VR data, it is impossible to secure the front view of the self-driving vehicle, making it impossible to cope with an unexpected situation. Therefore, in this case, a certain portion where VR data is output A forward driving image of the vehicle 1 photographed in real time through the front camera may be output simultaneously.

The metaverse server unit 40 is a component that provides augmented reality data in real time to the realistic output unit 30 provided in the vehicle 1 .

This metaverse server unit 40 includes each of the augmented reality data so that the driver can perform games, SNS, video conferences, video calls, Internet searches, and Internet shopping by the driver's selection.

And, in the virtual reality space provided by the metaverse server unit 40, it is configured in the form of an open world that can be used by a number of users (users), including many unspecified people in autonomous driving, through which the driver can use various users in the virtual reality space. and you can play the game

In this case, it is preferable that the game is mainly composed of a table game in which the game is played in a seated state in consideration of the spatial characteristics of the inside of the driving vehicle, but is not limited thereto.

In addition, it can be configured to enable SNS, video conference, and video call by accessing the driver's personal account, and to enable Internet search and Internet shopping by accessing the Internet.

Activities such as games, SNS, video conferences, video calls, Internet searches, and Internet shopping allow the space to be converted by moving the driver's character (avatar) or performing a set selection action in the virtual reality space implemented as augmented reality data. can be configured.

The motion detection unit 50 is a component for detecting a driver's motion during autonomous driving and matching it with the motion of a character (avatar) provided from the metaverse server unit 40.

The motion detection unit 50 detects the driver's motion and detects the driver's hand and head motion and direction through a sensor installed or a motion-sensing wristband worn on both wrists of the driver, a glass device, or a hairband. It can be configured to be detected.

The driver's motion detected as above is transmitted to the metaverse server unit 40, through which the character (avatar) moves in conjunction with the driver's motion in real time.

On the other hand, when a request for extended reality data occurs, the metaverse server unit 40 determines whether or not autonomous driving is performed through the autonomous driving unit 10, and when the vehicle 1 is autonomously driving, the extended reality data is transmitted in a realistic way. It is provided to the output unit 40, and is configured not to provide extended reality data when the vehicle 1 is not driving autonomously.

In addition, the self-driving unit 10 issues a warning when the biometric data collected by the biometric data collection unit 20 is greater than or equal to a set reference value to warn the driver, and if the abnormal condition continues for more than a preset time , may be configured to control the vehicle 1 in an emergency mode.

At this time, the emergency mode of the vehicle may be configured to automatically search for a hospital adjacent to the vehicle and then automatically change the destination route of the vehicle 1 to autonomously drive to the hospital.

Alternatively, autonomous driving may be terminated after moving the vehicle 1 to a shoulder, and at the same time, an emergency text message may be automatically generated with a contact number set in advance in an emergency contact network such as 119 to notify an emergency situation.

In this way, when the autonomous driving is terminated and the vehicle 1 is stopped by moving to the shoulder, the emergency light is turned on and the door lock of the vehicle 1 is automatically unlocked so that emergency personnel can quickly find and rescue the vehicle. .

In addition, when an emergency text message is automatically generated and transmitted, one or more pieces of information selected from GPS location information of the vehicle 1, abnormal type of biometric data, and emergency occurrence time may be provided together.

On the other hand, the autonomous driving unit 10 can be controlled to automatically terminate the connection of the metaverse server unit 40 according to changes in the driving safety of the vehicle 1 and allow the driver to quickly recognize the driving situation. Such changes in driving safety may include sudden vehicle congestion (sudden deceleration), lighting of various inspection lights such as insufficient fuel in the vehicle 1, tire puncture, deviation from the route, and the like.

As described above, the present invention enables the driver to perform various activities in the metaverse space during the free time that occurs when the driver controls the vehicle by autonomous driving, and also prevents the driver from falling asleep easily during long-term autonomous driving due to activities in the metaverse space. As a result, it is prevented from not being able to respond quickly due to not recognizing an unexpected situation that may occur during autonomous driving.

In addition, since the driver's biometric data is collected in real time while the vehicle is driving, it is possible to quickly detect the driver's health problem, thereby preventing accidents caused by the driver's health problem.

In the above, reference numerals and names have been given to the drawings showing preferred embodiments and the components shown in the drawings for convenience of explanation, but this is an embodiment according to the present invention, which corresponds to the shape shown on the drawings and the given name. The scope of the right should not be construed as being limited, and the change to various shapes predictable from the description of the invention and the simple substitution with a configuration that has the same action are within the range of change to be easily performed by those skilled in the art. You will find it extremely self-evident.

1: vehicle 10: autonomous driving unit
20: biometric data collection unit 30: realistic output unit
40: metaverse server unit 50: motion detection unit

Claims (3)

An autonomous driving unit 10 mounted on a vehicle and automatically controlling driving of the vehicle by a driver's selection;
a biometric data collection unit 20 mounted on a vehicle and collecting one or more biometric data selected from the driver's blood pressure, electrocardiogram, and body temperature;
a realistic output unit 30 that outputs augmented reality data through glasses worn by the driver or a display device provided in the vehicle;
A metaverse server unit 40 that provides extended reality data of a predetermined size in which the driver's avatar is active while being linked with the realistic output unit 30 through the wireless Internet;
A motion detection unit 50 for detecting the driver's motion and matching the motion with the avatar provided by the metaverse server unit 40;
including,
The metaverse server unit 40,
When there is a request for the extended reality data, it is determined whether the autonomous driving unit 10 is executed, and the extended reality data is transmitted to the tangible output unit 40 only when the vehicle is controlled for autonomous driving. An extended reality virtual content providing system for linking an autonomous vehicle with a metabus, characterized in that.
The method of claim 1,
The autonomous driving unit 10,
When the biometric data collected by the biometric data collection unit 20 is greater than or less than a set reference value, a warning is generated to alert the driver, and if the abnormal condition continues for more than a preset time, the vehicle is controlled in an emergency mode. An extended reality virtual content providing system for linking an autonomous vehicle with a metabus, characterized in that.
The method of claim 1,
The metaverse server unit 40,
It includes each extended reality data that enables games, SNS, video conferences, video calls, internet searches and internet shopping,
The game, SNS, video conference, video call, Internet search, and Internet shopping are configured so that the avatar moves in the virtual reality space on the expanded reality data or is switched through a set selection operation,
The game,
An extended reality virtual content providing system for linking a self-driving vehicle and a metaverse, characterized in that a plurality of users are configured to share and use virtual reality space.

Images