KR20210062774A - Smart car see-through display control system and method thereof - Google Patents

Abstract

Disclosed are a system and a method for controlling a see-through display of a smart car. According to an embodiment, the system and the method for controlling a see-through display of a smart car comprehensively consider a behavior shape of a user of a smart car and a driving situation, driving information, and state information of a vehicle to extract a display area in a see-through multi-screen mounted in the smart car and optimally control detailed information of content output, thereby improving display efficiency in the smart car and more conveniently using entertainment information search and delivery. Moreover, the present invention reflects a trend of a smart display and a user behavior analysis result in a general driving mode to improve performance and efficiency of a smart car display system.

Description

Smart car see-through display control system and method {SMART CAR SEE-THROUGH DISPLAY CONTROL SYSTEM AND METHOD THEREOF}

It relates to a see-through display control system for a smart car, and more specifically, to a display control system and method for selecting a see-through display area among multi-screens of a smart car and setting detailed information of output contents according to driving information and location information of a vehicle user will be.

Unless otherwise indicated herein, the material described in this section is not prior art to the claims of this application, and inclusion in this section is not an admission that it is prior art.

Recently, at representative information and communication technology (ICT)-related expositions such as the Consumer Electronics Show (CES) and the Mobile World Congress (MWC), automakers set up exclusive booths and introduced state-of-the-art infotainment systems, and the smart car industry is booming. is rising This trend is not unique to the automotive industry. Various operators related to smart cars, such as chipset and solution companies, participated in large numbers, showing high interest in smart cars of information and communication companies. Recently, the smart car industry is raising expectations from various experts that 'the automobile will be a representative smart terminal in the future and will lead a new growth industry in the future'.

The blooming of the smart ecosystem is expected to provide new business opportunities not only for the automobile industry, but also for all related industries. The initial smart car market is expected to evolve by forming an independent ecosystem due to the characteristics of the automobile industry, but as the connectivity between smart devices expands, it is expected to have similar characteristics to the smartphone ecosystem. In a smart environment where all terminals are connected, smart cars are evolving into another screen following smart phones, tablet PCs, and smart TVs.

As a new screen, the car will eventually have a mobile ecosystem as a new smart device. As all devices are connected and cloud-based services are activated, smart cars are also expected to become an important component of multi-screen (N-screen). In this respect, in the short term, the market for smart cars is expected to be formed at the level of providing various in-vehicle infotainment services linked to smartphones.

The recently launched smart car helps safe and comfortable driving through specialized services for automobiles and provides geographic information services such as telematics and location information service (LBS). You can also use various services such as social network services (SNS), games, and multimedia. In addition, the scope of service provision is also expanding from driver-oriented to passenger. As the demand for various infotainment services increases, the multi-screen terminal of vehicles is also in progress.

As such, car selection criteria are gradually changing from the hardware point of view such as performance, durability, design, and fuel efficiency to convenience and connectivity using information and communication technology. Therefore, smart car communication with the inside and outside of the vehicle is increased by strengthening interworking with networks and mobile devices, and the convenience of use is increasing by improving the user interface (UI). As consumers experience increased value through network connection and content consumption through it, the connectedization of automobiles is also inevitable.

In terms of market demand, customers who enjoyed the various services of smartphones are demanding the same experience in the car. As the automobile is perceived as another living space, the desire to transfer the smartphone experience to the automobile is also very large. In fact, there are many studies that show that the desire of drivers to consume various service experiences such as SNS, LBS, and multimedia even in the physical space of the vehicle is increasing recently. In addition, technology development that applies the latest display technology to smart cars, such as research on door trim using see-through display and touch technology, is active.

1. Laid-open Patent Publication No. 10-2017-0044471 (2017.04.25.) 2. Patent Publication No. 10-2019-0123368 (2019.11.01)

In the embodiment, an area in which content is displayed among the see-through multi-screens is selected according to vehicle driving information, state information, user setting, and location information of the user, and the content output to the display area is controlled. In addition, the see-through display control system and method of a smart car according to the embodiment provides a display-utilizing entertainment, information search, and information delivery system, and proposes a door trim system for a smart car to which a see-through concept is applied. In addition, by providing a display control method according to the result of user behavior analysis (Focus group discuss, Persona, Customer journey map) through the smart display in the smart car, in the smart car, the user's personal content, vehicle status information, and geographic information of the driving area and telematics, etc., so that various contents can be conveniently provided.

The see-through display control system of the smart car according to the embodiment identifies the user's position and boarding position, and biometric information including gaze, gesture, and fatigue, and predicts the user's next action related to the user's behavioral intention and driving process. behavioral information analysis module; a driving situation analysis module for understanding the driving position and surrounding driving situation of the smart car through the navigation system and sensors and cameras mounted on the smart car; a vehicle state information analysis module for identifying driving information including vehicle speed, driving state, gear state, and driving direction according to driving conditions and user control and autonomous driving, and vehicle state information including remaining fuel and battery charge; and selecting a display area for content among the multi-screens installed in the smart car according to user behavior information, vehicle driving situation and driving information, and outputting content including vehicle driving situation, driving information, and user control information to the selected display area. display control module; Includes.

In the see-through display control system of a smart car according to another embodiment, the user behavioral information analysis module recognizes the location and boarding position of the smart car user, and biometric information including gaze, gesture, and fatigue, so that the user's behavioral intention and driving process and a first step of predicting an associated user's next action; The driving situation analysis module comprises: a second step of identifying the driving position and surrounding driving situation of the smart car through the navigation system and sensors and cameras mounted on the smart car; The vehicle state information analysis module includes a third step of identifying driving information including vehicle speed, driving state, gear state, and driving direction according to driving conditions and user control and autonomous driving, and vehicle state information including remaining fuel and battery charge; and the display control module selects a display area for content among the multi-screens installed in the smart car according to user behavior type information, vehicle driving situation and driving information, and includes the vehicle driving situation, driving information, and user control information in the selected display area. a fourth step of outputting content; Includes.

The see-through display control system and method of a smart car according to an embodiment extracts the display area from the see-through multi-screen mounted on the smart car by comprehensively considering the behavior of the smart car user, the driving situation of the vehicle, driving information, and status information, , by optimally controlling content output details, it improves display efficiency in smart cars and makes entertainment information retrieval and information delivery more convenient.

In addition, the smart car display system performance and efficiency are improved by reflecting the smart display trend and user behavior analysis results in the normal driving mode.

The effects of the present invention are not limited to the above effects, and should be understood to include all effects that can be inferred from the configuration of the invention described in the detailed description or claims of the present invention.

1 is a view showing a data processing block of a see-through display control system of a smart car according to an embodiment;
2 is a view showing an example of outputting smart car control contents of a see-through display control system of a smart car according to an embodiment;
3 is a view showing a data processing block of the behavioral information analysis module 100 according to the embodiment;
4 is a diagram illustrating a data processing configuration of a display control module according to an embodiment;
5 is a view showing an example of content output according to the setting of detailed display information;
6 is a flowchart illustrating a data processing flow of a method for controlling a see-through display of a smart car according to an embodiment;
7 is a view showing an implementation example of a see-through display control system of a smart car;

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments make the disclosure of the present invention complete, and the general knowledge in the technical field to which the present invention pertains. It is provided to completely inform the scope of the invention to the possessor, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

In describing the embodiments of the present invention, if it is determined that a detailed description of a well-known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the terms to be described later are terms defined in consideration of functions in an embodiment of the present invention, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the contents throughout the present specification.

1 is a diagram illustrating a data processing block of a see-through display control system of a smart car according to an embodiment.

Referring to FIG. 1 , the see-through display control system of a smart car according to the embodiment includes a behavioral information analysis module 100 , a driving situation analysis module 200 , a vehicle state information analysis module 300 , and a display control module 400 . It may be composed of As used herein, the term 'module' should be construed to include software, hardware, or a combination thereof, depending on the context in which the term is used. For example, the software may be machine language, firmware, embedded code, and application software. As another example, the hardware may be a circuit, a processor, a computer, an integrated circuit, an integrated circuit core, a sensor, a Micro-Electro-Mechanical System (MEMS), a passive device, or a combination thereof.

The user behavioral information analysis module 100 predicts the user's behavioral intention and the next behavior by grasping the location, gaze, gesture, and biometric information of the user who is a registered user of the smart car and the passenger. For example, the user behavior information analysis module 100 builds in advance a database such as a driving process according to a user gesture and user gesture and gaze information predicted in each driving process. Specifically, in the driving start process, the user's gaze is directed to the cluster and the heads-up display, and behavioral information such as the user wearing a seat belt or inputting route information into the navigation system may be established in advance. That is, in the embodiment, the user's behavioral form and gesture sequence corresponding to the process of starting, driving, and ending driving, etc. may be established in advance in the database. In addition, in the embodiment, it is possible to predict other actions performed after a specific action for each user by identifying a behavior pattern for each user. This is data constructed to predict the user's next behavior in advance according to the driving process and user behavior when specific behavioral information of a user using a smart car is recognized, and to output smart car control contents and various contents for prediction in an appropriate location. . For example, when behavior information such as wearing a seat belt or inputting a navigation system is set in the performance start stage, and a user's seat belt fastening behavior is detected in the driving start stage of the smart car, vehicle driving is displayed in the left display area where the seat belt is installed. It enables to output initial information and vehicle setting contents for

The driving situation analysis module 200 detects the driving position and surrounding driving situation of the smart car through the navigation system and the sensors and cameras mounted on the smart car. In an embodiment, the driving situation analysis module 200 collects monitoring information around the vehicle through a sensor and a camera installed in the vehicle, and according to the driving path of the vehicle, the driving path such as left turn, right turn, straight forward, backward, etc. and objects existing around the vehicle get the information

The vehicle state information analysis module 300 recognizes driving information including vehicle speed, driving state, gear state, and driving direction according to driving conditions and user control and autonomous driving, and vehicle state information including remaining fuel and battery charge. In addition, the vehicle state information may include notification information about abnormal situations and emergency situations such as vehicle driving information change rate, fuel or battery shortage, route departure, and collision.

The display control module 400 selects a display area from among the multi-screens installed in the smart car according to user behavior information, vehicle driving situation and driving information, and includes the vehicle driving situation, driving information, and user control information in the selected display area. Outputs smart car control contents. A smart car according to an embodiment includes a see-through display multi-screen on a screen provided in a cockpit, a door trim, and a seat of the smart car. The see-through display multi-screen provides smart car control information to users by floating data such as text and figures on transparent glass. In the embodiment, an optical see-through device is provided, so that the real world and the virtual world are overlapped and displayed. In the optical see-through display, the user can see the virtual object (information) displayed on the transparent display by superimposing it while looking outside, and in an embodiment, the transparency of the see-through display can be automatically set according to the brightness of the vehicle surrounding environment and the user's eyesight. .

2 is a diagram illustrating an example of outputting smart car control contents of a see-through display control system of a smart car according to an embodiment.

Referring to FIG. 2A , the see-through display control system of the smart car according to the embodiment may control the display area according to the location of the user. In the embodiment, as shown in (a) of FIG. 2, when the user tries to check the inside of the vehicle in the process before boarding the smart car, the transparency of the vehicle's head-up display and the see-through display of the window and door is adjusted, and the display area Enlarge to allow the user to check the entire interior of the vehicle.

In addition, referring to FIG. 2B , the see-through display control system of the smart car is the number of users when there are several users in the smart car or when the user switches the smart car to the autonomous driving mode and moves to the back seat. The display area for outputting the smart car control contents can be adjusted according to and the boarding position. In addition, in the embodiment, the digital content that the user wants to check may be output to the see-through display area by interworking with the user's smart device in the autonomous driving mode. In addition, in a smart car in which many people ride, the display area allocated to each seat according to the seat position may be set as the door trim of each seat or the screen of the front seat.

Referring to FIG. 2C , the see-through display control system of the smart car according to the embodiment may output vehicle driving information or information to be recognized according to a user's gaze to the display area of the vehicle door trim. As shown in (c) of FIG. 2 , in the embodiment, the user changes his/her line of sight toward the right side mirror in order to check the vehicle approaching the rear of the right lane, operates the right turn indicator, or information on the driving route When it is determined that a right direction change is necessary in the , the vehicle in the right lane, which is conventionally reflected in the right side mirror, may be output on the display area of the door trim of the right passenger seat. At this time, driving guide information such as the distance between the vehicle approaching from the right lane and the smart car in which the user is riding, the speed of the rear object, and whether it is suitable for cutting may be output together on the see-through display area of the door trim, and the driving road Detailed information such as transparency and sharpness of the screen output to the door trim display area may be automatically set according to the illuminance of the vehicle and the brightness of the interior of the vehicle.

3 is a diagram illustrating a data processing block of the behavioral form information analysis module 100 according to an embodiment.

Referring to FIG. 3 , the behavioral information analysis module 100 according to the embodiment may include a sensing unit 110 , a comparison unit 130 , and a determination unit 150 . The sensing unit 110 detects gestures and positions of smart car occupants and users registered in the smart car system, and senses biometric information such as heart rate, gaze information, fatigue, and blood pressure.

The comparison unit 130 compares the user's gesture, location information, and biometric information detected by the sensing unit 110 with pre-stored user behavioral information. The determination unit 150 determines the driving process such as driving start, end, and driving according to the comparison result and the state information of the smart car, and predicts the next action of the user according to the identified driving process. Thereafter, the determination unit 150 extracts smart car control information predicted to be required by the user corresponding to the predicted driving process and the user's next action, vehicle state information, or user personal content stored in the user terminal.

4 is a diagram illustrating a data processing configuration of a display control module according to an embodiment, and FIG. 5 is a diagram illustrating an example of content output according to the setting of detailed display information.

Referring to FIG. 4 , the display control module according to the embodiment may include an information collecting unit 410 , an extracting unit 430 , and a setting unit 450 . The information collection unit 410 collects user behavior form information, user behavior prediction information, and vehicle driving information for output control and display content control of a multi-display area provided in the smart car. The extraction unit 430 extracts smart car control content or user personal content to be output to the see-through display area according to the collected user behavior form information, behavior prediction information, and vehicle driving information. The setting unit 450 sets detailed information of smart car control content or user personal content to be output and an area to be output from the smart car. Specifically, the setting unit 450 sets detailed information for improving the display quality of the output content, such as font, color, screen transparency, and brightness of the output content.

5 is a view showing an example in which the design, user interface, font size, color, sharpness, brightness, etc. of the content are set differently and displayed according to the detailed output content. As shown in FIG. 5, in the embodiment, the smart car is Weather information of a driving area, state information of vehicle door trim, and the like may also be output.

In addition, the setting unit 450 sets an area in which the output content is to be displayed according to the user's behavior prediction information and gaze information. For example, the setting unit 450 may display the see-through display area of the left door trim or the left display area of the driver when the gaze of the user riding in the driver's seat is sensed to the left and a left turn is scheduled on the expected driving route or a left direction indication is driven. It is set as an area for outputting driving information or user personal contents.

In the embodiment, the display control module 400 provides driving guidance information such as speed limit, school zone guidance, and one-way guidance according to the driving situation and the location of the smart car to the display area according to the tracking result and behavior prediction information by tracking the user's gaze. can be printed out. In addition, content that the user desires to access may be output in conjunction with the user's digital device.

Hereinafter, a method for controlling a see-through display of a smart car will be described in turn. Since the operation (function) of the method for controlling the see-through display of the smart car according to the embodiment is essentially the same as the function of the see-through display control system of the smart car, the description overlapping with FIGS. 1 to 3 will be omitted.

6 is a flowchart illustrating a data processing flow of a method for controlling a see-through display of a smart car according to an embodiment.

In step S100, behavioral information including the user's location, behavior, and gesture and gaze is identified from the image generated by the sensor and camera installed in the smart car, and the identified behavioral information is compared with the pre-stored behavioral behavioral information to form the user's behavior. It analyzes information and predicts the user's next action. For example, in step S100, the user's next gaze area or the next action is predicted, so that the output content appears on the display area included in the user's gaze movement path.

In step S200, driving conditions such as the location of the smart car, driving road conditions, and traffic conditions are analyzed, and driving conditions such as vehicle speed, sudden stop, driving direction, and gear status are analyzed.

In step S300, a display area in which smart car control contents and user personal contents are to be output is extracted according to the analysis result of driving situation information, vehicle status information, and user behavior type information. In an embodiment, the user's personal content may include all content that can be output by interworking with the user's smart device, such as a calendar, schedule, document, and the Internet.

In step S400, output details to be output of the content to be output to the see-through display area are set. In an embodiment, the detailed output information may include detailed setting information such as the size, text color, darkness, and transparency of an object output to the display area. In addition, step S400 is a process of converting the size and ratio of personal content to correspond to the size and aspect ratio of the extracted display area when the user's personal content is output to the see-through display area of the smart car in conjunction with the user's digital terminal may be included.

In step S500, smart car control content, vehicle state information, telematics information and external environment information content, personal content, etc. are output from the see-through display area of the smart car according to the set detailed information.

7 is a view showing an implementation example of a see-through display control system of a smart car.

As shown in (a) of FIG. 7 , the see-through display control system of the smart car according to the embodiment is displayed on the smart car control content, vehicle state information, vehicle driving information, content indicating vehicle surrounding environment information, and the user's smart phone. Various visual objects including the stored user personal content may be outputted from the see-through display area included in the smart car and provided to the user.

As shown in (b) of Figure 7, the smart car according to the embodiment is provided with multi-screen and see-through display areas in various parts of the vehicle such as a window, a head-up display, a cluster, a door trim, a seat, and the user's gestures and actions And according to biometric information such as gaze, the content can be output to a place that can be easily recognized by the user.

The see-through display control system and method of a smart car according to an embodiment extracts the display area from the see-through multi-screen mounted on the smart car by comprehensively considering the behavior of the smart car user, the driving situation of the vehicle, driving information, and status information, , by optimally controlling content output details, it improves display efficiency in smart cars and makes entertainment information retrieval and information delivery more convenient. In addition, the smart car display system performance and efficiency are improved by reflecting the smart display trend and user behavior analysis results in the normal driving mode.

The disclosed content is merely an example, and can be variously changed and implemented by those skilled in the art without departing from the gist of the claims claimed in the claims, so the protection scope of the disclosed content is limited to the specific It is not limited to an Example.

Claims (10)

In the see-through display control system of a smart car,
a user behavior form information analysis module for predicting the user's next behavior related to the user's behavioral intention and driving process by identifying the smart car user's location and boarding position, and biometric information including gaze, gesture, and fatigue;
a driving situation analysis module for understanding the driving position of the smart car and surrounding driving conditions through the navigation system and sensors and cameras mounted on the smart car;
a vehicle state information analysis module for identifying driving information including vehicle speed, driving state, gear state, and driving direction according to driving conditions and user control and autonomous driving, and vehicle state information including remaining fuel and battery charge; and
A display that selects a content display area from among the multi-screens installed in the smart car according to user behavior information, vehicle driving situation and driving information, and outputs content including vehicle driving situation, driving information, and user control information to the selected display area control module; A see-through display control system of a smart car comprising a.
According to claim 1, wherein the see-through display control system of the smart car
see-through display multi-screen on the cockpit, door trim and vehicle seat screens of smart cars; The see-through display control system of the smart car, characterized in that it further comprises.
The method of claim 1, wherein the user behavior information analysis module;
It interlocks with the user's smart terminal registered in the smart car to determine the user's location, and the pre-stored driving process including boarding, driving start, driving start, and driving end, and user behavior information included in each driving process and sensed By comparing the user's behavioral information, the user's behavioral information is identified according to the comparison result, and the user's next
the control module;
A see-through display control system for a smart car, characterized in that the display area is selected according to the predicted user's next action and location information.
The method of claim 1, wherein the display control module;
By grasping vehicle driving information, extracting a see-through display area corresponding to the driving direction and movement path of the vehicle, and outputting smart car control contents to the extracted see-through display area,
Smart car see-through, characterized in that any one of smart car control content, vehicle status information, and content linked to the user's smart terminal is output to the see-through display area of the door trim in the direction in which the user is detected among the left and right door trims display control system.
The method of claim 1, wherein the display control module;
A see-through display control system for a smart car, characterized in that the user's gaze information and behavior information including gestures of a user riding in the smart car are identified, and the smart car control content is output to a display area included in the user's gaze movement path.
In the see-through display control system of a smart car,
The user behavioral information analysis module includes: a first step of predicting the next user's behavior related to the user's behavioral intention and driving process by identifying the smart car user's location and boarding position, and biometric information including gaze, gesture, and fatigue;
The driving situation analysis module includes: a second step of identifying the driving position and surrounding driving situation of the smart car through the navigation system and sensors and cameras mounted on the smart car;
The vehicle state information analysis module includes a third step of identifying driving information including vehicle speed, driving state, gear state, and driving direction according to driving conditions and user control and autonomous driving, and vehicle state information including remaining fuel and battery charge; and
The display control module selects a content display area from among the multi-screens installed in the smart car according to user behavior information, vehicle driving situation and driving information, and includes the vehicle driving situation, driving information, and user control information in the selected display area. A fourth step of outputting; A see-through display control method of a smart car comprising a.
The method of claim 6, wherein the first step;
It interlocks with the user's smart terminal registered in the smart car to determine the user's location, and the pre-stored driving process including boarding, driving start, driving start, and driving end, and user behavior information included in each driving process and sensed Step 1-1 of comparing the user's behavioral information, grasping the user's behavioral information according to the comparison result, and predicting the user's next behavior; and
The method for controlling a see-through display of a smart car, comprising: a step 1-2 of selecting a display area according to the user's next action and location information predicted by the control module.
7. The method of claim 6, further comprising: the fourth step; Is
a step 4-1 of identifying vehicle driving information, extracting a see-through display area corresponding to the driving direction and moving path of the vehicle, and outputting smart car control contents to the extracted see-through display area; and
a step 4-2 of outputting any one of smart car control contents, vehicle status information, and contents linked to the user's smart terminal to the see-through display area of the door trim in the direction in which the user is detected among the left and right door trims; A see-through display control method of a smart car, characterized in that
7. The method of claim 6, further comprising: the fourth step; Is
A method for controlling a see-through display of a smart car, characterized in that the user's gaze information and behavior information including gestures of a user riding in the smart car are identified, and the smart car control content is output to a display area included in the user's gaze movement path.
7. The method of claim 6, further comprising: the fourth step; Is
A see-through display of a smart car, characterized in that the rear object appearing on the left or right side mirror is output to the display area of the left or right smart car door trim according to the driving direction of the smart car, the driving prediction path, and whether the left and right turn indicators are operated control method.

Images