KR102592675B1 - Method, system, and non-transitory computer-readable recording medium for providing contents - Google Patents

Abstract

According to one aspect of the present invention, a method of providing content, comprising: determining an image sequence regarding a virtual object with reference to at least one of situation information of at least one vehicle occupant and driving information of the vehicle; and A method is provided that includes determining content to be provided to the at least one vehicle occupant by matching an image sequence regarding the virtual object based on a real world outside the vehicle projected through at least one display.

Description

Methods, systems, and non-transitory computer-readable recording media for providing content {METHOD, SYSTEM, AND NON-TRANSITORY COMPUTER-READABLE RECORDING MEDIUM FOR PROVIDING CONTENTS}

The present invention relates to methods, systems, and non-transitory computer-readable recording media for providing content.

In recent years, with the development of communication and artificial intelligence (AI) technology, the autonomous vehicle market is becoming active, with autonomous vehicle driving services being commercialized.

These self-driving vehicles are equipped with numerous cameras to detect surrounding obstacles or determine the state of the vehicle's interior, through which the interior or exterior of the vehicle is continuously filmed.

However, the video recorded as above is simply deleted or overwritten with a new video immediately after driving the vehicle, and is not properly used for other purposes.

In addition, in recent years, as self-driving vehicles are commercialized and distributed, the time spent by vehicle occupants directly driving is decreasing, so there is an urgent need to develop content that can alleviate the boredom and stimulate interest of vehicle occupants while driving the vehicle. The situation is becoming.

Accordingly, the present inventor(s) propose a new and advanced technology that can interactively provide content to vehicle occupants while the vehicle is driving.

Korean Patent Publication No. 2018-0014910

The purpose of the present invention is to solve all the problems of the prior art described above.

In addition, the purpose of the present invention is to relieve the boredom and boredom of vehicle occupants that may occur while driving a vehicle and to increase the usability of vehicle images.

Additionally, the purpose of the present invention is to enable content to be provided in synchronization on at least two displays in a vehicle.

Additionally, the purpose of the present invention is to provide content for each vehicle occupant or each predetermined group of vehicle occupants.

Additionally, the purpose of the present invention is to support vehicle occupants to easily create and share content within the vehicle.

A representative configuration of the present invention to achieve the above object is as follows.

According to one aspect of the present invention, a method of providing content, comprising: determining an image sequence regarding a virtual object with reference to at least one of situation information of at least one vehicle occupant and driving information of the vehicle; and A method is provided that includes determining content to be provided to the at least one vehicle occupant by matching an image sequence regarding the virtual object based on a real world outside the vehicle projected through at least one display.

According to another aspect of the present invention, a system for providing content, comprising: a virtual object sequence management unit that determines an image sequence for a virtual object by referring to at least one of situation information of at least one vehicle occupant and driving information of the vehicle; and A system including a content manager that determines content to be provided to the at least one vehicle occupant by matching an image sequence regarding the virtual object based on the external real world of the vehicle projected through at least one display of the vehicle. provided.

In addition, another method for implementing the present invention, another system, and a non-transitory computer-readable recording medium for recording a computer program for executing the method are further provided.

According to the present invention, it is possible to relieve the boredom and boredom of vehicle occupants that may occur while driving a vehicle and increase the usability of vehicle images.

Additionally, according to the present invention, it is possible to synchronize and provide content on at least two displays in a vehicle.

Additionally, according to the present invention, it is possible to provide content to each vehicle occupant or to each predetermined group of vehicle occupants.

Additionally, according to the present invention, it is possible to support vehicle occupants to easily create and share content within the vehicle.

1 is a diagram showing a schematic configuration of an entire system for providing content according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating in detail the internal configuration of a content provision system according to an embodiment of the present invention.
3 to 8 are diagrams illustrating a situation in which content is provided according to an embodiment of the present invention.

The detailed description of the present invention described below refers to the accompanying drawings, which show by way of example specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different from one another but are not necessarily mutually exclusive. For example, specific shapes, structures and characteristics described herein may be implemented with changes from one embodiment to another without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description described below is not to be taken in a limiting sense, and the scope of the present invention should be taken to encompass the scope claimed by the claims and all equivalents thereof. Like reference numbers in the drawings indicate identical or similar elements throughout various aspects.

In this specification, content is a concept that collectively refers to digital information or individual information elements consisting of letters, codes, voices, sounds, images, videos, etc. Such content may include, for example, data such as text, images, videos, audio, links (eg, web links), or a combination of at least two of these data.

Hereinafter, several preferred embodiments of the present invention will be described in detail with reference to the attached drawings in order to enable those skilled in the art to easily practice the present invention.

Configuration of the entire system

1 is a diagram showing a schematic configuration of an entire system for providing content according to an embodiment of the present invention.

As shown in FIG. 1, the entire system according to an embodiment of the present invention may be configured to include a communication network 100, a content provision system 200, and at least one display 300.

First, the communication network 100 according to an embodiment of the present invention can be configured regardless of communication mode, such as wired communication or wireless communication, and can be used as a local area network (LAN) or a metropolitan area network (MAN). ), and a wide area network (WAN). Preferably, the communication network 100 referred to in this specification may be the known Internet or World Wide Web (WWW). However, the communication network 100 is not necessarily limited thereto and may include at least a portion of a known wired or wireless data communication network, a known telephone network, or a known wired or wireless television communication network.

For example, the communication network 100 is a wireless data communication network, including radio frequency (RF) communication, WiFi communication, cellular (LTE, etc.) communication, and Bluetooth communication (more specifically, Bluetooth low energy (BLE) communication). Conventional communication methods such as Bluetooth Low Energy (Bluetooth Low Energy) communication, infrared communication, ultrasonic communication, etc. may be implemented at least in part.

Next, the content providing system 200 according to an embodiment of the present invention may be a digital device equipped with a memory means and a microprocessor to have computing capabilities. For example, this content providing system 200 may be a server system or a vehicle electronic control system (ECU).

The content providing system 200 according to an embodiment of the present invention determines an image sequence related to a virtual object by referring to at least one of situation information of at least one vehicle occupant and driving information of the vehicle, and displays at least one display of the vehicle. By matching the image sequence regarding the above virtual object based on the external real world of the vehicle projected through 300, a function of determining content to be provided to at least one vehicle occupant can be performed.

Here, the virtual object according to an embodiment of the present invention is a concept that collectively refers to all objects that can be implemented on at least one display in a vehicle, for example, letters, numbers, signs, symbols, shapes, photos, It may consist of an image, sound, video, or a combination thereof, and may be a concept that even includes attributes that can be assigned to it (e.g., brightness, brightness, saturation).

The configuration and function of the content provision system 200 according to the present invention will be discussed in more detail below. Meanwhile, although the content providing system 200 has been described as above, this description is illustrative, and at least some of the functions or components required for the content providing system 200 may be included in at least one display 300 to be described later as needed. ) or an external system (not shown), or may be included in at least one display 300 or an external system above.

Next, at least one display 300 according to an embodiment of the present invention may perform a function of displaying content determined by the content providing system 200.

For example, at least one display 300 according to an embodiment of the present invention includes a transparent (e.g., 5 to 80% transparency) or translucent display on which at least a portion of the real world outside the vehicle can be projected. It can be done, and more specifically, it may be composed of transparent or translucent TFELD (Thin Film Electroluminescent Display), OLED, LCD, LED, etc.

For another example, at least one display 300 according to an embodiment of the present invention projects the external real world of the vehicle by displaying an image (e.g., a real-time captured image) related to the external real world of the vehicle on an opaque display. It can also perform the function

In addition, a plurality of the above displays 300 may be disposed in the vehicle, for example, disposed on the front, left, right, and left sides of the vehicle (or at least one vehicle occupant) (i.e., at least four displays are disposed). It can be. Additionally, this display 300 may be additionally mounted around a conventional vehicle window or may be installed as a replacement for the window.

Meanwhile, at least one display 300 according to an embodiment of the present invention may be a digital device equipped with a memory means and a microprocessor equipped with computing capabilities, and may have a content determination function (or content provision function) according to the present invention. Applications to support functions) may be included. Such an application may be downloaded from the content provision system 200 or an external application distribution server (not shown).

Configuration of content provision system

Below, we will look at the internal structure of the content providing system 200 and the function of each component, which performs important functions for implementing the present invention.

FIG. 2 is a diagram illustrating in detail the internal configuration of the content providing system 200 according to an embodiment of the present invention.

As shown in FIG. 2, it may include a virtual object sequence management unit 210, a content management unit 220, a communication unit 230, and a control unit 240. According to one embodiment of the present invention, at least some of the virtual object sequence management unit 210, content management unit 220, communication unit 230, and control unit 240 may be program modules that communicate with an external system. These program modules may be included in the content provision system 200 in the form of operating systems, application modules, or other program modules, and may be physically stored in various known storage devices. Additionally, these program modules may be stored in a remote memory device capable of communicating with the content providing system 200. Meanwhile, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform specific tasks or execute specific abstract data types according to the present invention.

First, the virtual object sequence management unit 210 according to an embodiment of the present invention may perform a function of determining an image sequence for a virtual object by referring to at least one of the situation information of at least one vehicle occupant and the driving information of the vehicle. You can. The situation information of the vehicle occupant according to an embodiment of the present invention may include information representing the situation inferred from the vehicle occupant (e.g., the situation inferred from the vehicle occupant's gaze, facial expression, behavior, motion, sound, etc.) there is. For example, a case database may be searched for cases that match information that is identical or similar to information about the vehicle occupants' gaze, facial expressions, actions, movements, etc., or machine learning (machine learning) may be conducted on the vehicle occupants' gaze, facial expressions, actions, movements, sounds, etc. The corresponding case can be specified based on the results learned through machine learning or deep learning, and information representing the above situation can be specified based on the specified case. Additionally, the vehicle's driving information according to an embodiment of the present invention may include information about the vehicle's origin, destination, current location, driving speed, and driving status (eg, start, stop, etc.). Meanwhile, the virtual object sequence management unit 210 specifies at least one vehicle occupant who is the target of situation information acquisition based on the internal image of the vehicle (for example, specifies the vehicle occupant whose movements such as gaze and facial expression occur). Or, perform a predetermined input action (for example, touching or manipulating the display 300 or a separate external system) within the vehicle, or perform a predetermined motion (in this case, the motion may be performed using a predetermined weight sensor, accelerometer sensor, etc. The vehicle occupant taking the vehicle (can be detected by, etc.) may be identified as at least one vehicle occupant subject to situation information acquisition. However, it should be noted that the above specific methods according to the present invention are not necessarily limited to those listed above, and may be changed in various ways within the scope of achieving the purpose of the present invention.

For example, the virtual object sequence management unit 210 analyzes at least one of the gaze, facial expression, motion, action, and sound of at least one vehicle occupant based on the internal image of the vehicle (or displays the vehicle occupant within the vehicle) By analyzing the act of making a predetermined input to 300), the emotions of the occupants of the vehicle can be classified into emotions (e.g., joy, anger, sadness, joy, etc.) ) or the surrounding atmosphere is specified as situation information of the occupants of the vehicle, and the virtual object associated with the situation information performs an action corresponding to the emotion or atmosphere. It can be determined as an image sequence for the virtual object. .

More specifically, when the vehicle occupant is in an angry emotional state, the virtual object sequence management unit 210 generates a thunderbolt (i.e., virtual object 2) while a devil including horns (i.e., virtual object 1) moves and the background It is possible to determine that rain (i.e., virtual object 3) is falling as an image sequence about the virtual object.

For another example, the virtual object sequence management unit 210 detects that a certain virtual object (e.g., text, image, etc.) set or authored by at least one vehicle occupant moves in conjunction with the driving speed of the vehicle. It can be determined as an image sequence about a virtual object. Meanwhile, the virtual object sequence management unit 210 may further provide a user interface for setting or authoring the above virtual object to at least one vehicle occupant.

More specifically, when a rabbit image is specified as a virtual object by a vehicle occupant, the virtual object sequence management unit 210 displays a rabbit running along the vehicle according to the driving speed of the vehicle as an image sequence regarding the virtual object. You can decide.

Additionally, the virtual object sequence management unit 210 may generate an image sequence related to the virtual object for each vehicle occupant or each group of vehicle occupants (for example, each group of vehicle occupants may be comprised of at least one occupant). can be decided.

For example, the virtual object sequence management unit 210 specifies the above situation information for each vehicle occupant or each group consisting of at least two vehicle occupants, and refers to each situation information to create an image related to the virtual object. Each sequence can be determined.

In addition, the virtual object sequence management unit 210 is configured for each seat or group of seats (for example, each group of seats may consist of at least one seat) in the vehicle on which the vehicle occupants are seated. An image sequence regarding a virtual object may also be determined.

For example, when the seats in the vehicle are configured in an arrangement of 2 rows and 2 rows (i.e., 4-seater seats), the virtual object sequence management unit 210 may configure the 1st row, 1st row, and 2nd row sharing the left side of the vehicle among the seats. An image sequence related to the virtual object corresponding to the first row and an image sequence related to the virtual object corresponding to the first row, two columns and the second row, two columns sharing the right side of the vehicle can be determined, respectively.

In addition, the virtual object sequence management unit 210 refers to at least one of the viewpoint of at least one vehicle occupant, the viewpoint according to the seat position of the vehicle, and the viewpoint according to the position of at least one display in the vehicle to create the virtual object. You can determine the video sequence related to

For example, when the seats in the vehicle are configured in an arrangement of 2 rows and 2 rows (i.e., 4-seater seats), the virtual object sequence management unit 210 may configure the 1st row, 1st row, and 2nd row sharing the left side of the vehicle among the seats. For row 1, the perspective of the virtual object even if it is the same virtual object, depending on the viewpoint of the vehicle occupant in row 1, column 1, the viewpoint of the vehicle occupant in row 2, column 1, the viewpoint according to the seat position of the vehicle, or the viewpoint according to the position of the display in the vehicle. The image sequence for the virtual object can be determined by specifying the size, shape, and direction of the virtual object differently, taking into account the direction, etc.

Additionally, the virtual object sequence management unit 210 may determine an image sequence for the virtual object by further referring to information about the real-world object specified from the external image of the vehicle.

For example, the virtual object sequence management unit 210 specifies at least one real-world object existing around the vehicle from an external image of the vehicle, and uses a virtual object associated with the specified at least one real-world object to The image sequence regarding the virtual object can be determined.

More specifically, when the virtual object sequence management unit 210 specifies that the real-world object existing around the vehicle is a sparrow based on the external image of the vehicle, the rice paddy or field image associated with the sparrow is displayed while the vehicle is running. The image sequence regarding the above virtual object can be determined by spreading widely around the vehicle in accordance with the speed (i.e., the above driving information). In addition, when the virtual object sequence management unit 210 specifies that a real-world object existing around the vehicle is a store based on the external image of the vehicle, information related to the store, such as the store's operating hours, items for sale, etc. The image sequence regarding the above virtual object can be determined by displaying the information box (i.e., virtual object) contained therein for a predetermined period of time.

Meanwhile, the virtual object sequence management unit 210 may determine the image sequence for the virtual object by further referring to the personal information of the vehicle occupants. The personal information of the vehicle occupant according to an embodiment of the present invention may include information about the vehicle occupant's age, gender, race, region, preferred virtual object, preferred virtual object sequence, interests, etc.

For example, the virtual object sequence management unit 210 specifies the identity of the vehicle occupant based on the vehicle occupant's face in the internal image of the vehicle, and according to the specified identity, the virtual object preferred by the vehicle occupant and the virtual object A video sequence regarding can be determined. That is, the virtual object or an image sequence related to the virtual object may be individually customized and determined based on the personal information of the vehicle occupants.

Next, the content management unit 220 according to an embodiment of the present invention creates an image related to a virtual object determined by the virtual object sequence management unit 210 based on the external real world of the vehicle projected through at least one display of the vehicle. By matching the sequence, a function of determining content to be provided to at least one vehicle occupant can be performed.

For example, the content management unit 220 may provide properties (e.g., type of object (e.g., person, animal, plant, etc.)) about an object in the external real world of the vehicle projected through at least one display of the vehicle, Augmented Reality (AR) content generated by matching the image sequence regarding the above-determined virtual object according to its properties with reference to the location of the object, the shape of the object, the shape of the object, etc.) is provided to at least one person. It can be determined as content to be provided to vehicle occupants.

More specifically, the content management unit 220 determines the coordinates of the external real world of the vehicle (specifically, the coordinates of at least one object in the external real world) projected through the at least one display above based on SLAM (Simultaneous Localization And Mapping) technology. ) can be tracked and 3D mapping (3D mapping) coordinates can be extracted, and the virtual object moving according to the image sequence regarding the above virtual object can be moved by referring to the information about the tracked coordinates and the extracted 3D mapping coordinates. Augmented reality based on the external real world of the vehicle by determining coordinates to be displayed on at least one display, and displaying the virtual object on the coordinates to be displayed and causing it to move according to an image sequence related thereto. You can create content.

However, the type of content according to an embodiment of the present invention is not necessarily limited to the augmented reality content, and within the scope of achieving the purpose of the present invention, virtual reality (VR) content, mixed reality ( Please note that it can be changed at any time due to Mixed Reality (MR) content, etc.

Additionally, the content manager 220 may provide corresponding content based on a display associated with at least one vehicle occupant among a plurality of displays in the vehicle.

For example, the content management unit 220 creates an image sequence regarding a virtual object corresponding to the vehicle occupant based on the external real world of the vehicle based on the display corresponding to the gaze of each vehicle occupant among the plurality of displays in the vehicle. By matching, it is possible to determine the content to be provided to the occupants of the vehicle.

Additionally, the content management unit 240 may perform the function of storing or sharing the content determined above.

For example, the content management unit 240 causes the content determined above to be stored in a device associated with the vehicle occupant (e.g., the vehicle occupant's smartphone, cloud, etc.), or a social network service associated with the vehicle occupant ( It can be shared (e.g., real-time sharing) with other vehicle occupants through SNS, cloud, web server, etc.

For another example, the content management unit 220 may allow the content to be shared based on the geographical location associated with the content determined above.

More specifically, the content manager 220 may allow the first content provided from the first vehicle to be shared with vehicle occupants of other vehicles based on the GPS coordinate location associated with the first content. For example, if the location associated with the above GPS coordinate location is specified or selected by the vehicle occupant of the second vehicle (i.e., the vehicle occupant of another vehicle), the content management unit 220 may store the first content above. It can be shared with the second vehicle (or the vehicle occupants of the second vehicle).

Meanwhile, when additional content creation (e.g., addition, deletion, change, etc.) is made to the first content shared by the vehicle occupants of the second vehicle, the content management unit 220 creates the first content. Based on the GPS coordinates associated with the additional authored content or the first content reflecting the additional authored content, it can be further shared with vehicle occupants of other vehicles.

Additionally, the content management unit 220 may refer to the sharing restriction conditions imposed on the above shared content and allow the content to be shared with vehicles or vehicle occupants that satisfy the restriction conditions.

For example, when a sharing restriction condition that allows content to be shared only with a specific vehicle (or a specific vehicle occupant) is given, the content management unit 220 may only share content with a specific vehicle (or a specific vehicle occupant) that satisfies the restriction condition. Content can be shared.

Next, the communication unit 230 according to an embodiment of the present invention may perform a function that allows data to be transmitted and received from/to the virtual object sequence management unit 210 and the content management unit 220.

Lastly, the control unit 240 according to an embodiment of the present invention may perform a function of controlling the flow of data between the virtual object sequence management unit 210, the content management unit 220, and the communication unit 230. That is, the control unit 240 according to the present invention controls the data flow to/from the outside of the content provision system 200 or the data flow between each component of the content provision system 200, thereby controlling the virtual object sequence management unit 210. , the content management unit 220 and the communication unit 230 can each be controlled to perform their own functions.

Hereinafter, an exemplary situation in which content is provided according to an embodiment of the present invention will be looked at with reference to FIGS. 3 to 8.

First Example

It can be assumed that the content provision system 200 according to an embodiment of the present invention is included in the electronic control system (ECU) of the vehicle.

Referring to FIG. 3, first, the gaze of the vehicle occupant 510 (or the vehicle occupant 510 who performed a predetermined input action on the display 300a while the vehicle was running) identified from the internal image of the vehicle while the vehicle is running. , by analyzing facial expressions, movements, and sounds, when it is determined that the vehicle occupant 510 is quietly immersed in thought while looking outside the vehicle, and a wide reed field 320 is spread around the vehicle from the exterior image of the vehicle, A plurality of wild geese 310 associated with the situation in which the passenger 510 is lost in thought and the reed field 320, which is an object in the external real world, are determined as virtual objects, and the plurality of wild geese 310 are adjusted to the driving speed of the vehicle. An image sequence for a virtual object can be determined by flying along the side of the vehicle.

Next, a sequence in which a plurality of geese 310 fly along the side of the vehicle according to the driving speed of the vehicle is matched against the background of the reed field 320, which is the external real world of the vehicle projected through the transparent display 300a of the vehicle. The augmented reality content generated by doing so may be determined as content to be provided to the occupants of the vehicle.

Next, the above-determined content may be displayed on the transparent display 300a of the vehicle and provided to the vehicle occupants 510.

Meanwhile, according to an embodiment of the present invention, the first vehicle occupant 510 is located in the first row and second row (i.e., passenger seat) of the vehicle (see FIG. 3), and the first vehicle occupant 510 is located in the second row and second row of the vehicle (i.e., the rear seat of the passenger seat). When the second vehicle occupant 520 is located (see FIG. 4), each viewpoint of the vehicle occupant (i.e., each viewpoint of the first vehicle occupant 510 and the second vehicle occupant 520) or the display An image sequence related to the above virtual object (for example, an image sequence related to the virtual object considering the shape and size of the plurality of wild geese 310) may be determined depending on the viewpoint according to the location within the vehicle.

For example, based on the right side (i.e., second row) of the vehicle, the viewpoint (or line of sight) of the first vehicle occupant 510 is located further to the left than the viewpoint (or line of sight) of the second vehicle occupant 520. (Or, based on the right side of the vehicle (i.e., second row), the transparent display 300a associated with the first vehicle occupant 510 is located further to the left than the transparent display 300b associated with the second vehicle occupant 520. Since), an image sequence (or In the case where the content) is determined, for the second vehicle occupant 510, a plurality of wild geese 310 (specifically, a side and rear shape that is smaller due to the perspective effect than the upper side shape of the plurality of wild geese 310) The image sequence (or content) regarding the virtual object may be determined to be flying from the left compared to the first vehicle occupant 510. In addition, an image sequence regarding the above virtual object determined for the first vehicle occupant 510 is displayed against the background of the reed field 320 projected through the transparent display 300a associated with the first vehicle occupant 510. Augmented reality content generated by matching is provided to the first vehicle occupants 510, and the second vehicle occupants 520 are presented with the reed field 320 in the background projected through the transparent display 300b associated with the second vehicle occupants 520. Augmented reality content generated by matching the image sequence regarding the above virtual object determined for the occupant 520 may be provided to the second vehicle occupant 520 .

Second embodiment

It can be assumed that the content provision system 200 according to an embodiment of the present invention is included in the electronic control system (ECU) of the vehicle.

Referring to FIG. 5, first, the line of sight of a vehicle occupant 510 (or a vehicle occupant 510 who performed a predetermined input action on the display 300a while driving the vehicle) identified from the internal image of the vehicle while driving the vehicle. , By analyzing facial expressions, movements, and sounds, a situation in which the vehicle occupant 510 is looking outside the vehicle and worrying that the road may become slippery due to heavy snow can be specified, and a slip warning authored by the vehicle occupant 510 If the display image 550 is specified as a virtual object, an image sequence related to the virtual object may be determined as the specified skid warning display image 550 blinks according to the driving speed of the vehicle.

Next, augmented reality is created by matching an image sequence in which a slip warning display image 550 flashes according to the vehicle's driving speed based on the snowy landscape outside the vehicle projected through the vehicle's transparent display 300a. The content may be determined as content to be provided to the vehicle occupants 510.

Meanwhile, the above-determined content may be shared with other vehicles (or vehicle occupants of other vehicles).

Referring to FIG. 6, the above-determined content may be shared with vehicle occupants of other vehicles based on the geographic location associated therewith (e.g., GPS coordinates of the road in question), and may be shared by vehicle occupants of other vehicles. Once the location 610 associated with that geographic location is specified or selected, the determined content may be shared to other vehicles.

For example, in the content shared as above, the text “Be careful because it snows a lot and is slippery” is included as a virtual object, and based on the geographical location associated with the shared content, the content is displayed on other vehicles. It can be shared with vehicle occupants. Then, if a vehicle occupant of another vehicle sharing that content additionally authored the content as a virtual object and re-shared the text “I’m good now,” that additional authoring would be based on the above geographic location. A virtual object (i.e., text) (or content that reflects the additional authored virtual object above for the shared content above) may be shared.

Third embodiment

Referring to FIG. 7, it can be assumed that the content providing system 200 according to an embodiment of the present invention is included in an external server (not shown). Specifically, it is connected to an electronic control system (ECU) (not shown) that controls at least one display (not shown) in the vehicle via wireless communication (e.g., WiFi, Bluetooth, etc.). There may be a situation where content is provided to vehicle occupants through the above external server.

Referring to FIG. 7, first, the gaze, facial expressions, movements, and By analyzing the sound, the first vehicle occupant 710 and the second vehicle occupant 720 are fighting with each other, and the second vehicle occupant 720 is angry, and the third vehicle occupant 730 is looking out the window. When a smiling situation is specified, two devil images corresponding to the first vehicle occupant 710 and the second vehicle occupant 720 are specified as virtual objects, and the two devils fighting each other are images related to the virtual objects. It can be determined as a sequence, and in response to the third vehicle occupant 730, an angel is determined as a virtual object, and the angel plays a specific song while following the vehicle according to the vehicle's driving speed as a video sequence for the virtual object. can be decided.

Next, the virtual object determined above is displayed against the background of the vehicle's driving road projected through the transparent display 300a of the vehicle corresponding to the gaze 701 and 702 of the first vehicle occupant 710 and the second vehicle occupant. By matching the video sequence about (i.e., the video sequence about two demons fighting each other), content to be provided to the first vehicle occupant 710 and the second vehicle occupant can be determined, and the content to be provided to the first vehicle occupant 710 and the second vehicle occupant 730. An image sequence regarding the virtual object determined above against the background of the vehicle's driving road projected through the vehicle's transparent display 300b corresponding to the line of sight 703 (i.e., the angel follows the vehicle according to the vehicle's driving speed and makes a specific By matching video sequences (about playing a song), content to be provided to the third vehicle occupant 730 can be determined.

Embodiment 4

Referring to FIG. 8, it can be assumed that content is provided on a plurality of displays 300d, 300e, and 300f of a vehicle.

For example, a duck (i.e., virtual object) 810 swimming is determined as an image sequence for the virtual object, and augmentation is generated by registering the image sequence for the virtual object against the background of the real world outside the vehicle. When real content is determined to be content to be provided to vehicle occupants (not shown), each of the plurality of displays (300d, 300e, 300f) displays duck ( 810), at least one of the size, shape, and form may vary.

Example 5

According to an embodiment of the present invention, another situation in which content is provided on a plurality of displays of a vehicle can be assumed.

For example, referring to Figure 8, a duck (i.e., virtual object) 810 swimming is determined as an image sequence for the virtual object, and an image sequence for the virtual object is displayed against the background of the real world outside the vehicle. When the augmented reality content created by matching is determined as the content to be provided to vehicle occupants, one content will be synchronized and provided on all displays on the front, left, and right sides of the vehicle based on the point in time that matches the position of each display within the vehicle. (For example, the duck 810 appears on the display on the front and left sides of the vehicle, and the duck 810 does not appear on the display on the right side and rear of the vehicle), and this allows the vehicle occupants to enter a specific space as if they were entering a specific space. It is possible to provide a similar user experience.

The embodiments according to the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., singly or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and usable by those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. A hardware device can be converted into one or more software modules to perform processing according to the invention and vice versa.

In the above, the present invention has been described in terms of specific details, such as specific components, and limited embodiments and drawings, but this is only provided to facilitate a more general understanding of the present invention, and the present invention is not limited to the above embodiments. Anyone with ordinary knowledge in the technical field to which the invention pertains can make various modifications and changes from this description.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the patent claims described below as well as all scopes equivalent to or equivalently changed from the scope of the claims are within the scope of the spirit of the present invention. It will be said to belong to

100: communication network
200: Content provision system
210: Virtual object sequence management unit
220: Content management department
230: Department of Communications
240: control unit
300: display

Claims (23)

As a method of providing content,
determining an image sequence regarding the virtual object with reference to situational information of at least one vehicle occupant, and
determining content to be provided to the at least one vehicle occupant by matching an image sequence regarding the virtual object based on a real world outside the vehicle projected through at least one display of the vehicle;
In the video sequence determination step, an emotional state or surrounding atmosphere inferred from the at least one vehicle occupant is specified as situation information of the at least one vehicle occupant, and a virtual object associated with the situation information of the at least one vehicle occupant is provided. Determining as an image sequence regarding the virtual object to perform an action corresponding to the emotional state or surrounding atmosphere.
method. According to paragraph 1,
In the step of determining the video sequence, at least one of the at least one vehicle occupant and the situation information of the at least one vehicle occupant is specified from the internal image of the vehicle.
method. According to paragraph 1,
In the image sequence determination step, determining an image sequence for the virtual object by further referring to at least one of information about a real-world object specified from an external image of the vehicle and driving information of the vehicle.
method. According to paragraph 1,
In the image sequence determining step, determining an image sequence related to the virtual object for each vehicle occupant or for each group of vehicle occupants.
method. According to paragraph 1,
In the step of determining the image sequence, determining an image sequence regarding the virtual object for each seat or each group of seats on which the vehicle occupant is seated in the vehicle.
method. According to paragraph 1,
In the step of determining the image sequence, an image related to the virtual object is referred to at least one of the viewpoint of the vehicle occupant, the viewpoint according to the seat position of the vehicle, and the viewpoint according to the position of the at least one display in the vehicle. determining the sequence
method. According to paragraph 1,
In the video sequence determination step, determining the video sequence for the virtual object further refers to personal information of the vehicle occupants.
method. According to paragraph 1,
In the content determination step, matching an image sequence about the virtual object with reference to properties about objects in the real world outside the vehicle projected through the at least one display.
method. According to paragraph 1,
The content determining step includes providing the determined content based on a display of the at least one display that is associated with the at least one vehicle occupant.
method. According to clause 9,
Providing the determined content based on a display corresponding to the gaze of each of the at least one vehicle occupant among the at least one display.
method. According to paragraph 1,
The content determining step includes synchronizing and providing the determined content on the at least one display based on a time point corresponding to the location of each of the at least one display within the vehicle.
method. A non-transitory computer-readable recording medium recording a computer program for executing the method according to claim 1. As a system that provides content,
A virtual object sequence management unit that determines an image sequence for a virtual object by referring to situation information of at least one vehicle occupant, and
A content management unit that determines content to be provided to the at least one vehicle occupant by matching an image sequence regarding the virtual object based on the external real world of the vehicle projected through at least one display of the vehicle,
The virtual object sequence management unit specifies an emotional state or surrounding atmosphere inferred from the at least one vehicle occupant as situation information of the at least one vehicle occupant, and the virtual object associated with the situation information of the at least one vehicle occupant is Determining as an image sequence regarding the virtual object to perform an action corresponding to the emotional state or surrounding atmosphere.
system. According to clause 13,
At least one of the at least one vehicle occupant and the situation information of the at least one vehicle occupant is specified from an internal image of the vehicle.
system. According to clause 13,
The virtual object sequence management unit determines an image sequence for the virtual object by further referring to at least one of information about a real-world object specified from an external image of the vehicle and driving information of the vehicle.
system. According to clause 13,
The virtual object sequence management unit determines an image sequence related to the virtual object for each vehicle occupant or for each group of vehicle occupants.
system. According to clause 13,
The virtual object sequence management unit determines an image sequence related to the virtual object for each seat on which the vehicle occupant is sitting in the vehicle or for each group of seats.
system. According to clause 13,
The virtual object sequence management unit determines an image sequence related to the virtual object with reference to at least one of the viewpoint of the vehicle occupant, the viewpoint according to the seat position of the vehicle, and the viewpoint according to the position of the at least one display in the vehicle.
system. According to clause 13,
The virtual object sequence management unit determines an image sequence for the virtual object by further referring to personal information of the vehicle occupants.
system. According to clause 13,
The content management unit matches an image sequence related to the virtual object with reference to properties related to objects in the real world outside the vehicle projected through the at least one display.
system. According to clause 13,
The content management unit provides the determined content based on a display associated with the at least one vehicle occupant among the at least one display.
system. According to clause 21,
The content management unit provides the determined content based on a display corresponding to the gaze of each of the at least one vehicle occupant among the at least one display.
system. According to clause 13,
The content management unit synchronizes and provides the determined content on the at least one display based on a time point corresponding to the location of each of the at least one display within the vehicle.
system.