KR20200137992A - Method, system, and non-transitory computer-readable recording medium for providing contents - Google Patents

Abstract

According to one aspect of the present invention, provided is a method for providing content, which comprises the steps of: determining an image sequence for a virtual object by referring to at least one of situation information of at least one vehicle occupant and driving information of a vehicle; and determining content to be provided to the at least one vehicle occupant by matching the image sequence for the virtual object based on the external real world of the vehicle projected through at least one display of the vehicle.

Description

Method of providing content, system, and non-transitory computer-readable recording medium {METHOD, SYSTEM, AND NON-TRANSITORY COMPUTER-READABLE RECORDING MEDIUM FOR PROVIDING CONTENTS}

The present invention relates to a method, system and non-transitory computer-readable recording medium for providing content.

In recent years, with the development of communication and artificial intelligence (AI) technologies, the autonomous vehicle driving service has been commercialized, and the autonomous vehicle market is revitalizing.

In such an autonomous vehicle, a number of cameras are installed in order to detect nearby obstacles or to determine the state of the vehicle, and through this, the inside or outside of the vehicle is continuously photographed.

However, the image captured as above is only deleted or overwritten with a new image immediately after driving the vehicle, and is not properly utilized for other purposes.

In addition, in recent years, as autonomous vehicles are commercialized and distributed, the amount of time for vehicle occupants to directly drive decreases, so there is an urgent need to develop content that can alleviate the boredom of vehicle occupants and inspire interest while driving. It is a situation.

Accordingly, the present inventor(s) proposes a new and advanced technology capable of interactively providing content to a vehicle occupant while driving a vehicle.

Korean Patent Application Publication No. 2018-0014910

An object of the present invention is to solve all the problems of the prior art described above.

In addition, an object of the present invention is to eliminate the boredom and boredom of vehicle occupants that may occur during vehicle driving, and to increase the utilization of vehicle images.

In addition, an object of the present invention is to enable content to be synchronized and provided on at least two displays in a vehicle.

It is another object of the present invention to provide content for each vehicle occupant or for each predetermined group of vehicle occupants.

In addition, an object of the present invention is to support a vehicle occupant to easily create and share content within a vehicle.

A typical configuration of the present invention for achieving the above object is as follows.

According to an aspect of the present invention, a method of providing content, comprising: determining an image sequence related to a virtual object by referring to at least one of situation information of at least one vehicle occupant and driving information of the vehicle, and A method is provided comprising the step of determining content to be provided to the at least one vehicle occupant by matching an image sequence relating to the virtual object based on the external real world of the vehicle projected through at least one display.

According to another aspect of the present invention, a system for providing content, comprising: a virtual object sequence management unit that determines an image sequence for a virtual object by referring to at least one of situation information of at least one vehicle occupant and driving information of the vehicle; and A system comprising a content management unit that determines content to be provided to the at least one vehicle occupant by matching the image sequence of the virtual object based on the external real world of the vehicle projected through at least one display of the vehicle. Is provided.

In addition to this, another method for implementing the present invention, another system, and a non-transitory computer-readable recording medium for recording a computer program for executing the method are further provided.

According to the present invention, it is possible to eliminate the boredom and boredom of a vehicle occupant that may occur while driving a vehicle, and increase the utilization of vehicle images.

Further, according to the present invention, it is possible to provide content by synchronizing at least two displays in the vehicle.

Further, according to the present invention, it is possible to provide content for each vehicle occupant or each predetermined group of vehicle occupants.

In addition, according to the present invention, it is possible to support a vehicle occupant to easily author and share content within the vehicle.

1 is a diagram showing a schematic configuration of an entire system for providing content according to an embodiment of the present invention.
2 is a diagram illustrating in detail the internal configuration of a content providing system according to an embodiment of the present invention.
3 to 8 are diagrams illustrating a situation in which content is provided according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION The detailed description of the present invention to be described later refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced. These embodiments are described in detail sufficient to enable those skilled in the art to practice the present invention. It is to be understood that the various embodiments of the present invention are different from each other, but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be changed from one embodiment to another and implemented without departing from the spirit and scope of the present invention. In addition, it should be understood that the positions or arrangements of individual elements in each embodiment may be changed without departing from the spirit and scope of the present invention. Therefore, the detailed description to be described below is not made in a limiting sense, and the scope of the present invention should be taken as encompassing the scope claimed by the claims of the claims and all scopes equivalent thereto. Like reference numerals in the drawings indicate the same or similar elements over various aspects.

In the present specification, content is a concept collectively referring to digital information or individual information elements composed of characters, codes, voices, sounds, images, videos, and the like. Such content may include, for example, data such as text, image, video, audio, link (eg, web link), or a combination of at least two of these data.

Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable those skilled in the art to easily implement the present invention.

Composition of the whole system

1 is a diagram showing a schematic configuration of an entire system for providing content according to an embodiment of the present invention.

As shown in FIG. 1, the entire system according to an embodiment of the present invention may include a communication network 100, a content providing system 200, and at least one display 300.

First, the communication network 100 according to an embodiment of the present invention may be configured regardless of a communication mode such as wired communication or wireless communication, and may be configured as a local area network (LAN) or a metropolitan area network (MAN). ), and a wide area network (WAN). Preferably, the communication network 100 referred to in this specification may be a known Internet or World Wide Web (WWW). However, the communication network 100 is not necessarily limited thereto, and may include a known wired/wireless data communication network, a known telephone network, or a known wired/wireless television communication network in at least part thereof.

For example, the communication network 100 is a wireless data communication network, such as radio frequency (RF) communication, Wi-Fi communication, cellular (LTE, etc.) communication, Bluetooth communication (more specifically, low power Bluetooth (BLE; Bluetooth Low Energy) communication), infrared communication, ultrasonic communication, and the like may be implemented in at least part of a conventional communication method.

Next, the content providing system 200 according to an embodiment of the present invention may be a digital device equipped with a memory means and equipped with a microprocessor to have computing power. For example, the content providing system 200 may be a server system or an electronic control system (ECU) of a vehicle.

The content providing system 200 according to an embodiment of the present invention determines an image sequence for a virtual object by referring to at least one of situation information of at least one vehicle occupant and driving information of the vehicle, and displays at least one display of the vehicle. By matching the image sequence of the above virtual object based on the external real world of the vehicle projected through 300, a function of determining content to be provided to at least one vehicle occupant may be performed.

Here, the virtual object according to an embodiment of the present invention is a concept that collectively refers to all objects that can be implemented on at least one display in a vehicle, for example, letters, numbers, symbols, symbols, figures, photos, It may be composed of images, sounds, moving pictures, or a combination thereof, and may be a concept including even attributes (eg, brightness, brightness, saturation) that may be assigned thereto.

The configuration and function of the content providing system 200 according to the present invention will be described in more detail below. Meanwhile, although the content providing system 200 has been described as above, this description is exemplary, and at least a part of the functions or components required for the content providing system 200 is at least one display 300 to be described later as needed. ) Or an external system (not shown), or may be included in at least one display 300 or an external system above.

Next, at least one display 300 according to an embodiment of the present invention may perform a function of displaying content determined by the content providing system 200.

For example, the at least one display 300 according to an embodiment of the present invention includes a transparent (eg, 5 to 80% transparency) or translucent display on which at least a part of the external real world of the vehicle can be projected. It may be, and more specifically, may be composed of a transparent or translucent Thin Film Electroluminescent Display (TFELD), OLED, LCD, LED, and the like.

For another example, at least one display 300 according to an embodiment of the present invention projects an external real world of the vehicle by displaying an image (for example, a real-time photographed image) of the vehicle on an opaque display. It can also perform the function of doing.

In addition, a plurality of displays 300 as described above may be arranged in the vehicle, for example, arranged in front, rear, left, and right sides (ie, at least four displays are arranged) based on the vehicle (or at least one vehicle occupant). Can be. In addition, the display 300 may be additionally mounted around a conventional vehicle window or may be mounted in place of the window.

On the other hand, at least one display 300 according to an embodiment of the present invention may be a digital device equipped with a memory means and equipped with a microprocessor to have computing power, and a content determination function (or content provision) according to the present invention. Function) may be included. Such an application may be downloaded from the content providing system 200 or an external application distribution server (not shown).

Composition of content provision system

Hereinafter, the internal configuration of the content providing system 200 that performs an important function for the implementation of the present invention and functions of each component will be described.

2 is a diagram illustrating in detail the internal configuration of a content providing system 200 according to an embodiment of the present invention.

As shown in FIG. 2, a virtual object sequence management unit 210, a content management unit 220, a communication unit 230, and a control unit 240 may be included. According to an embodiment of the present invention, the virtual object sequence management unit 210, the content management unit 220, the communication unit 230, and the control unit 240 may be program modules in which at least some of them communicate with an external system. Such a program module may be included in the content providing system 200 in the form of an operating system, an application program module, or other program module, and may be physically stored in various known storage devices. Further, such a program module may be stored in a remote storage device capable of communicating with the content providing system 200. Meanwhile, such a program module includes routines, subroutines, programs, objects, components, data structures, etc. that perform specific tasks or execute specific abstract data types according to the present invention, but is not limited thereto.

First, the virtual object sequence management unit 210 according to an embodiment of the present invention refers to at least one of situation information of at least one vehicle occupant and driving information of the vehicle to determine an image sequence for a virtual object. I can. The situation information of a vehicle occupant according to an embodiment of the present invention may include information indicating a situation inferred from the vehicle occupant (for example, a situation inferred from the vehicle occupant's gaze, facial expression, behavior, motion, sound, etc.) have. For example, search cases matching the same or similar information as information on the vehicle occupant's gaze, facial expression, behavior, motion, etc., on the case database, or machine learning about the vehicle occupant's gaze, facial expression, behavior, motion, sound, etc. A corresponding case may be specified based on a result of learning through machine learning) or deep learning, and information indicating the above situation may be specified based on the specified case. In addition, the driving information of the vehicle according to an embodiment of the present invention may include information about the vehicle's departure point, destination, current position, driving speed, driving state (eg, start, stop, etc.). On the other hand, the virtual object sequence management unit 210 specifies at least one vehicle occupant that is the object of obtaining context information based on the internal image of the vehicle (for example, specifies a vehicle occupant in which movement such as a gaze or facial expression occurs) Or, or perform a predetermined input action (for example, an action of touching or manipulating the display 300 or a separate external system) in the vehicle or a predetermined motion (in this case, the motion is a predetermined weight sensor, an accelerometer sensor) And the like) may be identified as at least one vehicle occupant who is a target of obtaining situation information. However, it should be noted that the above specific method according to the present invention is not necessarily limited to those listed above, and may be variously changed within the range capable of achieving the object of the present invention.

For example, the virtual object sequence management unit 210 analyzes at least one of the gaze, facial expression, motion, behavior, and sound of at least one vehicle occupant based on the internal image of the vehicle (or the vehicle occupant displays ( 300) by analyzing the behavior of performing a predetermined input) can be classified into the feelings of the vehicle occupant (e.g., joy, anger, sadness, and pleasure) ) Or its surrounding atmosphere as situation information of the vehicle occupant, and causing the virtual object associated with the situation information to perform an action corresponding to the emotion or the corresponding atmosphere can be determined as an image sequence for the corresponding virtual object. .

More specifically, when the vehicle occupant is in an angry emotional state, the virtual object sequence management unit 210 generates a thunderbolt (that is, virtual object 2) while a devil including a horn (ie, virtual object 1) moves, and the background It may be determined that rain (ie, virtual object 3) falls as an image sequence for the virtual object.

For another example, the virtual object sequence management unit 210 corresponds to a movement of a predetermined virtual object (for example, text, image, etc.) set or created by at least one vehicle occupant in association with the driving speed of the vehicle. It can be determined as an image sequence for a virtual object. Meanwhile, the virtual object sequence management unit 210 may further provide a user interface for setting or authoring the above virtual object to at least one vehicle occupant.

More specifically, when a rabbit image is specified as a virtual object by a vehicle occupant, the virtual object sequence management unit 210 determines that the rabbit jumps along the vehicle according to the driving speed of the vehicle as an image sequence related to the virtual object. You can decide.

In addition, the virtual object sequence management unit 210 is an image sequence for a virtual object for each vehicle occupant or a group of vehicle occupants (for example, each group of vehicle occupants may consist of at least one occupant). Can be determined.

For example, the virtual object sequence management unit 210 specifies the above situation information for each vehicle occupant or for each group consisting of at least two vehicle occupants, and refers to each situation information to an image of the virtual object. Each sequence can be determined.

In addition, the virtual object sequence management unit 210 is configured for each seat in the vehicle in which the vehicle occupant is seated or a group of seats (for example, each group of seats may consist of at least one seat). It is also possible to determine an image sequence for a virtual object.

For example, the virtual object sequence management unit 210, when the seats in the vehicle are configured in an arrangement of 2 rows and 2 rows (ie, 4 seats), among the seats, the 1 row 1 row and 2 rows sharing the left side of the vehicle An image sequence for a virtual object corresponding to column 1 and an image sequence for a virtual object corresponding to column 1 and column 2 and column 2 and column 2 sharing the right side of the vehicle may be determined, respectively.

In addition, the virtual object sequence management unit 210 refers to at least one of a view point of at least one vehicle occupant, a view point according to the seat position of the vehicle, and a view point according to the in-vehicle position of at least one display. It is possible to determine an image sequence related to.

For example, the virtual object sequence management unit 210, when the seats in the vehicle are configured in an arrangement of 2 rows and 2 rows (ie, 4 seats), among the seats, the 1 row 1 row and 2 rows sharing the left side of the vehicle For column 1, depending on the viewpoint of the vehicle occupant in the first row and the first column, the viewpoint of the vehicle occupant in the second row and the first column, the viewpoint according to the seat position of the vehicle, or the viewpoint according to the in-vehicle position of the display, even the same virtual object By making the size, shape, and direction of the virtual object differently specified in consideration of the direction, etc., an image sequence for the virtual object may be determined.

In addition, the virtual object sequence management unit 210 may determine an image sequence for the virtual object by further referring to information on a real-world object specified from an external image of the vehicle.

For example, the virtual object sequence management unit 210 specifies at least one real-world object existing around the vehicle from an external image of the vehicle, and uses a virtual object associated with the specified at least one real-world object. The image sequence for the virtual object of may be determined.

More specifically, when the virtual object sequence management unit 210 specifies that the real-world object existing around the vehicle is a sparrow based on the external image of the vehicle, the image of the paddy field or field associated with the sparrow is driven by the vehicle. According to the speed (that is, the above driving information), the image sequence of the above virtual object can be determined by spreading widely around the vehicle. In addition, the virtual object sequence management unit 210 determines that the real-world object existing around the vehicle is a store based on the external image of the vehicle, and the information related to the store such as the operating time of the store and sales items is stored. As the contained information box (ie, the virtual object) is displayed for a predetermined time, an image sequence of the above virtual object may be determined.

Meanwhile, the virtual object sequence management unit 210 may determine an image sequence for the virtual object by further referring to personal information of the vehicle occupant. The personal information of a vehicle occupant according to an embodiment of the present invention may include information about the age, sex, race, region, preferred virtual object, preferred virtual object sequence, interest, and the like of the vehicle occupant.

For example, the virtual object sequence management unit 210 specifies the identity of the vehicle occupant based on the face of the vehicle occupant in the internal image of the vehicle, and the virtual object and the virtual object preferred by the vehicle occupant according to the specified identity. An image sequence for can be determined. That is, a virtual object or an image sequence related to the virtual object may be individually customized and determined based on personal information of the vehicle occupant.

Next, the content management unit 220 according to an embodiment of the present invention is an image of a virtual object determined by the virtual object sequence management unit 210 based on the external real world of the vehicle projected through at least one display of the vehicle. By matching the sequences, it is possible to perform a function of determining content to be provided to at least one vehicle occupant.

For example, the content management unit 220 may include properties (eg, types of objects (eg, people, animals, plants, etc.)) of the external real-world object of the vehicle projected through at least one display of the vehicle, By referring to the location of the object, the shape of the object, the shape of the object, etc.), at least one augmented reality content (AR) generated by matching the image sequence of the virtual object determined above according to its properties It can be determined as content to be provided to vehicle occupants.

More specifically, the content management unit 220 is based on the SLAM (Simultaneous Localization And Mapping) technology, the coordinates of the external real world of the vehicle projected through the at least one display above (specifically, the coordinates of at least one object of the external real world) ) Can be tracked and 3D Mapping coordinates can be extracted, and a virtual object moving according to the image sequence of the above virtual object can be moved by referring to the information about the tracked coordinates and the extracted 3D mapping coordinates. Augmented reality based on the real world outside of the vehicle above by determining the coordinates to be displayed on at least one display of and displaying the virtual object on the coordinates to be displayed above and moving according to the image sequence You can create content.

However, the type of content according to an embodiment of the present invention is not necessarily limited to the augmented reality content, and within a range that can achieve the object of the present invention, virtual reality (VR) content, mixed reality ( Mixed Reality (MR) content, etc.

In addition, the content management unit 220 may provide the corresponding content based on a display associated with at least one vehicle occupant among a plurality of displays in the vehicle.

For example, the content management unit 220 may generate an image sequence for a virtual object corresponding to the vehicle occupant based on the external real world of the vehicle based on the display corresponding to the respective gaze of the vehicle occupant among a plurality of displays in the vehicle. By matching, content to be provided to the vehicle occupant can be determined.

In addition, the content management unit 240 may perform a function of storing or sharing the determined content.

For example, the content management unit 240 allows the above-determined content to be stored in a device associated with a vehicle occupant (eg, a vehicle occupant's smartphone, cloud, etc.), or a social network service associated with a vehicle occupant ( SNS), cloud, web server, etc., can be shared with other vehicle occupants (for example, real-time sharing).

For another example, the content management unit 220 may allow the corresponding content to be shared based on a geographic location associated with the determined content.

More specifically, the content management unit 220 may allow the first content provided from the first vehicle to be shared with a vehicle occupant of another vehicle based on a GPS coordinate position associated with the first content. For example, when a location associated with the GPS coordinate location above is specified or selected by a vehicle occupant of a second vehicle (ie, a vehicle occupant of another vehicle), the above first content is It can be shared with a second vehicle (or a vehicle occupant of the second vehicle).

On the other hand, when the content management unit 220 additionally creates content (eg, addition, deletion, change, etc.) for the first content shared by the vehicle occupant of the second vehicle, the first content The additional authored content or the first content reflecting the additional authored content may be further shared with vehicle occupants of other vehicles based on the GPS coordinates associated with the.

In addition, the content management unit 220 may refer to a sharing restriction condition imposed on the above shared content so that the content is shared with a vehicle or vehicle occupant that satisfies the restriction condition.

For example, if the content management unit 220 is given a sharing restriction condition that allows content to be shared only to a specific vehicle (or a specific vehicle occupant), the content management unit 220 applies only to a specific vehicle (or a specific vehicle occupant) that satisfies the restriction condition. Content can be shared.

Next, the communication unit 230 according to an embodiment of the present invention may perform a function of enabling data transmission and reception from/to the virtual object sequence management unit 210 and the content management unit 220.

Finally, the control unit 240 according to an embodiment of the present invention may perform a function of controlling the flow of data between the virtual object sequence management unit 210, the content management unit 220, and the communication unit 230. That is, the control unit 240 according to the present invention controls the data flow from/to the outside of the content providing system 200 or the data flow between each component of the content providing system 200, so that the virtual object sequence management unit 210 , The content management unit 220 and the communication unit 230 may control each to perform a unique function.

Hereinafter, an exemplary situation in which content is provided according to an embodiment of the present invention will be described with reference to FIGS. 3 to 8.

Embodiment 1

It can be assumed that the content providing system 200 according to an embodiment of the present invention is included in the electronic control system (ECU) of a vehicle.

Referring to FIG. 3, first, the gaze of the vehicle occupant 510 specified from the internal image of the vehicle while driving the vehicle (or the vehicle occupant 510 performing a predetermined input action targeting the display 300a while driving the vehicle) , When the vehicle occupant 510 is quietly locked in thought while looking at the outside of the vehicle by analyzing facial expressions, motions and sounds, and when it is specified that a wide reed field 320 is spread around the vehicle from the external image of the vehicle, the vehicle A situation in which the occupant 510 is immersed in thought and a plurality of geese 310 associated with the reed field 320, which is an object of the external real world, are determined as virtual objects, and the plurality of geese 310 are matched to the driving speed of the vehicle. An image sequence of the virtual object may be determined by flying along the side of the vehicle.

Then, the sequence of the plurality of geese 310 flying along the side of the vehicle in accordance with the driving speed of the vehicle is matched with the background of the reed field 320, which is the external real world of the vehicle projected through the vehicle's transparent display 300a. Thus, the augmented reality content generated may be determined as the content to be provided to the vehicle occupant.

Then, the above-determined content may be displayed on the vehicle's transparent display 300a and provided to the vehicle occupant 510.

Meanwhile, according to an embodiment of the present invention, the first vehicle occupant 510 is located in the first row and the second row (ie, the passenger seat) of the vehicle (see FIG. 3), and the second row and the second row of the vehicle (ie, the rear seat of the passenger seat) In the case where the second vehicle occupant 520 is located (see FIG. 4), the respective viewpoints of the vehicle occupants (that is, the respective viewpoints of the first vehicle occupant 510 and the second vehicle occupant 520) or the display An image sequence related to the above virtual object (eg, an image sequence related to the virtual object considering the shape and size of the plurality of geese 310) may be determined according to a viewpoint according to a position in the vehicle.

For example, since the viewpoint (or line of sight) of the first vehicle occupant 510 based on the right side of the vehicle (that is, the second row) is more left than the viewpoint (or line of sight) of the second vehicle occupant 520 (Or the transparent display 300a associated with the first vehicle occupant 510 based on the right side of the vehicle (i.e., second row) is present further to the left than the transparent display 300b associated with the second vehicle occupant 520 Therefore, a sequence of images about a virtual object (or a plurality of geese 310 (specifically, a side shape of the plurality of geese 310) is flying from the front of the passenger for the first vehicle occupant 510 (or Content) is determined, for the second vehicle occupant 510, a plurality of geese 310 (specifically, a side-rear shape smaller due to a perspective effect than an upper side shape of the plurality of geese 310) As compared to the first vehicle occupant 510, the image sequence (or content) of the virtual object may be determined as flying from the left. In addition, an image sequence regarding the above virtual object determined for the first vehicle occupant 510 is displayed with the reed field 320 projected through the transparent display 300a associated with the first vehicle occupant 510 as a background. Augmented reality content generated by matching is provided to the first vehicle occupant 510, and the second vehicle with the reed field 320 projected through the transparent display 300b associated with the second vehicle occupant 520 as a background. Augmented reality content generated by matching the image sequence of the above virtual object determined for the occupant 520 may be provided to the second vehicle occupant 520.

Embodiment 2

It can be assumed that the content providing system 200 according to an embodiment of the present invention is included in the electronic control system (ECU) of a vehicle.

Referring to FIG. 5, first, the gaze of the vehicle occupant 510 specified from the internal image of the vehicle while driving the vehicle (or the vehicle occupant 510 performing a predetermined input action targeting the display 300a while driving the vehicle) , By analyzing facial expressions, motions, and sounds, a situation in which the vehicle occupant 510 is concerned that the road becomes slippery due to snowy while looking outside the vehicle can be specified, and a slip warning written by the vehicle occupant 510 When the display image 550 is specified as a virtual object, an image sequence relating to the virtual object may be determined as the specified slip warning display image 550 flashes according to the driving speed of the vehicle.

Next, the augmented reality generated by matching the image sequence in which the slip warning display image 550 flashes according to the driving speed of the vehicle based on the snow scene outside the vehicle projected through the vehicle's transparent display 300a. The content may be determined as content to be provided to the vehicle occupant 510.

Meanwhile, the above-determined content may be shared with another vehicle (or a vehicle occupant of another vehicle).

Referring to FIG. 6, the content determined above may be shared with a vehicle occupant of another vehicle based on a geographic location associated therewith (eg, GPS coordinates of a corresponding road), and by the vehicle occupant of another vehicle. When the location 610 associated with the geographic location is specified or selected, the above-determined content may be shared with other vehicles.

For example, the text "Be careful because it snows a lot and is slippery" is included as a virtual object in the shared content as above, and the content is a different vehicle based on the geographical location associated with the shared content. It can be shared with vehicle occupants. Then, when the vehicle occupant of another vehicle sharing the content additionally authors and re-shares the text "Now it's okay" for the content as a virtual object, the additional authoring based on the above geographic location The created virtual object (ie, text) (or the content in which the additionally authored virtual object is reflected in the above shared content) may be shared.

Embodiment 3

Referring to FIG. 7, it can be assumed that a content providing system 200 according to an embodiment of the present invention is included in an external server (not shown). Specifically, connection with an electronic control system (ECU) (not shown) that controls at least one display (not shown) in the vehicle and wireless communication (for example, WiFi, Bluetooth, etc.) It may be a situation in which content is provided to a vehicle occupant through the external server above.

Referring to FIG. 7, first, the gazes, facial expressions, motions, and eyes of a plurality of vehicle occupants (or a plurality of vehicle occupants who have performed a predetermined input action on the display while driving the vehicle) specified from the internal image of the vehicle while driving the vehicle By analyzing the sound, the first vehicle occupant 710 and the second vehicle occupant 720 quarrel with each other, and the second vehicle occupant 720 is angry, and the third vehicle occupant 730 is looking out the window. When the laughing situation is specified, in response to the first vehicle occupant 710 and the second vehicle occupant 720, two demon images are specified as virtual objects, and the two demons fighting each other is an image of the corresponding virtual object. It may be determined as a sequence, and in response to the third vehicle occupant 730, an angel is determined as a virtual object, and the angel follows the vehicle according to the vehicle driving speed and plays a specific song as an image sequence related to the virtual object. Can be determined.

Next, a virtual object determined on the background of the driving road of the vehicle projected through the transparent display 300a of the vehicle corresponding to the eyes of the first vehicle occupant 710 and the second vehicle occupant 701 and 702 By matching the image sequence related to (that is, the image sequence about two demons fighting each other), the content to be provided to the first vehicle occupant 710 and the second vehicle occupant can be determined, and the third vehicle occupant 730 An image sequence about the virtual object determined above with the background of the vehicle's driving road projected through the vehicle's transparent display 300b corresponding to the line of sight 703 (that is, an angel follows the vehicle according to the vehicle driving speed, The content to be provided to the third vehicle occupant 730 may be determined by matching an image sequence related to playing a song).

Embodiment 4

Referring to FIG. 8, a situation in which content is provided on a plurality of displays 300d, 300e, and 300f of a vehicle can be assumed.

For example, what the duck (i.e., virtual object) 810 is swimming is determined as an image sequence related to the virtual object, and augmentation generated by matching the image sequence related to the virtual object with the real world outside of the vehicle as a background When the actual content is determined as the content to be provided to the vehicle occupant (not shown), in each of the plurality of displays (300d, 300e, 300f), the duck (in accordance with the viewpoints 801, 802, 803) viewed by each vehicle occupant At least one of the size, shape, and shape of the 810) may vary.

Embodiment 5

According to an embodiment of the present invention, another situation in which content is provided on a plurality of displays of a vehicle may be assumed.

For example, referring to FIG. 8, what a duck (that is, a virtual object) 810 is swimming is determined as an image sequence related to a virtual object, and an image sequence related to the virtual object based on the external real world of the vehicle When the augmented reality content generated by matching is determined as the content to be provided to the vehicle occupant, one content will be synchronized and provided on the basis of the time point at which each display corresponds to the in-vehicle position in all displays of the vehicle. (For example, the duck 810 appears on the display on the front and left sides of the vehicle, and the duck 810 does not appear on the display on the right and rear of the vehicle), through which the vehicle occupant enters a specific space. You will be able to provide the same user experience as there is.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention or may be known and usable to those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks. medium), and a hardware device specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device can be changed to one or more software modules to perform the processing according to the present invention, and vice versa.

In the above, the present invention has been described by specific matters such as specific elements and limited embodiments and drawings, but this is provided only to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. Anyone with ordinary knowledge in the technical field to which the invention belongs can make various modifications and changes from these descriptions.

Accordingly, the spirit of the present invention is limited to the above-described embodiments and should not be defined, and all ranges equivalent to or equivalently changed from the claims to be described later as well as the claims to be described later are the scope of the spirit of the present invention. It will be said to belong to.

100: communication network
200: content providing system
210: virtual object sequence management unit
220: content management unit
230: communication department
240: control unit
300: display

Claims (23)

As a method of providing content,
Determining an image sequence for a virtual object by referring to at least one of situation information of at least one vehicle occupant and driving information of the vehicle, and
And determining content to be provided to the at least one vehicle occupant by matching the image sequence of the virtual object based on the external real world of the vehicle projected through at least one display of the vehicle.
Way. The method of claim 1,
In the step of determining the image sequence, at least one of the situation information of the at least one vehicle occupant and the at least one vehicle occupant is specified from an internal image of the vehicle.
Way. The method of claim 1,
In the step of determining the image sequence, determining an image sequence for the virtual object by further referring to information on a real-world object specified from an external image of the vehicle
Way. The method of claim 1,
In the step of determining the image sequence, determining an image sequence for the virtual object for each of the vehicle occupants or each of the group of vehicle occupants
Way. The method of claim 1,
In the step of determining the image sequence, determining an image sequence relating to the virtual object for each seat in the vehicle occupant or for each group of seats in the vehicle
Way. The method of claim 1,
In the step of determining the image sequence, an image of the virtual object is referred to at least one of a view point of the vehicle occupant, a view point according to a seat position of the vehicle, and a view point according to a position in the vehicle of the at least one display. To determine the sequence
Way. The method of claim 1,
In the step of determining the image sequence, determining an image sequence for the virtual object by further referring to personal information of the vehicle occupant
Way. The method of claim 1,
In the content determination step, matching an image sequence related to the virtual object by referring to a property related to an external real world object of the vehicle projected through the at least one display
Way. The method of claim 1,
The content determining step includes providing the determined content based on a display associated with the at least one vehicle occupant among the at least one display.
Way. The method of claim 9,
Providing the determined content based on a display corresponding to the gaze of each of the at least one vehicle occupant among the at least one display
Way. The method of claim 1,
The content determination step includes synchronizing and providing the determined content on the at least one display based on a time point corresponding to the in-vehicle location of each of the at least one display.
Way. A non-transitory computer-readable recording medium recording a computer program for executing the method according to claim 1. As a system for providing content,
A virtual object sequence management unit that determines an image sequence for a virtual object by referring to at least one of situation information of at least one vehicle occupant and driving information of the vehicle, and
A content management unit that determines content to be provided to the at least one vehicle occupant by matching the image sequence of the virtual object based on the external real world of the vehicle projected through at least one display of the vehicle.
system. The method of claim 13,
At least one of the situation information of the at least one vehicle occupant and the at least one vehicle occupant is specified from an internal image of the vehicle
system. The method of claim 13,
The virtual object sequence management unit determines an image sequence related to the virtual object by further referring to information on a real-world object specified from an external image of the vehicle.
system. The method of claim 13,
The virtual object sequence management unit is configured to determine an image sequence for the virtual object for each of the vehicle occupants or for each group of vehicle occupants.
system. The method of claim 13,
The virtual object sequence management unit is configured to determine an image sequence for the virtual object for each seat in which the vehicle occupant is seated in the vehicle or for each group of seats.
system. The method of claim 13,
The virtual object sequence management unit determines an image sequence for the virtual object by referring to at least one of a viewpoint of the vehicle occupant, a viewpoint according to a seat position of the vehicle, and a viewpoint according to a position in the vehicle of the at least one display.
system. The method of claim 13,
The virtual object sequence management unit further determines an image sequence for the virtual object by referring to personal information of the vehicle occupant.
system. The method of claim 13,
The content management unit matches an image sequence related to the virtual object by referring to a property related to an object of the external real world of the vehicle projected through the at least one display.
system. The method of claim 13,
The content management unit provides the determined content based on a display associated with the at least one vehicle occupant among the at least one display.
system. The method of claim 21,
The content management unit is configured to provide the determined content based on a display corresponding to each line of sight of the at least one vehicle occupant among the at least one display.
system. The method of claim 13,
The content management unit synchronizes and provides the content determined on the at least one display based on a time point corresponding to a position in the vehicle of each of the at least one display.
system.

Images