KR102477703B1 - Method, system, and non-transitory computer-readable recording medium for authoring contents based on in-vehicle video - Google Patents

Abstract

According to one aspect of the present invention, there is provided a method for authoring content based on an image inside a vehicle, comprising: acquiring an effect parameter associated with a vehicle occupant from the image inside the vehicle; specifying at least one effect object based on the effect parameter. and determining a sequence of the specified at least one effect object, and authoring content by combining the sequence of the at least one effect object with the image inside the vehicle.

Description

Method, system and non-temporary computer readable recording medium for authoring contents based on video inside a vehicle

The present invention relates to a method, system, and non-transitory computer readable recording medium for authoring contents based on images inside a vehicle.

In recent years, the self-driving car market has been activated, such as the commercialization of self-driving vehicles along with the development of communication and artificial intelligence (AI) technologies. These self-driving cars are equipped with numerous cameras in order to detect surrounding obstacles or to grasp conditions inside the vehicle, through which the interior or exterior of the vehicle is photographed while driving.

However, the captured video inside the vehicle has no other purpose than to determine the presence or absence of occupants, so it was automatically deleted after the vehicle was driven (or while the vehicle was running), or it was overwritten with a new image and recorded.

Accordingly, the inventors of the present invention propose a new and advanced technology that can stimulate the interest of vehicle occupants and increase the utilization of images by supporting content authoring based on images inside the vehicle.

Korean Patent Publication No. 2018-0014910

The present invention has as its object to solve all the problems of the prior art described above.

In addition, an object of the present invention is to support anyone to easily and conveniently create content based on an image inside a vehicle.

Another object of the present invention is to increase the usability of vehicle interior images and to provide interest to vehicle occupants who may be bored while driving the vehicle.

Representative configurations of the present invention for achieving the above object are as follows.

According to one aspect of the present invention, there is provided a method for authoring content based on an image inside a vehicle, comprising: acquiring an effect parameter associated with a vehicle occupant from the image inside the vehicle; specifying at least one effect object based on the effect parameter. and determining a sequence of the specified at least one effect object, and authoring content by combining the sequence of the at least one effect object with the image inside the vehicle.

According to another aspect of the present invention, a system for authoring content based on an image inside a vehicle, comprising: an effect parameter obtaining unit acquiring an effect parameter associated with a vehicle occupant from the image inside the vehicle; and based on the effect parameter, at least one Includes a sequence determining unit that specifies an effect object and determines a sequence related to the specified at least one effect object, and a content authoring unit that creates content by combining the sequence related to the at least one effect object with the image inside the vehicle A system is provided.

In addition to this, another method for implementing the present invention, another system, and a non-transitory computer readable recording medium recording a computer program for executing the method are further provided.

According to the present invention, anyone can easily and conveniently create content based on an image inside a vehicle.

In addition, according to the present invention, it is possible to increase the usability of the image inside the vehicle and to give interest to vehicle occupants who may be bored while driving the vehicle.

1 is a diagram showing a schematic configuration of an entire system for authoring content based on an image inside a vehicle according to an embodiment of the present invention.
2 is a diagram showing in detail the internal configuration of a content authoring system according to an embodiment of the present invention.
3 to 5 exemplarily illustrate situations in which content is authored based on an image inside a vehicle according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description of the present invention which follows refers to the accompanying drawings which illustrate, by way of illustration, specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable any person skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different from each other but are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented from one embodiment to another without departing from the spirit and scope of the present invention. It should also be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Therefore, the detailed description to be described later is not performed in a limiting sense, and the scope of the present invention should be taken as encompassing the scope claimed by the claims and all scopes equivalent thereto. Like reference numbers in the drawings indicate the same or similar elements throughout the various aspects.

In this specification, content is a concept that collectively refers to digital information or individual information elements composed of text, code, voice, sound, image, video, etc. accessible through a communication network. Such content may include, for example, data such as text, image, video, audio, link (eg, web link), or a combination of at least two of these data.

In addition, in this specification, an object is a concept that collectively refers to an object that can be implemented or added to a predetermined area in an image, and is, for example, a letter, number, sign, symbol, figure, photograph, image, sound, video, or It may be made of a combination of these, and may be a concept including even attributes (eg, brightness, brightness, saturation) that may be assigned thereto.

Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable those skilled in the art to easily practice the present invention.

Composition of the entire system

1 is a diagram showing a schematic configuration of an entire system for authoring content based on an image inside a vehicle according to an embodiment of the present invention.

As shown in FIG. 1 , the entire system according to an embodiment of the present invention may include a communication network 100 , a content authoring system 200 and a device 300 .

First, the communication network 100 according to an embodiment of the present invention may be configured regardless of communication aspects such as wired communication or wireless communication, and may include a local area network (LAN) and a metropolitan area network (MAN). ), a wide area network (WAN), and the like. Preferably, the communication network 100 referred to in this specification may be the well-known Internet or the World Wide Web (WWW). However, the communication network 100 may include at least a part of a known wired/wireless data communication network, a known telephone network, or a known wired/wireless television communication network without being limited thereto.

For example, the communication network 100 is a wireless data communication network, Radio Frequency (RF) communication, Wi-Fi (WiFi) communication, cellular (LTE, etc.) communication, Bluetooth communication (more specifically, Bluetooth Low Energy (BLE; It may implement a conventional communication method such as Bluetooth Low Energy (Bluetooth Low Energy) communication), infrared communication, ultrasonic communication, and the like, at least in part.

Next, the content authoring system 200 according to an embodiment of the present invention may be a digital device equipped with a memory means and equipped with a microprocessor to have arithmetic capability. For example, this content authoring system 200 may be a server system.

The content authoring system 200 according to an embodiment of the present invention obtains an effect parameter associated with a vehicle occupant from an image inside a vehicle, specifies at least one effect object based on the effect parameter, and specifies at least one specified effect parameter. It is possible to perform a function of determining a sequence of effect objects of and combining the sequence of at least one effect object with the image inside the vehicle to create content.

An image inside a vehicle according to an embodiment of the present invention includes at least one camera module (eg, an RGB camera, an infrared camera, an ultraviolet camera, and a time-of-flight (TOF) camera) mounted so that the photographing direction faces the inside of the vehicle. etc.) or at least one scanner module (eg, LiDAR sensor, radar sensor, etc.) may refer to an image captured in real time or for a predetermined period of time. In addition, the vehicle interior image as above may include a 2D image or a 3D image. On the other hand, the type of camera module or scanner module according to an embodiment of the present invention is not necessarily limited to the above-listed embodiments, and can be variously changed within the scope of achieving the object of the present invention. make it clear

The configuration and functions of the content authoring system 200 according to the present invention will be described in more detail below. On the other hand, although the content authoring system 200 has been described as above, this description is exemplary, and at least some of the functions or components required for the content authoring system 200 may be provided by the device 300 or an external device, which will be described later, as necessary. It is apparent to those skilled in the art that it may be implemented within a system (not shown) or included within the device 300 or an external system.

Next, the device 300 according to an embodiment of the present invention is a digital device having a function to communicate after accessing the content authoring system 200 through the communication network 100, such as a smartphone, tablet PC, etc. Likewise, any portable digital device equipped with a memory means and equipped with a microprocessor and equipped with an arithmetic capability can be adopted as the device 300 according to the present invention. Also, according to an embodiment of the present invention, the device 300 may further include a display module (eg, LCD, LED display, OLED display, etc.) for reproducing content.

Meanwhile, the device 300 according to an embodiment of the present invention may include an application for supporting a content authoring function according to the present invention. Such an application may be downloaded from the content authoring system 200 or an external application distribution server (not shown).

Configuration of content authoring system

Hereinafter, the internal configuration of the content authoring system 200 that performs important functions for the implementation of the present invention and the functions of each component will be reviewed.

2 is a diagram showing in detail the internal configuration of a content authoring system 200 according to an embodiment of the present invention.

As shown in FIG. 2, it is composed of an effect parameter acquisition unit 210, a sequence determination unit 220, a contents authoring unit 230, a contents management unit 240, a communication unit 250, and a control unit 260. can According to an embodiment of the present invention, the effect parameter acquisition unit 210, the sequence determination unit 220, the contents authoring unit 230, the contents management unit 240, the communication unit 250, and the control unit 260 include at least one of them. Some may be program modules that communicate with external systems. These program modules may be included in the content authoring system 200 in the form of an operating system, application program module, or other program module, and may be physically stored in various known storage devices. Also, these program modules may be stored in a remote storage device capable of communicating with the content authoring system 200 . Meanwhile, these program modules include routines, subroutines, programs, objects, components, data structures, etc. that perform specific tasks or execute specific abstract data types according to the present invention, but are not limited thereto.

First, the effect parameter acquisition unit 210 according to an embodiment of the present invention may perform a function of obtaining an effect parameter associated with a vehicle occupant from a vehicle interior image.

For example, the effect parameter acquisition unit 210 may specify at least one feature point or feature region related to the effect parameter in the vehicle interior image, and obtain the vehicle interior image based on the at least one feature point or feature region. Effect parameters may be obtained by analysis (eg, analysis of shape, size, color, frequency, pitch, etc.).

More specifically, the effect parameter obtaining unit 210 may (i) the size, shape, and shape of a predetermined body part (eg, eyes, eyebrows, pupils, nose, mouth, etc.) of a vehicle occupant specified in a vehicle interior image; Or information about changes in size or shape, (ii) the position of a predetermined body part (eg, head, hand, foot, etc.) of a vehicle occupant specified in the vehicle interior image or a change in the position (or the corresponding body part) Gesture by part), (iii) sound generated from a predetermined body part of the vehicle occupant specified in the vehicle interior image (eg, sound generated from the mouth) or sound generated by the predetermined body part Information on (eg, sound generated by tapping a headrest or steering wheel of a vehicle with a hand) may be obtained as an effect parameter associated with a vehicle occupant.

Meanwhile, the effect parameter obtaining unit 210 separates an image inside the vehicle into at least one of a video sequence and an audio sequence, and an effect parameter related to time (eg, an effect related to the expression or movement of a vehicle occupant) from the separated image sequence. parameters), and an effect parameter related to hearing (eg, an effect parameter related to a sound generated by a vehicle occupant) may be obtained from the separated voice sequence.

Next, the sequence determination unit 220 according to an embodiment of the present invention specifies at least one effect object based on the effect parameters obtained by the above effect parameter acquisition unit 210, and the specified at least one It can perform the function of determining the sequence of the effect object of. An effect object according to an embodiment of the present invention may include an object based on at least one of sight, hearing, touch, and smell. For example, when the effect object is a visual object, an image or video based object may be included as the above effect object.

For example, the sequence determiner 220 may determine the emotion of the vehicle occupant (eg, joy or anger) based on information on the shape of a predetermined body part in the vehicle occupant's face (ie, effect parameter). ), sadness (哀), joy (樂, etc.) can be estimated, at least one effect object associated with the estimated emotion can be specified, and at least one effect object that is specified And based on the above effect parameter (or information about the above emotion), a sequence related to the at least one effect object may be determined. More specifically, when the emotion of the vehicle occupant estimated based on the information about the shape of a predetermined body part in the face of the vehicle occupant is "joy", the sequence determiner 220 selects at least one musical note image as "joy". It can be specified as an effect object associated with "joy", and according to the corresponding emotion (ie, "joy") (ie, according to the emotion estimated based on the effect parameter), at least one note image above is active for a predetermined time. It can be determined as a sequence of at least one effect object above. In addition, when the emotion of the vehicle occupant estimated based on the information about the shape of a predetermined body part in the face of the vehicle occupant is "sadness", the sequence determining unit 220 sets the background image to "sadness". It can be specified as an effect object, and the brightness of the background image darkens for a predetermined time according to the corresponding emotion (ie, “sadness”) (ie, according to the emotion estimated based on the effect parameter) at least one of the above It can be determined as a sequence about the effect object of

For another example, the sequence determination unit 220 may generate a sound generated by a vehicle occupant (eg, a sound directly generated through a vehicle occupant's body organ or an indirect sound generated by a vehicle occupant's body organ hitting another object). It is possible to specify at least one effect object based on information (eg, information on at least one of the intensity, pitch, and timbre of the above sound) (ie, an effect parameter) on the generated sound), and the specific A sequence related to the at least one effect object may be determined based on information about the at least one effect object and the sound. More specifically, the sequence determiner 220 may specify a background image as an effect object based on information on the intensity of sound generated by a vehicle occupant, and whenever the above sound is generated, the background image It may be determined that the brightness changes (or flickers) in proportion to the intensity of the above sound as a sequence related to the at least one effect object.

As another example, the sequence determination unit 220 may specify at least one effect object based on information about a gesture generated by a vehicle occupant, and the specified at least one effect object and the gesture A sequence related to the at least one effect object may be determined based on the related information. More specifically, the sequence determination unit 220 effects a wave image and a wave sound based on information on a vehicle occupant's hand gesture (eg, including information on a generation position, speed, direction, etc. of the hand gesture). It can be specified as an object, a wave image is generated along the generation position and direction of the above hand gesture, and the beat of the wave sound is changed according to the speed of the above hand gesture. can be determined as

However, it should be noted that the effect objects and sequences related thereto according to an embodiment of the present invention are not necessarily limited to those listed above, and may be changed as much as possible within the scope of achieving the object of the present invention.

Meanwhile, the sequence determining unit 220 may determine a sequence of at least one effect object by referring to at least one of a lookup table for an effect object associated with an effect parameter and a lookup table for a sequence associated with an effect parameter. there is.

In addition, the sequence determination unit 220 may specify the at least one effect object based on personal information of the vehicle occupant specified in the vehicle interior image, and determine a sequence related to the specified at least one effect object. . Personal information of a vehicle occupant according to an embodiment of the present invention may include information about the vehicle occupant's age, gender, race, region, preferred effect object, interest, and the like.

For example, the sequence determining unit 220 specifies the identity of the vehicle occupant based on the face of the vehicle occupant in the vehicle interior image, and at least one effect object preferred by the vehicle occupant according to the identified identity and at least one effect object preferred by the vehicle occupant. A sequence can be determined for one effect object. That is, based on personal information of a vehicle occupant, an effect object or a sequence related to the effect object may be differently customized and determined.

Meanwhile, the sequence determiner 220 may further specify the at least one effect object based on situation information related to the vehicle occupant and determine a sequence related to the specified at least one effect object. Circumstance information according to an embodiment of the present invention may include the vehicle occupant or the atmosphere around (or in the vehicle) the vehicle occupant (eg, joy, pleasure, sadness, anger, etc.), the vehicle occupant's movement, and the vehicle occupant's It may include information indicating a situation inferred from a voice or the like. For example, a case corresponding to the same or similar information as information about the vehicle occupant or the atmosphere around (or in the vehicle) of the vehicle occupant, the movement of the vehicle occupant, the voice of the vehicle occupant, etc. may be retrieved from the case database, Accordingly, the searched case may be information indicating a corresponding situation. This case database may be dynamically updated by performing deep learning, machine learning, and the like, based on various cases related to a plurality of vehicle occupants. That is, even when the same effect parameter is obtained, an effect object or a sequence related to the effect object may be determined differently according to situational information (eg, atmosphere) associated with a vehicle occupant.

In addition, the sequence determiner 220 may further specify the at least one effect object based on driving information of the vehicle, and determine a sequence related to the specified at least one effect object. The driving information according to an embodiment of the present invention may include information about a starting point, a destination, a current location, a driving speed, and a driving state (eg, stop, start, etc.) of a vehicle.

For example, when the vehicle approaches the destination, the sequence determination unit 220 determines at least one effect object associated with the finish point (eg, a finish line ribbon image that can be created under the theme of the finish point, etc.) and its specific It is possible to determine a sequence (eg, a sequence in which a ribbon image of a finish line that can be created based on the theme of a finish point is scattered) related to at least one effect object.

For another example, when the speed of the vehicle is less than or equal to a predetermined level, the sequence determining unit 220 determines at least one effect object associated with the slowdown (eg, a turtle image) and at least one effect object specified therein. A related sequence (eg, a sequence in which a tortoise image slowly crawls in response to a vehicle driving speed) may be determined.

Next, the content authoring unit 230 according to an embodiment of the present invention combines a sequence of at least one effect object determined by the above sequence determining unit 220 with an image inside the vehicle to author content. can be performed.

For example, the content authoring unit 230 combines a sequence related to at least one effect object above based on an image inside a vehicle to generate augmented reality (AR) images and mixed reality (MR) images. At least one content can be authored.

Next, the content management unit 240 may perform a function of storing or sharing content authored by the content authoring unit 230 .

For example, the content management unit 240 allows the above authored content to be stored in the device 300 (eg, a smartphone of the vehicle occupant, a cloud, etc.) associated with the vehicle occupant, or a social network associated with the vehicle occupant. Through a network service (SNS), cloud, web server, etc., the authored content above can be shared with other vehicle occupants (eg, real-time sharing).

Next, the communication unit 250 according to an embodiment of the present invention transmits and receives data from/to the effect parameter acquisition unit 210, the sequence determination unit 220, the content authoring unit 230, and the content management unit 240. It can perform functions that make it possible.

Finally, the control unit 260 according to an embodiment of the present invention controls data between the effect parameter acquisition unit 210, the sequence determination unit 220, the content authoring unit 230, the content management unit 240, and the communication unit 250. It can perform the function of controlling the flow of That is, the control unit 260 according to the present invention controls the data flow from/to the outside of the content authoring system 200 or the data flow between each component of the content authoring system 200, so that the effect parameter acquisition unit 210 , The sequence determination unit 220, the contents authoring unit 230, the contents management unit 240, and the communication unit 250 can each be controlled to perform unique functions.

3 to 5 exemplarily illustrate situations in which content is authored based on an image inside a vehicle according to an embodiment of the present invention.

Example 1

Referring to FIG. 3 , it may be assumed that a content authoring system 200 according to an embodiment of the present invention is included in a device 300 in a vehicle. Here, the device 300 may include a tablet or a smart phone that can be fixed to or detachable from a predetermined place (eg, a headrest, a steering wheel, etc.) in a vehicle.

First, the in-vehicle device 300 provides information on at least one of the intensity, pitch, and tone of sound generated by the vehicle occupant repeatedly tapping the steering wheel from an image inside the vehicle (see FIG. 3(a)). can be obtained as an effect parameter.

Then, the in-vehicle device 300 specifies at least one note image as at least one effect object based on information on at least one of the above sound intensity, pitch, and tone, and determines the above sound pitch. Accordingly, the color of at least one note image above is changed (for example, the higher the pitch of the above sound, the more red the color becomes, and the lower it becomes blue), and according to the intensity of the above sound, the color of the above at least The size of one note image is changed (for example, the size increases as the intensity of the above sound increases and decreases as the intensity of the above sound decreases), and the brightness of the image of at least one note above changes according to the timbre of the above sound. A sequence may be determined as a sequence related to the above at least one musical note image.

Next, the in-vehicle device 300 combines the sequence of at least one musical note image determined above with the image inside the vehicle (ie, the real world image) to obtain augmented reality content (see (b) of FIG. 3). can author

Second embodiment

A situation in which the content authoring system 200 according to an embodiment of the present invention is included in the in-vehicle device 300 may be assumed. Here, the device 300 may refer to a tablet or smartphone that can be fixed or detached from a predetermined place (eg, a headrest, a steering wheel, etc.) in a vehicle.

First, the in-vehicle device 300 may obtain, as an effect parameter, information on at least one of the beat and rhythm of a sound generated by a song from an image inside a vehicle in which a vehicle occupant sings a song.

Next, the in-vehicle device 300 specifies at least one note image as at least one effect object based on information on at least one of the beat and rhythm of the above sound, and among the beat and rhythm of the above sound. A sequence in which the at least one note image moves rapidly or increases in number according to at least one change may be determined as a sequence related to the at least one note image.

Next, the in-vehicle device 300 may create augmented reality content by combining the sequence of the at least one musical note image with the in-vehicle image.

Third embodiment

Referring to FIG. 4, a part of the content authoring system 200 according to the present invention is included in an in-vehicle device (not shown) (device 300 according to the present invention), and another part is included in an external server (not shown). ) can be assumed. Specifically, it may be a situation in which content is authored through the above device and an external server connected to the device through wireless communication (eg, WiFi, Bluetooth, etc.).

First, from the image inside the vehicle (refer to (a) of FIG. 4 ) in which one of the vehicle occupants is making a sleepy expression (or sleeping) and the other is making a happy expression, the shape of the predetermined body part of the face of the two vehicle occupants is determined. Information about it can be obtained as an effect parameter.

Then, it can be estimated that one vehicle occupant's emotion is "sleepy" and the other vehicle occupant's emotion is "joyful" based on the above information about the shape of certain body parts of the vehicle occupant's face, and , each of the water surface image and the angel image associated with the estimated emotion may be specified as at least one effect object.

Then, following each of the above water image and the above angel image by tracking the shape of a predetermined body part of the face of each of the above two vehicle occupants can be determined as a sequence related to the above water image and angel image.

Then, augmented reality content (refer to FIG. 4(b)) may be authored by combining the determined sequence of the water image and the angel image with the image inside the vehicle.

4th embodiment

Referring to FIG. 5 , it may be assumed that the content authoring system 200 according to an embodiment of the present invention is included in an external server (not shown). Specifically, content is authored through an external server connected to an in-vehicle device (not shown) (device 300 according to the present invention) and wireless communication (eg, WiFi, Bluetooth, etc.) it could be a situation

First, information about the location, speed, and direction of the fist may be obtained as an effect parameter from an inside image of a vehicle occupant swinging a fist (refer to FIG. 5(a) ).

Then, specifying at least one boxing glove image and boxing punching bag image as at least one effect object based on the information on the generation position, speed and direction of the above fist, and the generation position, speed and direction of the above fist. A sequence in which the boxing glove is swung so as to correspond to and the boxing punching bag moves accordingly can be determined as a sequence related to the above boxing glove image and boxing punching bag image.

Then, the above image of the inside of the vehicle (real world image) is added to the above image of the outside of the vehicle (virtual reality image; such an image may be generated based on the above image inside the vehicle through virtual reality generation technology)) ) and the above sequence of the boxing glove image and the boxing punching bag image may be combined to create mixed reality content (see (b) of FIG. 5 ).

Embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler. A hardware device may be modified with one or more software modules to perform processing according to the present invention and vice versa.

Although the present invention has been described above with specific details such as specific components and limited embodiments and drawings, these are only provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments, and the present invention Those with ordinary knowledge in the technical field to which the invention belongs may seek various modifications and changes from these descriptions.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments and should not be determined, and all scopes equivalent to or equivalently changed from the claims as well as the claims described below are within the scope of the spirit of the present invention. will be said to belong to

100: communication network
200: content authoring system
210: effect parameter acquisition unit
220: sequence determining unit
230: content authoring department
240: content management unit
250: communication department
260: control unit
300: device

Claims (21)

As a method of authoring content based on an image inside a vehicle,
Obtaining an effect parameter associated with a vehicle occupant from a vehicle interior image, which is a real world image;
specifying at least one effect object based on at least one sense based on the effect parameter, and determining a sequence related to the specified at least one effect object; and
Authoring at least one content of an Augmented Reality (AR) image and a Mixed Reality (MR) image by combining a sequence related to the at least one effect object based on the image inside the vehicle.
method. According to claim 1,
The acquisition step is
specifying at least one feature point or feature region associated with the effect parameter in the vehicle interior image; and
Obtaining the effect parameter by analyzing the vehicle interior image based on the at least one feature point or feature region
method. According to claim 1,
The acquisition step is
Separating the vehicle interior image into at least one of a video sequence and an audio sequence; and
Obtaining an effect parameter associated with time among the effect parameters from the video sequence, and acquiring an effect parameter associated with hearing among the effect parameters from the audio sequence.
method. According to claim 1,
In the determining step, the at least one effect object and the sequence related to the at least one effect object are associated with information about the emotion of the vehicle occupant.
method. According to claim 1,
In the determining step, the at least one effect object and the sequence related to the at least one effect object are associated with information about a sound generated by the vehicle occupant.
method. According to claim 1,
In the determining step, the at least one effect object and the sequence related to the at least one effect object are associated with information about a gesture generated by the vehicle occupant.
method. According to claim 1,
In the determining step, determining the at least one effect object and a sequence related to the at least one effect object further based on at least one of personal information of the vehicle occupant and situation information associated with the vehicle occupant.
method. According to claim 1,
In the determining step, determining the at least one effect object and a sequence related to the at least one effect object further based on driving information of the vehicle
method. delete According to claim 1,
Further comprising the step of allowing the authored content to be stored or shared
method. A non-temporary computer readable recording medium storing a computer program for executing the method according to claim 1. A system for authoring contents based on images inside a vehicle,
an effect parameter acquiring unit for obtaining an effect parameter associated with a vehicle occupant from a vehicle interior image, which is a real world image;
a sequence determiner configured to specify at least one effect object based on at least one sense based on the effect parameter, and to determine a sequence related to the specified at least one effect object; and
A content authoring unit for authoring at least one content of an augmented reality (AR) image and a mixed reality (MR) image by combining a sequence related to the at least one effect object based on the image inside the vehicle. doing
system. According to claim 12,
The effect parameter acquiring unit identifies at least one feature point or feature region associated with the effect parameter in the vehicle interior image, and analyzes the vehicle interior image based on the at least one feature point or feature region, thereby determining the effect parameter. to acquire
system. According to claim 12,
The effect parameter acquiring unit separates the interior image of the vehicle into at least one of a video sequence and a sound sequence, obtains an effect parameter related to time among the effect parameters from the video sequence, and obtains an effect parameter related to time among the effect parameters from the audio sequence, and hearing and hearing among the effect parameters. Acquiring the associated effect parameters
system. According to claim 12,
The at least one effect object and the sequence related to the at least one effect object are related to the information about the emotion of the vehicle occupant.
system. According to claim 12,
The at least one effect object and the sequence related to the at least one effect object are associated with information about sound generated by the vehicle occupant.
system. According to claim 12,
The at least one effect object and the sequence related to the at least one effect object are related to information about a gesture generated by the vehicle occupant.
system. According to claim 12,
The sequence determining unit determines the at least one effect object and a sequence related to the at least one effect object further based on at least one of personal information of the vehicle occupant and situation information associated with the vehicle occupant.
system. According to claim 12,
The sequence determining unit further determines the at least one effect object and a sequence related to the at least one effect object based on driving information of the vehicle.
system. delete According to claim 12,
Further comprising a content management unit for storing or sharing the authored content
system.

Images