WO2024076206A1

WO2024076206A1 - Apparatus and method for playing responsive video based on operation pattern

Info

Publication number: WO2024076206A1
Application number: PCT/KR2023/015424
Authority: WO
Inventors: 이철우
Original assignee: 이철우
Priority date: 2022-10-07
Filing date: 2023-10-06
Publication date: 2024-04-11

Abstract

The present invention relates to an apparatus and method for playing a responsive video based on an operation pattern. The apparatus comprises: an input module that displays a responsive video and detects an input operation from a user; a storage module that stores one or more processes for playing the responsive video on the basis of one or more operation patterns and storing pieces of video frame information respectively corresponding to the one or more operation patterns; a handler module that identifies an event corresponding to the input operation among one or more events and generates and delivers input operation information including information about the input operation according to the identified event; a playback module that identifies an operation pattern corresponding to the input operation among the one or more operation patterns on the basis of the input operation information and plays the responsive video by generating video data using video frame information corresponding to the identified operation pattern; and a control module that performs a series of control operations to play the responsive video on the basis of the at one or more processes.

Description

Responsive video playback device and method based on operation patterns

The present disclosure relates to a responsive video playback device and method. More specifically, the present disclosure relates to a responsive video playback device and method based on an operation pattern.

Recently, video recording technology has developed greatly. Not only camcorders and digital cameras, but also mobile devices such as smartphones can capture high-resolution images. Additionally, 360-degree cameras and 3D video cameras are appearing.

Videos are captured by a video recording device, stored in a specific format, and played back by a terminal capable of playing them. Video playback is provided unilaterally in chronological order without interaction with the viewer. In other words, the viewer can only feel the visual sense through the video being played.

By implementing and providing such images as responsive images that respond to input operations based on the user's various gestures, the user's interest can be stimulated.

Therefore, there is a need to develop technology that enables quick and smooth playback of responsive videos according to such input operations.

The present disclosure provides a responsive video playback device and method based on operating patterns that allow the responsive video to be played quickly and smoothly even according to operating patterns input from different users.

In addition, the present disclosure is based on Web Real Time Communication (Web RTC), and when a second operation pattern is input from a specific user while the first operation pattern, which is the basic operation pattern, is applied, the first operation pattern is applied. To provide an operation pattern-based responsive video playback device and method that stops the operation and applies the second operation pattern from that point to play and output the responsive image.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below.

A responsive image playback device based on an operation pattern according to an aspect of the present disclosure for achieving the above-described technical problem includes an input module that displays a responsive image and detects an input operation from a user; a storage module that stores at least one process for playing a responsive video based on at least one operation pattern and stores image frame information corresponding to each of the at least one operation pattern; a handler module that identifies an event corresponding to the input operation among at least one event, and generates and transmits input operation information including information about the input operation according to the confirmed event; Based on the input operation information, the operation pattern corresponding to the input operation among the at least one operation pattern is confirmed, and image data is generated using the image frame information corresponding to the confirmed operation pattern to display the responsive video. a playback module that performs playback operations; and a control module that performs a series of control operations for playing the responsive image based on the at least one process, wherein the at least one operation pattern is basically set for each of the at least one event. It may include a first operation pattern and a second operation pattern that is stored differently for each user by reflecting the input characteristics of each user.

Meanwhile, a method of playing a responsive image based on an operation pattern according to an aspect of the present disclosure includes the steps of displaying a responsive image and detecting a user's input operation on the responsive image; When the input manipulation is detected, checking an event corresponding to the input manipulation among at least one event; generating and delivering input operation information including information about the input operation according to the confirmed event; Confirming an operation pattern corresponding to the input operation among at least one operation pattern based on the input operation information; and performing a playback operation on the responsive video by generating video data using video frame information corresponding to the identified manipulation pattern, wherein the at least one manipulation pattern is one of the at least one event. It may include a first operation pattern that is basically set for and a second operation pattern that is stored differently for each user by reflecting the input characteristics of each user.

In addition to this, a computer program stored in a computer-readable recording medium for executing the method for implementing the present disclosure may be further provided.

In addition, a computer-readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

According to the means for solving the above-described problem of the present disclosure, the responsive video can be played quickly and smoothly even in response to operation patterns input from different users.

In addition, according to the present disclosure, it is based on Web Real Time Communication (Web RTC), and when a second operation pattern is input from a specific user while the first operation pattern, which is a basic operation pattern, is applied, the first operation pattern is Stop applying and apply the second operation pattern from that point to play and output the responsive video.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned may be clearly understood by those skilled in the art from the description below.

1 is a block diagram showing the configuration of a responsive video playback device based on an operation pattern according to an embodiment of the present disclosure.

Figure 2 is a flow chart showing a responsive video playback method based on an operation pattern according to an embodiment of the present disclosure.

FIG. 3 is a diagram illustrating a specific operation of performing a playback operation for a responsive video in the operation pattern-based responsive video playback method according to an embodiment of the present disclosure.

Like reference numerals refer to like elements throughout this disclosure. The present disclosure does not describe all elements of the embodiments, and general content or overlapping content between the embodiments in the technical field to which the present disclosure pertains is omitted. The term 'unit, module, member, block' used in the specification may be implemented as software or hardware, and depending on the embodiment, a plurality of 'unit, module, member, block' may be implemented as a single component, or It is also possible for one 'part, module, member, or block' to include multiple components.

Additionally, when a part is said to “include” a certain component, this means that it may further include other components, rather than excluding other components, unless specifically stated to the contrary.

Singular expressions include plural expressions unless the context clearly makes an exception. The identification code for each step is used for convenience of explanation. The identification code does not explain the order of each step, and each step may be performed differently from the specified order unless a specific order is clearly stated in the context. there is.

Before explaining the operating principles and embodiments of the present disclosure with reference to the attached drawings, some terms will be explained as follows.

Content (content or content) may include various contents provided visually, such as videos, still images, holograms, etc., and may include various contents provided through auditory, gustatory, olfactory, etc. The example is not limited to this. Additionally, content may be provided in virtual reality (VR), but the embodiment is not limited thereto.

Objects included in content may be included in part of the content or may include the entire content. For example, if the content is a video, it includes various objects located within the entire frame or part of the frame of the video. It can be done, and it can also mean the video itself.

Responsive content is a command that triggers a 'reaction (reaction, feedback, etc.)' of an object included in the content (which can be called an input operation, such as a reserved command including touch operation, sound command, motion command, etc.) ) is input, it may include various contents related to the corresponding object. Hereinafter, the description will be made on the assumption that the responsive content according to the present disclosure is a responsive video, and the type of responsive content according to the present disclosure is not limited to being a video.

Here, the response may include movement of the object, change in shape/shape of the object, occurrence of a specific event, and/or occurrence of change in content according to a command (input operation), but the embodiment is not limited thereto.

Responsive video refers to a video that is played in a form corresponding to a command (eg, touch operation) by a user (ie, viewer) watching the video. For example, a responsive image may refer to an image in which the movement of touching an object is played when a user operation in the form of touching a specific object (for example, a pillow) is applied to the touch screen. Also, for example, responsive video refers to an image in which when a user manipulation in the form of pressing a specific object is applied to the touch screen, the movement of the object being pressed and the movement restored after the user manipulation are played. You can.

A 'command for triggering a response' (input manipulation) of an object included in content may include a user manipulation of the content received through an input means of a computer that provides a responsive image. For example, the user operation is an operation that can be input at a specific point or area in the content through an input means such as a mouse or a touch screen (e.g., a click operation, a drag operation, a contact touch operation for a certain period of time, or a force touch) It may include manipulation (i.e., touch manipulation of applying specific pressure to a touch screen or touch pad), etc.). In addition, for example, the user operation involves the arrangement or movement of the terminal itself, which can be obtained by using a sensor (e.g., acceleration sensor, gyro sensor, etc.) provided by the computer (or terminal) as an input means. It can be included. When the content is VR content, a command for triggering a reaction may be performed by sensing the movement of a worn terminal or manipulating a terminal such as a joystick, but the embodiment is not limited thereto.

Below, a method for creating a responsive image will be described. The description will assume that the creation of a responsive image is performed by a processor (control module 1500) of a device (eg, a computer).

The control module 1500 may determine a command to trigger a response of an object included in one or more original images. Here, the original video can be called a basic video, and the original video may be content that is not implemented in a responsive manner. For example, the original video may be captured content and may include a combination of a plurality of frames storing frames for each position of an object in space. The original video may be content collected through communication, may be three-dimensional content, or may be VR content, but the embodiment is not limited thereto.

The control module 1500 may receive an input specifying a frame section of the original video, and the frame section may include a specific section to be implemented in a responsive manner among all frames of the original video. The frame section can be set by the user through various methods. In an embodiment, the control module 1500 receives from the user the starting frame of the original video (i.e., the first frame in the time domain to be produced as a responsive video) to the final frame (i.e., the last frame in the time domain to be produced as a responsive video). You can choose up to Additionally, in an embodiment, a time section may be designated by the user.

The control module 1500 is connected to a responsive image and can directly receive commands from the user to trigger a response of an object. For example, in the case of a device equipped with a touch screen, the control module 1500 provides a process for receiving a specific input operation from the user, and in the process, can receive a specific command along an object moving on the touch screen. there is.

Additionally, the control module 1500 can receive a user's selection of a command type to be connected to a responsive image and receive an operation that can replace the corresponding command type. For example, when creating a responsive video on a device with a touch screen and using a computer (including VR devices) that does not have a touch screen, the computer uses mouse operations instead of touch operations on the touch screen. By receiving it, you can create a responsive video.

The command for triggering the object's response may match or correspond to the movement of the object included in the frame section. The location or area where the command for triggering the object's reaction is set to be input may correspond to the area corresponding to the movement of the object within the frame included in the frame section.

The control module 1500 can apply a method of creating a virtual layer in the entire area or a specific area of each frame within a designated frame section of the original image in order to connect an object and a command to trigger the object's response. A virtual layer may refer to a layer that is overlaid on the frames that make up the original image and that can receive user input without being visually expressed on the screen.

The control module 1500 uses a command to trigger a response of an object by moving a specific area on the frame (i.e., the path on which the object moves within the frame section) (e.g., moving the mouse cursor through mouse manipulation or moving an object on a touch screen). In the case of a drag operation from point 1 to point 2, a virtual layer consisting of a specific number of detailed cells can be created on the frame.

Additionally, the control module 1500 may generate a virtual layer composed of a plurality of detailed cells corresponding to the frame section. The control module 1500 can calculate the number of frames included in the frame section, apply the number of frames in the frame section as the number of detailed cells, and sequentially match each frame in the frame section to each detailed cell. there is.

For example, when the control module 1500 wants to generate content so that n frames are variably played (i.e., manipulated playback) according to a command to trigger a response of an object, the control module 1500 selects a specific area as n detailed cells. It can be divided into

Thereafter, the control module 1500 may match each frame to each divided detailed cell, so that the matched frame is provided when a specific detailed cell is selected or designated. That is, when an object (e.g., a hand) moves in a specific direction and a virtual layer is created along the movement path of the object, the control module 1500 operates within the frame section in order from the detailed cell at the first point where the object starts to move. It can be matched to each frame.

Additionally, the control module 1500 may generate a plurality of detailed cells constituting the virtual layer with different sizes or spacing. If the speed at which an object moves changes during a frame section in the original video, if the virtual layer is divided into detail cells of the same size, the position of the object in the frame and the position of the detail cells may not match. Accordingly, the control module 1500 may vary the size or spacing of detailed cells to match the movement of the object. In other words, when playing video content, the speed changes at the same time interval to obtain frames of moving objects, so in fast-moving sections, the gap between object positions within successive frames is large, and in slow movements, the gap between object positions within successive frames is narrow. do. Therefore, the control module 1500 must create a plurality of detailed cells to match the object spacing within the frame so that the position of the user's input operation (command to trigger the object's response) and the object position within the frame can be matched. there is.

Additionally, the control module 1500 can determine the length of the virtual layer. In an embodiment, the control module 1500 can recognize the path on which the object moves by determining the location of each object (or a specific feature point of the object) within the frame section, and the control module 1500 determines the length including the path. A virtual layer can be formed. Additionally, the control module 1500 can determine the shape of the virtual layer and detailed cells.

Responsive video may be content that has been filmed and stored in advance, or it may be content that adds or synthesizes additional content to the original video. For example, a responsive video may include a video, still image, hologram, etc., and additional content may be played together when the original video is played, turning the original video into a responsive video.

Additionally, a responsive video may include multiple responsive videos, which may mean that the content changes or plays in a form corresponding to a specific input operation by a user (i.e., viewer) watching the content. You can. For example, it may mean an image that is played back as if the user's input operation corresponding to a specific movement of the captured original image is connected to the object in the image moving according to the user's operation.

In an embodiment, a compressed image refers to an image compressed into a minimum movement unit to implement a basic image as a responsive image. For example, if the basic image contains the same movement repeatedly, the compressed image deletes the repeated movement and leaves only one. Additionally, for example, if the base image includes both movement from the first position to the second position and movement from the second position to the first position, the compressed image is the base image moving from the first position to the second position. Leaving it alone, movement from the second position to the first position can play the remaining basic image in the reverse direction.

In addition, a multi-responsive video creation file is a content file created by compressing a plurality of basic images and can play various actions according to the user's manipulation, or can be implemented as a multi-responsive video by being played together with one or more basic images. This means metadata that can be used. In an embodiment, the control module 1500 may generate a responsive image without generating a compressed image even if duplication is allowed.

In an embodiment, the control module 1500 does not use compressed images and uses all repeated movements as is without deleting them, thereby generating or outputting a responsive image from a basic image (the basic image may be a responsive image). can do.

Hereinafter, a method for generating a multi-responsive video according to the present disclosure will be described, assuming that the content is an video.

The control module 1500 can acquire a basic image. The basic image may be an original image that includes the movement of an object to be implemented in a responsive manner according to the user's manipulation. Responsive video creators (e.g., content providers or individual users) can shoot videos containing actions they want to implement in a responsive manner.

In an embodiment, the control module 1500 may obtain images of multiple movements of the same object from the user and then generate them in the form of multiple responsive images. For example, when the object is the user's hand, the control module 1500 (190) can acquire a plurality of images of the user's index finger moving or bending in various directions while spread.

Additionally, if you want to create a responsive video that rotates your head up, down, left, right, or changes facial expressions in response to a specific manipulation, you can capture an image that includes all of the desired head movements and expressions.

In addition, if you want to create a responsive video in which a water balloon bursts or bounces up from the floor according to a manipulation input by the user, the user must drop a water balloon of the same color and size so that the first image of the popping water balloon and the water balloon are You can sequentially film a second video that bounces without exploding.

Additionally, after performing a specific movement, one basic image in which a different event occurs rather than a repetition of an existing movement can be obtained. That is, the control module 1500 can acquire an image in which a plurality of events occur with respect to an object as a basic image.

The control module 1500 can generate a compressed image based on the basic image. The compressed video may be responsive and include only the movement of the object to be implemented according to the user's manipulation. In embodiments, images that allow duplication may also be applied instead of compressed images.

The control module 1500 can receive multiple responsive image creation conditions for compressed images. The multi-responsive image creation condition may be a plurality of manipulation inputs corresponding to responses that can be generated from the compressed image.

The control module 1500 may generate a stack structure of the compressed image, and each extraction area (e.g., a first extraction area and a second extraction area) may include a plurality of stacks for different events. For example, a first stack represented by a solid line and a second stack represented by a dotted line may be included in each extraction area. The control module 1500 may be implemented by determining the stack to be executed among the first stack and the second stack based on the location where the first operation is input to each extraction area from the user.

Additionally, the first event and the second event in each extraction area may include overlapping pixels, and the control module 1500 may leave only one of the overlapping pixels among the stack for the first event and the stack for the second event. there is. Even if the compressed image contains only one data about an overlapping pixel within a specific extraction area, one of the first event and the second event depending on the user's next operation (e.g., change in the direction of movement of the touch operation or the intensity of applied pressure) The stack can be determined. Through this, the computer can create compressed images with minimal data.

Below, a user interface for creating a responsive image according to an embodiment of the present disclosure will be described. Responsive images may include the above-described multi-type responsive images and multi-dimensional responsive images.

Referring to FIG. 1, an operation pattern-based responsive video playback device (hereinafter referred to as 'playback device') 1000 according to an embodiment of the present disclosure includes an input module 1100, a storage module 1200, and a handler module. It may be configured to include (1300), a playback module (1400), and a control module (or processor) (1500).

First, the input module 1100 may be a display module that forms a layered structure or is integrated with the touch sensor, and may implement a touch screen. This touch screen can function as a user input unit that provides an input interface between the playback device 1000 and the user, and at the same time, can provide an output interface between the playback device 1000 and the user.

Specifically, when information is input through the touch screen, the control unit 1500 can control the operation of the playback device 1000 to correspond to the input information. At this time, the information may be specific data or a touch gesture. As an example, the playback device 1000 displays a virtual key, soft key, or visual key on the touch screen through software processing, or displays a virtual key, soft key, or visual key on a part other than the touch screen. A touch key may be further provided. Meanwhile, the virtual key or visual key can be displayed on the touch screen in various forms, for example, graphic, text, icon, video or these. It can be made up of a combination of .

Meanwhile, when the touch screen is capacitive, a proximity sensor (not shown) may be configured to detect the proximity of a conductive object through a change in the electric field according to the proximity of the object. In this case, the touch screen (or touch sensor) itself can be classified as a proximity sensor.

Meanwhile, for convenience of explanation, the act of approaching an object on the touch screen without touching it so that the object is recognized as being located on the touch screen is called "proximity touch," and the touch The act of actually touching an object on the screen is called “contact touch.” The position at which an object is close-touched on the touch screen means the position at which the object corresponds perpendicularly to the touch screen when the object is close-touched. The proximity sensor may detect a proximity touch and a proximity touch pattern (e.g., proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch location, proximity touch movement state, etc.). Meanwhile, the control unit 1500 processes information (data, touch gestures, etc.) corresponding to the proximity touch operation and proximity touch pattern detected through the proximity sensor, as described above, and further provides visual information corresponding to the processed information. It can be output on the touch screen. Furthermore, the control module 1500 can control the playback device 1000 so that different actions or information are processed depending on whether a touch to the same point on the touch screen is a proximity touch or a contact touch.

The touch sensor detects a touch (or touch input) applied to the touch screen using at least one of several touch methods, such as a resistive method, a capacitive method, an infrared method, an ultrasonic method, and a magnetic field method.

As an example, a touch sensor may be configured to convert changes in pressure applied to a specific part of the touch screen or capacitance occurring in a specific part into an electrical input signal. The touch sensor may be configured to detect the position, area, pressure at the time of touch, capacitance at the time of the touch, etc., at which the touch object that touches the touch screen is touched on the touch sensor. Here, the touch object is an object that applies a touch to the touch sensor, and may be, for example, a finger, a touch pen or stylus pen, or a pointer.

In this way, when there is a touch input to the touch sensor, the corresponding signal(s) are sent to the touch controller. A touch controller (not shown) processes the signal(s) and then transmits corresponding information to the control module 1500. As a result, the control module 1500 can determine which area of the touch screen has been touched. Here, the touch controller may be a separate component from the control module 1500 or may be the control module 1500 itself.

Meanwhile, the touch sensor and proximity sensor discussed above are used independently or in combination to provide short (or tap) touch, long touch, multi touch, and drag touch on the touch screen. ), flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, etc. Touch can be sensed.

Meanwhile, the storage module 1200 includes at least one process for playing a responsive video based on at least one operation pattern, as well as at least one process necessary to provide various services related to the responsive video or perform other operations. One more process can be saved. In addition, the storage module 1200 can store image frame information corresponding to each of at least one manipulation pattern.

Additionally, the storage module 1200 can store data supporting various functions of the playback device 1000. In addition, the storage module 1200 can store a plurality of application programs (application programs) running on the playback device 1000, data for the operation of the production device 1000, and commands. These applications At least some of the programs may exist for the basic functions of the playback device 1000. Meanwhile, the application program is stored in the storage module, installed on the playback device 1000, and played by the control module 1500. It may be driven to perform the operation (or function) of the device 1000.

In particular, the storage module 1200 stores at least one basic image data and at least one responsive image generated based on the at least one basic image data. At this time, various responsive image creation files may be included in the folder where the basic image data is stored, or corresponding responsive image creation files may be matched in the database. As a result, it is possible to implement a responsive video by playing the corresponding responsive video creation file when playing basic video data.

In addition, the storage module 1200 provides set information for a plurality of basic image data and/or a plurality of responsive images, and a selection order for a plurality of basic image data and/or a plurality of responsive images included in each set. Based playback pattern information may be stored. At this time, the playback pattern information includes playback patterns corresponding to the order in which a plurality of basic video data or a plurality of responsive videos can be selected.

Meanwhile, the handler module 1300 confirms an event corresponding to an input manipulation among at least one event, and generates and delivers input manipulation information including information about the input manipulation according to the confirmed event. At this time, the input manipulation information may include information about at least one of the number of manipulations, manipulation strength, movement length, manipulation speed, manipulation time length, manipulation direction, and manipulation position for a specific object in the responsive image. In addition, the at least one event represents at least one action that occurs within the responsive video, such as scene change (go to scene), play clip, adjust scene point, and sound play. It may include at least one of (play sound), add to combo, open web page, send web message, broadcast event, and show finale.

Meanwhile, the playback module 1400 determines the manipulation pattern corresponding to the user's input manipulation among at least one manipulation pattern based on the input manipulation information transmitted by the handler module 1300. Additionally, the playback module 1400 can perform a playback operation on a responsive video by generating video data using video frame information corresponding to the confirmed manipulation pattern.

Specifically, the playback module 1400 may include a first player module and a second player module.

First, the first player module generates the video frame information by extracting at least one frame corresponding to the first operation pattern among a plurality of video frames, and generates video data using the generated video frame information to input the input. Plays a responsive video according to the operation.

Additionally, when the input characteristics of the user's input manipulation are confirmed based on the input manipulation information, the second player module identifies a second manipulation pattern corresponding to the confirmed input characteristics among the plurality of video frames. Afterwards, the second player module extracts at least one frame corresponding to the second operation pattern to generate video frame information, and generates video data using the generated video frame information to create a responsive type according to the input manipulation. Play the video.

Here, the first operation pattern is an operation pattern corresponding to the user's input operation and corresponds to a basic operation pattern basically stored in response to the input operation. In addition, the second operation pattern is an operation pattern that is stored according to the input characteristics of each user. As the input characteristics are confirmed by the user's input operation, the second operation pattern corresponding to the user is confirmed, and from the time of confirmation, the first operation pattern is confirmed. Application of the operation pattern, that is, the basic operation pattern, is stopped and the second operation pattern is applied.

In other words, the at least one operation pattern includes a first operation pattern that is basically set for each of at least one event and a second operation pattern that is stored differently for each user by reflecting the input characteristics of each user.

The control module 1500 may control the overall operation of the playback device 1000 in addition to operations related to an application program for providing responsive video. The control module 1500 can provide or process appropriate information or functions to users by processing signals, data, information, etc. input or output through the components discussed above, or by running the application program.

First, when at least one basic image data including at least one object is input, and a specific object among the at least one objects is designated, the control module 1500 selects a playback section related to the designated specific object on the basic video data. Set . At this time, a specific object may be designated by selection by the provider of the basic image data, or a specific object for which a characteristic change is detected within the basic image data may be automatically designated. The characteristic change may include at least one of a motion change, a resolution change, and a sharpness change.

In addition, the control module 1500 generates image data consisting of a plurality of frames included in the set playback section, determines the operation pattern of the input operation according to the change in the state of the specific object, and then selects the specific object in the plurality of frames. Each touch area is set based on , and a responsive image is generated by applying the operation pattern of the input operation to each set touch area.

Here, at least one basic image data may be an original image including the movement of an object to be implemented in a responsive manner by a user's input manipulation. Here, the input operation is based on continuous gestures and may be a touch gesture, and the operation pattern of the touch gesture is drag, rotate, rubber, pinch, and spread. It can be configured based on any one of the following.

Specifically, when setting each touch area, the control module 1500 creates a virtual layer that combines on a plurality of frames included in the corresponding image data, divides the virtual layer into a plurality of detailed cells, and then divides the virtual layer into a plurality of detailed cells. In the frame, the frames corresponding to each detailed cell are mapped to each other. At this time, the control module 1500 recognizes feature points for a specific object in each of a plurality of frames, determines the location of each feature point, confirms the movement path of the specific object, and then creates a virtual layer to include the movement path. do. Additionally, when dividing a virtual layer, the control module 1500 projects the position of each feature point onto the virtual layer and creates a plurality of detailed cells by dividing the positions of each feature point.

If the control module 1500 determines that the position of each feature point moves from the first position to the second position, the control module 1500 generates a plurality of detailed cells and Determines the shape of the virtual layer.

Meanwhile, the control module 1500 may perform different controls or perform the same control depending on the type of object that touches the touch screen (or a touch key provided in addition to the touch screen). Whether to perform different controls or the same controls depending on the type of object may be determined depending on the current operating state of the playback device 1000 or the application program being executed.

In addition, the control module 1500 generates responsive image creation conditions for a plurality of input operations corresponding to reactions that can be generated from the image data, and applies the responsive image creation conditions to the image data to create a responsive image. Create. Here, the image data is data in a stack structure that includes information about the combination of images created by dividing or correcting the at least one basic image data, and the responsive image creation condition is coupled to a specific area of the image data. It may contain data containing information for responsive implementation.

Afterwards, the control module 1500 registers the generated responsive images on a platform for service provision, and may register a plurality of responsive images as one set according to the set information stored in the storage module 1200. . At least one set may be configured, and in this case, one responsive video may be included in multiple sets.

In addition, the control module 1500 extracts frames at preset time intervals from among all frames included in the playback section to form a plurality of frames, and sequentially arranges the plurality of frames in chronological order to generate image data. there is.

Meanwhile, the control module 1300 controls the input module 1100, storage module 1200, handler module 1300, and playback module 1400 described above to play the responsive video based on at least one process. Ensure that a series of operations are performed.

Specifically, when performing a playback operation for a responsive image, the control module 1500 performs a playback operation for a responsive image by applying a first operation pattern corresponding to an input operation, while performing the playback operation. When the user's input characteristics are confirmed according to the input manipulation information, the application of the first manipulation pattern is stopped, and the second manipulation pattern corresponding to the confirmed input characteristics is applied to perform a playback operation for the responsive video. do.

Meanwhile, the control module 1500 stores input manipulation information and image data according to the input manipulation (image data output according to the input manipulation or movement/movement of a specific object in the responsive video according to the input manipulation) to the pre-stored deep learning learning model. By entering a state change), you can check the input characteristics of the user's input operation.

In addition, the control module 1500 stores the second operation pattern reflecting the confirmed input characteristics separately for each user, so that when a user according to the input characteristics of the input operation is confirmed, the control module 1500 responds to the input operation of the confirmed user. Image data can be generated by extracting a frame based on a pre-stored second manipulation pattern.

Furthermore, when storing the second manipulation pattern, the control module 1500 may generate image data based on this, map it to the second manipulation pattern, and store it in advance. In other words, if only the second manipulation pattern is confirmed, the video data mapped to the second manipulation pattern can be loaded and played without the need to separately generate video data based on it.

In this case, when applying the second operation pattern corresponding to the confirmed input characteristic, the control module 1500 confirms that image frame information or image data corresponding to the second operation pattern has already been created (stored). , pre-generated video frame information or video data can be used as is.

Meanwhile, although not shown in FIG. 14, a communication module, a resource management module, a layout management module, etc. may be further provided, and other components may be added or some components may be excluded.

At this time, the communication module (not shown) transmits and receives at least one information or data to at least one user terminal. Here, at least one user terminal is a terminal of a user who wishes to receive various services related to responsive video, and can use various services related to responsive video through platforms, web pages, etc., and limits the method and means of providing them. I never do that.

In addition, the communication module may include one or more components that enable communication with other devices, for example, at least one of a wired communication module, a wireless communication module, a short-distance communication module, and a location information module. may include.

In addition, specific operations of the control module 1500 will be described below based on each drawing.

Figure 2 is a flowchart showing the operation of processing and playing video data according to the type of responsive video according to an embodiment of the present disclosure.

Referring to FIG. 2, the control module 1500 of the playback device 1000 displays a responsive image, detects the user's input operation for the responsive image (S2100), and when the user's input operation is detected, Among at least one event, the event corresponding to the input operation is checked (S2200).

Next, the control module 1500 generates and transmits input manipulation information including information about the input manipulation according to the event confirmed in step S2200 (S2300), and performs at least one manipulation based on the input manipulation information. Among the patterns, check the operation pattern corresponding to the input operation (S2400).

Next, the control module 1500 performs a playback operation on the responsive video by generating video data using video frame information corresponding to the manipulation pattern confirmed in step S2400 (S2500).

Referring to FIG. 3, when performing a playback operation for a responsive video, the control module 1500 of the playback device 1000 applies a first operation pattern corresponding to the input operation to play the responsive video. Perform the operation (S2510).

Next, the control module 1500 performs the playback operation and checks the input characteristics based on the input manipulation information generated in step S2300 (S2520), and creates a second manipulation pattern corresponding to the confirmed input characteristics. Check whether it exists (S2530).

If, as a result of confirmation in step S2530, it is confirmed that a second manipulation pattern corresponding to the input characteristic exists, the control module 1500 stops applying the first manipulation pattern for the current playback operation (S2540) .

Next, the control module 1500 performs a playback operation for the responsive video by applying the confirmed second operation pattern (S2550).

Meanwhile, although not shown in FIG. 3, when the control module 1500 determines that a second operation pattern corresponding to the input characteristic does not exist as a result of checking in step S2530, it is regarded as a new user and the input A second operation pattern corresponding to the user is additionally stored by reflecting the characteristics. That is, the second operation pattern stored in the database can be updated.

The above-mentioned program is C, C++, and It may include code encoded in a computer language such as JAVA or machine language. These codes may include functional codes related to functions that define the necessary functions for executing the methods, and related execution procedures necessary for the control module 1500 of the computer to execute the functions according to predetermined procedures. May contain control code. In addition, this code contains a memory reference code indicating which location (address address) of the computer's internal or external memory the additional information or media required for the computer's control module 1500 to execute the above functions should be referenced. More may be included. In addition, if the computer's control module 1500 needs to communicate with any other remote computer or server to execute the functions, the code can be sent to any other remote computer using the computer's communication module. It may further include communication-related codes on how to communicate with a server, etc., and what information or media should be transmitted and received during communication.

The storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., but are not limited thereto. That is, the program may be stored in various recording media on various servers that the computer can access or on various recording media on the user's computer. Additionally, the medium may be distributed to computer systems connected to a network, and computer-readable code may be stored in a distributed manner.

The steps of the method or algorithm described in connection with the embodiments of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. The software module may be RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which this disclosure pertains.

Above, embodiments of the present disclosure have been described with reference to the attached drawings, but those skilled in the art will understand that the present disclosure can be implemented in other specific forms without changing its technical idea or essential features. You will be able to understand it. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

Claims

An input module that displays responsive images and detects input operations from the user;

a storage module that stores at least one process for playing a responsive video based on at least one operation pattern and stores image frame information corresponding to each of the at least one operation pattern;

a handler module that identifies an event corresponding to the input operation among at least one event, and generates and transmits input operation information including information about the input operation according to the confirmed event;

Based on the input operation information, the operation pattern corresponding to the input operation among the at least one operation pattern is confirmed, and image data is generated using the image frame information corresponding to the confirmed operation pattern to display the responsive video. a playback module that performs playback operations; and

It includes a control module that performs a series of control operations for playing the responsive video based on the at least one process,

The at least one operation pattern includes a first operation pattern that is basically set for each of the at least one event and a second operation pattern that is stored differently for each user by reflecting the input characteristics of each user. to,

Responsive video playback device.
According to paragraph 1,

The control module is,

When performing a playback operation for the responsive video,

A playback operation for the responsive video is performed by applying a first operation pattern corresponding to the input operation, and if the input characteristic is confirmed by the input operation information while performing the playback operation, the first operation pattern is used. stop applying,

Characterized in performing a playback operation on the responsive video by applying a second operation pattern corresponding to the confirmed input characteristic,

Responsive video playback device.
According to paragraph 2,

The control module is,

Characterized by inputting the input manipulation information and the image data into a pre-stored deep learning learning model to confirm the input characteristics of each user, and storing the second manipulation pattern reflecting the confirmed input characteristics for each user. doing,

Responsive video playback device.
According to paragraph 3,

The input operation information is,

Characterized in that it includes information on at least one of the number of manipulations, manipulation intensity, movement length, manipulation speed, manipulation time length, manipulation direction, and manipulation position for a specific object in the responsive image.

Responsive video playback device.
According to paragraph 4,

The playback module is,

a first player module that generates video frame information by extracting at least one frame corresponding to the first manipulation pattern among a plurality of video frames, and plays the responsive video using the generated video frame information; and

When the input characteristic is confirmed by the input manipulation information, the second manipulation pattern corresponding to the confirmed input characteristic among the plurality of video frames is confirmed, and at least one frame corresponding to the second manipulation pattern is extracted Characterized in that it includes a second play module that generates the video frame information and plays the responsive video using the generated video frame information.

Responsive video playback device.
According to clause 5,

The control module is,

When applying the second operation pattern corresponding to the confirmed input characteristics,

When video frame information corresponding to the second operation pattern is previously stored, the previously generated video frame information is used,

Responsive video playback device.
In a manipulation pattern-based responsive video playback method performed by a device,

Displaying a responsive image and detecting a user's input operation on the responsive image;

When the input manipulation is detected, checking an event corresponding to the input manipulation among at least one event;

generating and delivering input operation information including information about the input operation according to the confirmed event;

Confirming an operation pattern corresponding to the input operation among at least one operation pattern based on the input operation information; and

A step of performing a playback operation on the responsive video by generating video data using video frame information corresponding to the confirmed manipulation pattern,

The at least one operation pattern includes a first operation pattern that is basically set for each of the at least one event and a second operation pattern that is stored differently for each user by reflecting the input characteristics of each user. to,

How to play responsive video.
In clause 7,

The step of performing a playback operation on the responsive video is,

A playback operation for the responsive video is performed by applying a first operation pattern corresponding to the input operation, and if the input characteristic is confirmed by the input operation information while performing the playback operation, the first operation pattern is used. stop applying,

Characterized in performing a playback operation on the responsive video by applying a second operation pattern corresponding to the confirmed input characteristic,

How to play responsive video.
According to clause 8,

The step of performing a playback operation on the responsive video is,

Characterized by inputting the input manipulation information and the image data into a pre-stored deep learning learning model to confirm the input characteristics of each user, and storing the second manipulation pattern reflecting the confirmed input characteristics for each user. doing,

How to play responsive video.
According to clause 9,

The input operation information is,

Characterized in that it includes information on at least one of the number of manipulations, manipulation intensity, movement length, manipulation speed, manipulation time length, manipulation direction, and manipulation position for a specific object in the responsive image.

How to play responsive video.