KR101888244B1 - Method of playing multimedia contents by integrating multi-devices and apparatus performing the same - Google Patents

Abstract

A method and apparatus for reproducing multimedia contents using a plurality of devices are disclosed. A method of reproducing contents according to an exemplary embodiment includes a step of assigning a type of multimedia contents to be reproduced using a plurality of devices to each of the plurality of devices, .

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method and apparatus for reproducing multimedia contents by combining a plurality of devices,

The embodiments described below relate to a method and apparatus for reproducing multimedia contents using a plurality of devices.

Smart devices such as smart phones and tablet PCs have computational abilities that are as high as those of desktop computers, and they are widely used for content reproduction of moving images. However, as one individual owns not only a smartphone but also a tablet PC, a single-person N-terminal environment is equipped. In this environment, a technique of reproducing multimedia contents using a plurality of devices is utilized.

Embodiments may provide a technique for a user to play multimedia content using a touch-based interface by at least one of a plurality of devices.

In addition, the embodiments can provide a technique of synchronizing a plurality of devices, monitoring the context of the user and device information, and adaptively controlling the content so that the content can be reproduced stably.

A method of reproducing contents according to an exemplary embodiment includes a step of assigning a type of multimedia contents to be reproduced using a plurality of devices to each of the plurality of devices, Step < / RTI >

The setting may include rotating or moving the plurality of devices to set a layout type.

The method may further include adjusting a screen size of the multimedia content.

The adjusting step may include adjusting the screen size to be larger or smaller than the size of the entire display of the plurality of devices.

The method may further include receiving multimedia content rendered based on context information and device information of a user, wherein the context information includes at least one of a type of the multimedia content requested by the user, The device information including at least one of a device identification, a display specification, a rendering capability, a networking capability, and a device type.

The rendered multimedia contents may be contents including at least one of images at different resolutions of the original multimedia contents, video at different resolutions, and audio of another channel or different sound quality.

A content reproduction apparatus according to an exemplary embodiment includes a first interface for assigning a type of multimedia content to be reproduced using a plurality of devices to each of the plurality of devices in response to a first user input, And an input interface including a second interface for setting a layout type of the plurality of devices.

The first user input may be an input in the form of a drag and drop and the second user input may be a form of a rotation gesture or an input in the form of a drag and drop.

The input interface may further include a third interface for adjusting a screen size of the multimedia content in response to a third user input.

The screen size may be adjusted to be larger or smaller than the size of the entire display of the plurality of devices according to the third user input.

The apparatus further comprises a transceiver for receiving the rendered multimedia contents based on the context information of the user and the device information, and the context information includes at least one of the type and the arrangement type of the multimedia contents requested by the user, The device information may include at least one of a device ID, a display specification, a rendering capability, a networking capability, and a device type.

The rendered multimedia contents may be contents including at least one of images at different resolutions of the original multimedia contents, video at different resolutions, and audio of another channel or different sound quality.

The plurality of devices may include the content reproduction apparatus, and the content reproduction apparatus may be a control device for controlling the plurality of devices.

1 is a schematic block diagram of a content reproduction system according to an embodiment.
2 is a schematic block diagram of the server shown in FIG.
3 is a schematic block diagram of the content reproduction apparatus shown in FIG.
Fig. 4 shows an example of an algorithm using the input interface of the content reproduction apparatus shown in Fig. 3 for reproducing the content.
Figs. 5A to 5C show embodiments using each interface constituting the input interface of Fig.
6A and 6B show embodiments in which different types of multimedia contents are reproduced according to context information of different users.
7A to 7D show embodiments for reproducing multimedia contents using a plurality of devices.
8 shows an example of an algorithm by which a content reproduction system reproduces multimedia contents.

It is to be understood that the specific structural or functional descriptions of embodiments of the present invention disclosed herein are presented for the purpose of describing embodiments only in accordance with the concepts of the present invention, May be embodied in various forms and are not limited to the embodiments described herein.

The embodiments according to the concept of the present invention can make various changes and can take various forms, so that the embodiments are illustrated in the drawings and described in detail herein. It should be understood, however, that it is not intended to limit the embodiments according to the concepts of the present invention to the particular forms disclosed, but includes all modifications, equivalents, or alternatives falling within the spirit and scope of the invention.

The terms first, second, or the like may be used to describe various elements, but the elements should not be limited by the terms. The terms may be named for the purpose of distinguishing one element from another, for example without departing from the scope of the right according to the concept of the present invention, the first element being referred to as the second element, Similarly, the second component may also be referred to as the first component.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms " comprises ", or " having ", or the like, specify that there is a stated feature, number, step, operation, , Steps, operations, components, parts, or combinations thereof, as a matter of principle.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these embodiments. Like reference symbols in the drawings denote like elements.

A module in this specification may mean hardware capable of performing the functions and operations according to the respective names described in this specification and may mean computer program codes capable of performing specific functions and operations , Or an electronic recording medium, e.g., a processor or a microprocessor, equipped with computer program code capable of performing certain functions and operations.

In other words, a module may mean a functional and / or structural combination of hardware for carrying out the technical idea of the present invention and / or software for driving the hardware.

1 is a schematic block diagram of a content reproduction system according to one embodiment, and FIG. 2 is a schematic block diagram of a server shown in FIG.

1 and 2, the content reproduction system 10 may include a server 100 and content reproduction apparatuses 200-1, 200-2, ..., 200-n.

The server 100 may include a memory 110 and a controller 120.

The controller 120 may log the index number of the last frame transmitted to the plurality of devices in order to synchronize the plurality of devices. The plurality of devices may transmit frame indexes to the server 120 for synchronization. Each thread of the system 10 independently adjusts the content stream and the system 10 can signal the thread exhibiting asynchronous behavior when synchronization impairments are detected.

The controller 120 may include a contents manager 121, a context manager 123, a device manager 125, and a communication module 127.

The content manager 121 may render the multimedia content using the information of the context manager 123 and the information of the device manager 125. For example, the content manager 121 may render the multimedia content of the video type into a video having a different resolution. The content manager 121 can render the multimedia content of the image type into an image having a different resolution. The content manager 121 may render the multimedia contents of the audio type into audio having a different channel or different sound quality.

Also, the content manager 121 can transmit the rendered multimedia contents to the memory 110. That is, the memory 110 may receive and store the rendered multimedia contents. The memory 110 may be located inside or outside the content manager 121.

The multimedia content may include at least one of video, image, audio, and text information. The audio may include music files, audio files, and dubbing files, and may include other versions according to the standard language used by country. The textual information may include captions and descriptions (e.g., subtitles for movies, descriptions of music) for multimedia content. Subtitles, and descriptions may include other versions according to the standard language used by country.

The content manager 121 may adaptively transmit multimedia content to a plurality of devices through the communication module 127. [ For example, initially, the server 100 may transmit multimedia content set to the same resolution as the device information to the content playback apparatuses 200-1, 200-2, ..., 200-n. However, if the frame per second (FPS) is lower than a preset reference value, the server 100 may render the multimedia content having a lower resolution and transmit the multimedia content to the content playback apparatuses 200-1, 200-2, ..., 200-n have.

The context manager 123 may include context information of the user. For example, the context information of a user includes an ID of a user, an ID of a plurality of devices possessed by the user, a service history, a preference service type, a user's currently activated devices, The context information of the user may be information about the user holding each of the plurality of devices or information of the user holding the control device. The control device is at least one apparatus included in the content reproduction apparatuses 200-1, 200-2, ..., 200-n and includes a plurality of content reproduction apparatuses 200-1, 200-2, ..., 200- Lt; / RTI > devices.

The context information of the user may include at least one of a type of multimedia contents requested by a user and a layout type of a plurality of devices. The type of multimedia content may include at least one of video, image, audio, and text information. That is, the user can request at least one of the multimedia contents of the video, image, audio, and text information to the server 100.

The layout of the plurality of devices may include a schematic diagram in which at least one of the plurality of devices is arranged. That is, the content reproducing apparatuses 200-1, 200-2, ..., 200-n may be one device or a plurality of devices.

The device manager 125 may include device information.

The device information may be information of a plurality of devices or information of a control device. The device information may include at least one of a device identification, a display specification, a rendering capability, a networking capability, and a device type.

The device specification may be information about the density, size, and resolution of the pixels that the device can display. The controller 120 can accurately calculate and optimize the screen size of the multimedia contents to be reproduced by each of the content reproducing apparatuses 200-1, 200-2, ..., 200-n using the device specification.

The rendering performance may be information regarding the capacity of the graphics to be processed. Rendering performance may be information regarding the refresh rate of the frame image per second upon content reproduction.

Networking performance may be information that measures the time at which a certain size of data is transmitted under the current network environment.

The device type may be one of a stationary device (e.g., including a fixed display such as a desktop personal computer, a monitor, a television, or the like), or a mobile device.

The context manager 123 and the device manager 125 can monitor the context information and the device information of the user of the content reproducing apparatuses 200-1, 200-2, ..., 200-n. When the context information and the device information of the user are changed, the context manager 123 and the device manager 125 can receive the changed context information and device information. The content manager 121 may render the multimedia content based on the information of the text manager 123 and the information of the device manager 125.

The communication module 127 may be a module for managing communication and data transmission between the server 100 and a user. The communication module 127 may be located inside or outside the controller 120. The communication module 127 can communicate with the content reproduction apparatuses 200-1, 200-2, ..., 200-n through the communication network 50. [ That is, the server 100 may transmit the multimedia contents to the content reproducing apparatuses 200-1, 200-2, ..., 200-n using the communication module 127. [

Also, the communication module 127 can receive the context information and the device information of the user from the content reproduction apparatuses 200-1, 200-2, ..., 200-n. The communication module 127 may transmit the context information of the user to the context manager 123 and may transmit the device information to the device manager 125. [ That is, the context manager 123 can receive and store the context information from the communication module 127, and the device manager 125 can receive and store the device information from the communication module 127.

The content reproducing apparatuses 200-1, 200-2, ..., 200-n have a display screen and can be implemented as a desktop PC, a monitor, a TV, or a mobile device.

The mobile device may be a laptop computer, a mobile phone, a smart phone, a tablet PC, a mobile internet device (MID), a personal digital assistant (PDA), an enterprise digital assistant (EDA) A digital still camera, a digital video camera, a portable multimedia player (PMP), a personal navigation device or a portable navigation device (PND), a handheld game console, an e-book e-book, or a smart device.

Smart devices can be implemented as smart watch watches or smart bands. That is, the content reproducing apparatuses 200-1, 200-2, ... 200-n may be a wearable apparatus that the user can wear or is suitable for wearing.

The content reproduction apparatuses 200-1, 200-2, ..., 200-n can communicate with the server 100 through the communication network 50. [ For example, the communication network 50 may be an Internet communication network, an intranet, a local area network (LAN), a wireless LAN, or an IEEE 802.11 based Wi-Fi.

FIG. 3 is a schematic block diagram of the content reproduction apparatus shown in FIG. 1, and FIG. 4 shows an example of an algorithm using the input interface of the content reproduction apparatus shown in FIG. 3 for reproducing the content.

5A to 5C show embodiments using respective interfaces constituting the input interface of Fig. 3, Figs. 6A to 6B show embodiments in which types of different multimedia contents are reproduced according to context information of different users , FIGS. 7A to 7D show embodiments for reproducing multimedia contents using a plurality of devices.

3 shows only the content reproducing apparatus 200-1 of one of the content reproducing apparatuses 200-1, 200-2, ... 200-n, the other content reproducing apparatuses 200-1, 200-2, ... 200-n) may have a configuration as shown in FIG. In other words, at least one of the content reproducing apparatuses 200-1, 200-2, ..., 200-n may have the configuration as shown in FIG.

3 to 4, the content reproduction apparatus 200-1 may include an input interface 210, a controller 220, and a communication module 230. [

The input interface 210 may include a first interface 211, a second interface 213, and a third interface 215.

In response to the first user input, the first interface 211 may provide an environment for allocating a type of multimedia content to be played back to each of the plurality of devices using the plurality of devices (S410). The first user input may be an input in the form of a drag and drop. That is, the user can assign a content type to each of a plurality of devices by dragging and dropping.

In response to the second user input, the second interface 213 may provide an environment for setting the layout of the plurality of devices (S420). The second user input may be in the form of a rotation gesture or in the form of a drag and drop. That is, the user can set the arrangement type of the plurality of devices by rotating each of the plurality of devices using the rotation gesture, or by dragging and dropping them.

In response to the third user input, the third interface 215 may provide an environment for adjusting the screen size of the multimedia contents (S430). The third user input may be an input in the form of a zoom-in and zoom-out gesture. That is, the user can adjust the screen size of the multimedia contents using the zoom-in and zoom-out gesture. The user can adjust the screen size of the multimedia contents to be larger or smaller than the entire display size of the plurality of devices.

The process of assigning a multimedia content type to be reproduced by a user to each of a plurality of devices using the content reproduction apparatus 200-1 is as shown in FIG. 5A. When a user uses the content reproduction apparatus 200-1 The process of determining the arrangement type of the plurality of devices is as shown in FIG. 5B, and the process of adjusting the screen size of the multimedia contents by the user using the content reproduction apparatus 200-1 may be as shown in FIG. 5C have.

The controller 220 may acquire the allocation information and the layout type of the plurality of devices and transmit them to the server 100. The server 100 may transmit the rendered multimedia contents to the content reproduction apparatuses 200-1, 200-2, ..., 200-n based on the allocation information and the arrangement type of the plurality of devices.

The controller 220 may obtain the context information and the device information of the user. The user's context information can be directly input by the user using the input interface. The device information is a device-specific specification, and the controller 220 can acquire it from the device.

The controller 220 may transmit the context information and the device information of the user to the server 100 through the communication module 230. [ That is, the server 100 may render the multimedia contents based on the received context information and device information of the user.

The controller 220 can reproduce the multimedia contents received from the server 100 based on the user's context information and device information.

For example, the user can request a Chinese version of the text file to the content reproduction apparatus 200-1. In response to the request of the user, the controller 220 transmits the Chinese version of the text file to the user . As another example, the user can request an English version of the text file to the content reproduction apparatus 200-2, which responds to the user's request by sending an English version of the text file to the user . That is, the user may request the content reproduction apparatus 200-1 to use a different version of the text file according to the standard language used for each country.

In addition, a user can reproduce multimedia contents using a single device, or arrange a plurality of devices to operate as a single display. That is, the user can assign the multimedia contents of the video type to each of the plurality of devices using the first interface 211, and determine the arrangement type of the plurality of devices using the second interface 213. The controller 220 may provide the multimedia contents to the user in response to the user's request, as shown in Figs. 7A to 7D.

A plurality of devices can simultaneously play at least one multimedia content. For example, at least one of the plurality of devices can reproduce audio simultaneously while reproducing video, or simultaneously reproduce a text file while playing a video. That is, the user may assign at least one of the plurality of devices using the first interface 213 to reproduce at least one of the different types of media media contents.

The controller 220 may monitor the user's context information and device information. At this time, if the user's context information and device information are changed, the controller 220 can cause the communication module 230 to transmit the changed context information and device information to the server 100. [ That is, the server 100 receives the changed context information and the device information, and can transmit the rendered multimedia contents based on the changed context information and the device information.

The communication module 230 can transmit the context information and the device information of the user to the server 100. [ In addition, the communication module 230 can receive the rendered multimedia contents from the server 100 based on the user's context information and device information.

In addition, the user using the content reproduction apparatuses 200-1, 200-2, ..., 200-n may be a single user or a multiple user. The plurality of devices may be fixed or mobile devices. That is, the content reproducing apparatuses 200-1, 200-2, ..., 200-n can reproduce other multimedia contents according to the conditions as shown in Table 1. [

In the first situation, there may be a situation where a single user has a plurality of mobile devices and uses the content reproduction device. Considering the mobility of the mobile devices, the user can change the arrangement of the mobile devices without difficulty using the second interface 213. [ A user can integrate a plurality of mobile devices and set them as one display area.

In a second situation, there may be a situation where a single user uses a plurality of mobile devices and a fixed device. In this case, as in the first situation, a plurality of devices can be integrated into one display area, but the application may be difficult due to the immobility of the fixing device. Usually, since the fixing device includes a relatively large display area, it may be difficult for the user to focus attention on both the mobile device and the fixing device. Thus, the user can set the fixed device as the main device and allocate using the first interface to reproduce main information (e.g., video). The user can assign using the first interface to allow the mobile device to play additional information (e.g., audio, image, text file). For example, when a user has a smart TV and a tablet PC, the user can use the first interface to assign a smart TV to play a movie and a tablet PC to play a movie introduction or a movie subtitle.

In a third situation, there may be cases where a plurality of users use a plurality of mobile devices. Also in this case, when a plurality of users have matching context information, a plurality of devices can be integrated into one display area as in the first situation. When a plurality of users do not have matching context information, the same situation as the first situation can be used by matching the context information of a plurality of users.

In a fourth situation, there may be a situation where a plurality of users use a plurality of mobile devices and a fixed device. In this case, each of the plurality of users can have their own mobile device. The user can integrate the mobile devices into one display area except his / her own mobile device. For example, when two users have one smart TV and two tablet PCs, two users can watch a movie by matching their gaze to a smart TV and have each tablet PC post a movie subtitle in their own language You can rewind and replay the movie individually or individually. That is, multimedia content that is personalized in the context of a plurality of users can be played back. When two users whose context information is Chinese and English version text files play a movie on a smart TV and a tablet PC, the screen of each user's tablet PC may be as shown in FIGS. 6A to 6B.

8 shows an example of an algorithm by which a content reproduction system reproduces multimedia contents.

Referring to FIG. 8, a user may create an account for storing his / her information in the server 100 and register a plurality of devices to be used by the user (S810 and S813). A plurality of devices may be a single device.

The server 100 may determine the context information of the user (S815). The server 100 can automatically detect the user's context in real time.

The server 100 may obtain device information from devices and determine device information (S817). For example, the control device can acquire the context information and device information of the user from a plurality of connectable devices and transmit the information to the server 100.

A plurality of devices may individually transmit the user's context information and device information to the server 100. [

In addition, the server 100 can directly obtain the context information and device information of the user. For example, when the camera acquires a state in which a plurality of devices are arranged as an image and transmits the acquired image to the server 100, the server 100 analyzes the image to determine display specifications, rendering capabilities, And the like can be obtained.

The server 100 distributes the multimedia contents to a plurality of devices based on the determined user context information and device information (S819), and the plurality of devices can receive and reproduce the multimedia contents (S821).

When the context information of the user is changed, the server 100 detects it and newly determines the context information of the user (S823, S815).

If the context of the user has not been changed, the server 100 monitors whether the device information has been changed (S825). When the device information is changed, the server 100 detects it and newly determines the device information (S817).

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (13)

Allocating a type of multimedia contents to be reproduced by a content reproduction apparatus to each of the plurality of devices using a plurality of devices;
Obtaining a state in which the plurality of devices are arranged from the camera by the content reproduction apparatus;
Setting a layout type by rotating or moving the plurality of devices on the display based on the arrangement state of the content reproduction apparatus; And
The content reproduction apparatus receives and renders multimedia content rendered on the basis of context information and device information of a user from a server
Lt; / RTI >
Wherein the type of the multimedia content includes video and text information,
Wherein the context information includes at least one of a type and a layout type of the multimedia content,
Wherein the device information comprises at least one of a device identification, a display specification, a rendering capability, a networking capability, and a device type.
delete The method according to claim 1,
Adjusting a screen size of the multimedia contents
And reproducing the content.
The method of claim 3,
Wherein the adjusting comprises:
Adjusting the screen size to be larger or smaller than the size of the entire display of the plurality of devices,
And reproducing the content.
delete The method according to claim 1,
The rendered multimedia content may include:
A content including at least one of images at different resolutions of original multimedia contents, video at different resolutions, and audio of another channel or different sound quality
/ RTI >
A first interface for assigning to each of the plurality of devices a type of multimedia content to be played using a plurality of devices in response to a first user input; And
A second interface for setting a layout type by rotating or moving the plurality of devices on the display based on a state in which the plurality of devices acquired from the camera are arranged in response to a second user input,
An input interface; And
A controller for receiving multimedia contents rendered on the basis of the context information of the user and the device information from the server and reproducing the multimedia contents,
Lt; / RTI >
Wherein the type of the multimedia content includes video and text information,
Wherein the context information includes at least one of a type and a layout type of the multimedia content,
Wherein the device information includes at least one of a device ID, a display specification, a rendering capability, a networking capability, and a device type.
8. The method of claim 7,
Wherein the first user input is an input in the form of a drag and drop,
The second user input may be in the form of a rotation gesture or an input in the form of a drag and drop
Content playback apparatus.
8. The method of claim 7,
Wherein the input interface comprises:
A third interface for adjusting a screen size of the multimedia contents in response to a third user input,
Further comprising:
10. The method of claim 9,
Wherein the screen size is adjusted to be larger or smaller than the size of the entire display of the plurality of devices according to the third user input
Content playback apparatus.
delete 8. The method of claim 7,
The rendered multimedia content may include:
A content including at least one of images at different resolutions of original multimedia contents, video at different resolutions, and audio of another channel or different sound quality
Content playback apparatus.
8. The method of claim 7,
Wherein the plurality of devices include the content reproduction apparatus,
Wherein the content reproduction apparatus is a control device for controlling the plurality of devices
Content playback apparatus.

Images