KR20240045045A - Display device and operating method for the same - Google Patents

Abstract

A display device according to an embodiment includes a communication unit, a display, a memory storing one or more instructions, and at least one processor executing one or more instructions, wherein the at least one processor executes one or more instructions, 1 Obtains the first user image captured by the user, controls the communication unit to transmit the capability information of the display device to the server, and receives information about the configuration method of the group service determined based on the capability information from the server. Receive and, based on the received information, obtain a first avatar image and a second avatar image corresponding to another display device user, and set the display to display a group image including the first avatar image and the second avatar image. You can control it.

Description

Display device and operating method thereof {Display device and operating method for the same}

Various embodiments relate to a display device capable of displaying a user image as an avatar image in a non-face-to-face group service and a method of operating the same.

Recently, non-face-to-face services have been increasing in line with the untact era, and non-face-to-face group services that allow multiple users to participate remotely are increasing. In particular, as interest in health increases, home fitness services that guide people to exercise in their personal space are being actively launched.

In the case of existing home fitness services, the method was to watch and follow the trainer's various guided exercise videos, or to conduct group fitness using a video conference system.

When doing group fitness using a video conferencing system, the user's video captured through the camera may be shared with other users, which may cause problems in protecting personal privacy. In particular, when exercising with strangers, users may not want their appearance (e.g., face, workout clothes, etc.) shared with other users.

Therefore, a method is needed to provide a non-face-to-face group service in which multiple users participate together while protecting the privacy of individual users.

A display device according to an embodiment may include a communication unit, a memory storing one or more display instructions, and at least one processor executing the one or more instructions.

The at least one processor may obtain a first user image captured by the first user by executing the one or more instructions.

The at least one processor may control the communication unit to transmit capability information of the display device to a server by executing the one or more instructions.

The at least one processor may receive information about a group service configuration method determined based on the capability information from the server by executing the one or more instructions.

The at least one processor may acquire the first avatar image and a second avatar image corresponding to another display device user based on the received information by executing the one or more instructions.

The at least one processor may control the display to display a group image including the first avatar image and the second avatar image by executing the one or more instructions.

A method of operating a display device according to an embodiment may include acquiring a first user image that captures a first user.

A method of operating a display device according to an embodiment may include transmitting capability information of the display device to a server.

A method of operating a display device according to an embodiment may include receiving information about a configuration method of a group service determined based on the capability information from the server.

A method of operating a display device according to an embodiment may include obtaining the first avatar image and a second avatar image corresponding to another display device user based on the received information.

A method of operating a display device according to an embodiment may include displaying a group image including the first avatar image and the second avatar image.

1 is a diagram illustrating a group service provision system according to an embodiment.
Figure 2 is a flowchart showing a method by which a server determines a group service configuration method according to an embodiment.
Figure 3 is a flowchart showing a method by which a display device determines a service configuration method according to an embodiment.
FIG. 4 is a flowchart showing a method of operating a plurality of display devices when a group service configuration method according to an embodiment is determined on a device basis.
FIG. 5 is a diagram illustrating operations of a plurality of display devices when the configuration of a group service according to an embodiment is device-based.
Figure 6 is a diagram illustrating an example of a user interface screen for setting avatar information according to an embodiment.
FIG. 7 is a flowchart illustrating a method of operating a plurality of display devices when a group service configuration method according to an embodiment is determined on a device basis.
FIG. 8 is a diagram illustrating operations of a plurality of display devices when the configuration of a group service according to an embodiment is device-based.
FIG. 9 is a flowchart illustrating a method of operating a plurality of display devices and a server when a group service configuration method according to an embodiment is determined to be server-based.
Figure 10 is a flowchart showing a method of operating a plurality of display devices and a server when a group service configuration method according to an embodiment is determined to be server-based.
Figure 11 is a block diagram showing the configuration of a display device according to an embodiment.
Figure 12 is a block diagram showing the configuration of a display device according to an embodiment.
Figure 13 is a block diagram showing the configuration of a server according to an embodiment.

The terms used in this specification will be briefly explained, and the present invention will be described in detail.

The terms used in the present invention are general terms that are currently widely used as much as possible while considering the function in the present invention, but this may vary depending on the intention or precedent of a person working in the art, the emergence of new technology, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the relevant invention. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than simply the name of the term.

When it is said that a part "includes" a certain element throughout the specification, this means that, unless specifically stated to the contrary, it does not exclude other elements but may further include other elements. In addition, terms such as "... unit" and "module" used in the specification refer to a unit that processes at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software. .

Below, with reference to the attached drawings, embodiments will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

In embodiments herein, the term “user” refers to a person who controls a system, function, or operation, and may include a developer, administrator, or installer.

Additionally, in the embodiments of the present specification, 'image' or 'picture' may refer to a still image, a moving image composed of a plurality of consecutive still images (or frames), or a video.

1 is a diagram illustrating a group service provision system according to an embodiment.

Referring to FIG. 1, a group service providing system according to an embodiment may include a server 200 and a plurality of display devices 101, 102, and 103. A plurality of display devices 101, 102, and 103 according to one embodiment may be devices connected to a group service. A group service according to one embodiment may be a service that provides group activities in which multiple users can participate together non-face-to-face.

A plurality of display devices 101, 102, and 103 connected to the group service may share user images with each other. Each of the plurality of display devices 101, 102, and 103 may display one group image including user images of the plurality of display devices 101, 102, and 103.

Each of the plurality of display devices 101, 102, and 103 according to an embodiment is a device for processing image signals and displaying images and may include a display.

Each of the plurality of display devices 101, 102, and 103 according to an embodiment includes a TV, a smart monitor, a mobile phone, a smartphone, a tablet PC, a digital camera, a camcorder, a laptop computer, a desktop, E-book terminals, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), navigation, MP3 players, DVD (Digital Video Disk) players, wearable devices, video walls, It can be implemented in various forms such as digital signage, digital information display (DID), projector display, refrigerator, washing machine, etc. Additionally, the display device 1500 may be a fixed electronic device placed in a fixed location or a mobile electronic device in a portable form, and may be a digital broadcast receiver capable of receiving digital broadcasting. However, it is not limited to this.

As shown in FIG. 1, when the group service is a service that provides group fitness, the plurality of display devices 101, 102, and 103 each capture a user's exercise image and display the exercise images of the plurality of users into one. It can be provided as a group video. Accordingly, users of a plurality of display devices can perform home fitness non-face-to-face while sharing not only their own exercise videos but also other users' exercise videos.

At this time, the user's exercise video may be an image of an avatar moving, created based on an image of an actual user, rather than an image of the actual user.

When the exercise videos of a plurality of users included in the group fitness screen are displayed as actual images of the users, there may be cases where some users do not want to show their faces, exercise clothes, etc. to other users. Therefore, individual privacy can be protected by displaying the user's exercise video displayed on the group fitness screen as an avatar image.

The avatar image according to one embodiment may be an image that moves like an actual user based on a captured image of the user. The server 200 or a plurality of display devices 101, 102, and 103 according to an embodiment detects an object (e.g., a user) in an image taken of a user, obtains tracking information of the detected object, and , Based on tracking information, an avatar image can be created.

Meanwhile, the server 200 according to one embodiment may determine whether to configure the group service based on the server or device based on the capacity information of the plurality of display devices 101, 102, and 103. there is. For example, the server 200 determines whether the operation of generating an avatar image will be performed on the server 200 (server-based service configuration) or on each of the display devices 101, 102, and 103 (device-based service configuration). configuration) can be determined.

If it is decided to configure the service based on a server, each of the plurality of display devices 101, 102, and 103 may transmit a captured image of a user (user image) or information about the user image to the server 200.

The server 200 according to an embodiment may generate an avatar image corresponding to the user image based on the user image or information about the user image received from each of the plurality of display devices 101, 102, and 103. . The server 200 may generate one group image including avatar images corresponding to users of a plurality of display devices and transmit it to the plurality of display devices 101, 102, and 103.

The plurality of display devices 101, 102, and 103 may display one group image including avatar images corresponding to a plurality of users received from the server 200.

Alternatively, when it is decided to configure the service based on the device, each of the plurality of display devices 101, 102, and 103 displays the user image from each of the plurality of display devices 101, 102, and 103 without going through the server 200. Based on this, an avatar image corresponding to the user image can be generated. Additionally, each of the plurality of display devices 101, 102, and 103 may transmit the generated avatar image to other display devices. Each of the plurality of display devices 101, 102, and 103 may generate and display one group image including an avatar image generated by the display device and an avatar image received from other display devices. However, it is not limited to this.

A method in which a display device according to an embodiment acquires one group image including an avatar image of a display device user and an avatar image of another display device user will be described in detail with reference to the drawings below.

Figure 2 is a flowchart showing a method by which a server determines a group service configuration method according to an embodiment.

The display device 100 according to the embodiment of FIG. 2 may be one of the plurality of display devices 101, 102, and 103 of FIG. 1.

Referring to FIG. 2, the display device 100 according to one embodiment may obtain device capability information (S210). For example, device capacity information includes type information about at least one processor included in the display device 100, capacity information, information about the number of cores, performance information, and type of memory included in the display device 100. It may include at least one of information, capacity information, and performance information.

The display device 100 may previously store device capability information. When the display device 100 receives a request for device capability information from the server 200, the display device 100 may transmit the previously stored device capability information to the server 200 (S220).

The server 200 may receive device capability information from the display device 100 and determine a group service configuration method based on the received device capability information (S230).

For example, the server 200 may store policy information (service decision policy information) for determining a service configuration method. Service decision policy information includes device specifications (e.g., processor type information, capacity information, information on the number of cores, performance information, memory) required to perform operations included in group services such as creating an avatar on the device. It may include standards for type information, capacity information, performance information, etc.). However, it is not limited to this.

Additionally, service decision policy information may change depending on the number of concurrent users connected to the server or network conditions. Additionally, as the functions provided by the group service change, service decision policy information may also change. However, it is not limited to this.

The server 200 may determine whether to configure the group service based on device or server based on device capability information and service decision policy information.

In addition, the server 200 may determine the configuration method of the group service by considering the capacity information of other display devices connected to the group service. For example, the device specifications of some display devices connected to the group service meet the standards required to perform operations included in the group service, such as creating an avatar on the device, but the device specifications of the remaining display devices do not meet the above standards. If not satisfied, the server 200 may determine the configuration method of the group service based on the server to suit low-specification display devices. However, it is not limited to this.

The server 200 may transmit information about the determined service configuration method to the display device 100 (S240).

The display device 100 and the server 200 according to one embodiment may perform operations according to the determined service configuration method (S250, S260).

For example, when the configuration of the group service is device-based, the display device 100 according to one embodiment generates a first avatar image for the user image captured, and uses the generated first avatar image to be used by another user. It can be transmitted to a display device. Additionally, a second avatar image for a user of another display device may be received from another display device. The display device 100 may generate and display a group image including a first avatar image and a second avatar image.

On the other hand, when the configuration of the group service is server-based, the display device 100 according to one embodiment transmits the user image or information about the user image captured to the server 200, and the server 200 may generate a first avatar image based on the user image or information about the user image. Additionally, the server 200 may receive a user image or information about a user image of a user of another display device from another display device and generate a second avatar image. The server 200 may generate a group image including a first avatar image and a second avatar image and transmit it to display devices. Display devices can display group images received from the server 200.

Figure 3 is a flowchart showing a method by which a display device determines a service configuration method according to an embodiment.

The display device 100 according to the embodiment of FIG. 3 may be one of the plurality of display devices 101, 102, and 103 of FIG. 1.

Referring to FIG. 3, the display device 100 according to one embodiment may obtain device capability information (S310). For example, device capacity information includes type information about at least one processor included in the display device 100, capacity information, information about the number of cores, performance information, and type of memory included in the display device 100. It may include at least one of information, capacity information, and performance information.

The display device 100 may previously store device capability information.

The display device 100 according to one embodiment may receive service policy information from the server 200 (S320).

For example, the server 200 may store service decision policy information for determining a service configuration method. Service decision policy information includes device specifications (e.g., processor type information, capacity information, information on the number of cores, performance information, memory) required to perform operations included in group services such as creating an avatar on the device. It may include standards for type information, capacity information, performance information, etc.). However, it is not limited to this.

Additionally, service decision policy information may change depending on the number of concurrent users connected to the server or network conditions. Additionally, as the functions provided by the group service change, service decision policy information may also change. However, it is not limited to this.

When the server 200 receives a request for service policy information from the display device 100, the server 200 may transmit the service policy information to the display device 100. However, it is not limited to this.

The display device 100 according to one embodiment may determine a service configuration method based on device capability information and service decision policy information (S330).

For example, the display device 100 can determine whether the configuration of the group service will be device-based or server-based, and can transmit information about the determined service configuration method to the server 200 (S340).

The display device 100 and the server 200 according to one embodiment may perform operations according to the determined service configuration method (S350, S360).

For example, when the configuration of the group service is device-based, the display device 100 according to one embodiment generates a first avatar image for the user image captured, and uses the generated first avatar image to be used by another user. It can be transmitted to a display device. Additionally, a second avatar image for a user of another display device may be received from another display device. The display device 100 may generate and display a group image including a first avatar image and a second avatar image.

On the other hand, when the configuration of the group service is server-based, the display device 100 according to one embodiment transmits the user image or information about the user image captured to the server 200, and the server 200 may generate a first avatar image based on the user image or information about the user image. Additionally, the server 200 may receive a user image of another display device user or information about the user image from another display device and generate a second avatar image. The server 200 may generate a group image including a first avatar image and a second avatar image and transmit it to display devices. Display devices can display group images received from the server 200.

FIG. 4 is a flowchart showing a method of operating a plurality of display devices when a group service configuration method according to an embodiment is determined on a device basis.

A plurality of display devices according to an embodiment may include a first display device 101 and a second display device 102. A plurality of display devices may be devices connected to each other through a group service.

Referring to FIG. 4, the first display device 101 according to an embodiment may use a camera to obtain a first user image that captures a user of the first display device (the first user) (S410). At this time, the camera may be a camera mounted on the first display device 101 or a camera connected to the first display device 101 through wired or wireless communication.

The first display device 101 according to an embodiment may detect an object (e.g., the first user) from the first user image and obtain tracking information (first tracking information) for the first user. (S420).

For example, the first display device 101 may detect the first user by analyzing the first user image using an object detection module. The object detection module may include appropriate logic, circuitry, interface, and/or code to detect a specific object included in an image captured by a camera. The object detection module may detect the first user by analyzing the first user image using at least one of a machine learning, neural network, or deep learning algorithm as a rule-based or artificial intelligence algorithm.

Additionally, the first display device 101 may obtain first tracking information using a tracking module. The tracking module may include appropriate logic, circuits, interfaces, and/or code that can track the movement of a specific object included in an image captured by a camera. The tracking module is a rule-based or artificial intelligence algorithm and may use at least one of machine learning, neural network, or deep learning algorithms to obtain movement information of the first user from the first user image.

For example, the tracking module may estimate the pose of the first user, identify a specific body part (e.g., face, hand, etc.) of the first user based on the estimated pose, and Movement can be detected in more detail.

However, the object detection module and tracking module described above are only examples, and the first display device 101 according to one embodiment detects the first user in the first user image in various ways and provides first tracking information. It can be obtained.

The first display device 101 according to one embodiment may generate a first avatar image corresponding to the first user image (S430).

The first display device 101 may generate a first avatar image based on the avatar information set by the user and the first tracking information obtained in step S420. At this time, the first avatar image may be an image that moves to correspond to the first user in the first user image.

The first display device 101 according to one embodiment may transmit the first avatar image to the second display device 102 (S440).

Additionally, the second display device 102 according to one embodiment may acquire a second user image captured by a user of the second display device (second user) using a camera (S420). At this time, the camera may be a camera mounted on the second display device 102 or a camera connected to the second display device 102 through wired or wireless communication.

Additionally, the second display device 102 may detect an object (e.g., a second user) from the second user image and obtain tracking information (second tracking information) for the second user (S425). .

For example, the second display device 102 may detect the second user by analyzing the first user's image using an object detection module. Additionally, the second display device 102 may obtain second tracking information using a tracking module.

Since the object detection module and tracking module were explained in detail in step S420, detailed descriptions will be omitted. However, the object detection module and tracking module are only examples, and the second display device 102 according to one embodiment may detect the second user in the second user image and obtain second tracking information in various ways. You can.

The second display device 102 according to one embodiment may generate a second avatar image corresponding to the second user image (S435).

The second display device 102 may generate a second avatar image based on the avatar information set by the second user and the tracking information obtained in step S425. At this time, the second avatar image may be an image that moves to correspond to the second user in the second user image.

The second display device 102 according to one embodiment may transmit the second avatar image to the first display device 101 (S445).

The first display device 101 according to an embodiment may generate and display a group image including the generated first avatar image and the second avatar image received from the second display device 102 (S450).

For example, the first display device 101 divides one screen into a plurality of areas including a first area and a second area, displays the first avatar image in the first area, and displays the first avatar image in the second area. A second avatar image may be displayed. At this time, the size of the first area may be larger than the sizes of other areas. Additionally, the first area may be located at the center of the screen. Alternatively, the first display device 101 may generate a group image by combining the first avatar image and the second avatar image with a background screen set by the first user. However, it is not limited to this.

Additionally, the second display device 102 according to one embodiment may generate and display a group image including the generated second avatar image and the first avatar image received from the first display device 101 (S460) ).

For example, the second display device 102 divides one screen into a plurality of areas including a first area and a second area, displays a second avatar image in the first area, and displays a second avatar image in the second area. The first avatar image may be displayed. At this time, the size of the first area may be larger than the sizes of other areas. Additionally, the first area may be located at the center of the screen. Alternatively, the second display device 102 may generate a group image by combining the first avatar image and the second avatar image with a background screen set by the second user. However, it is not limited to this.

FIG. 5 is a diagram illustrating operations of a plurality of display devices when the configuration of a group service according to an embodiment is device-based.

Referring to FIG. 5, a plurality of display devices according to an embodiment may include a first display device 101, a second display device 102, and a third display device 103. However, it is not limited to this.

The first display device 101, the second display device 120, and the third display device 103 may be devices connected through a group service provided by the server 200. The first display device 101, the second display device 102, and the third display device 103 may be connected as a group based on their respective identification information and can share user images with each other.

For example, the first to third display devices 101, 102, and 103 transmit each user information to the server 200, and the server 200 transmits the first to third display devices 101 and 102. , 103) Based on each user information, authentication may be performed for the first to third display devices 101, 102, and 103. Accordingly, the first to third display devices 101, 102, and 103 can log in to the group fitness service.

In addition, the server 200 according to one embodiment performs an operation of generating an avatar image based on the capability information of each of the first to third display devices 101, 102, and 103. It may be decided to perform each of the fields 101, 102, and 103. Since this was explained in detail in FIG. 2, detailed description will be omitted.

Each of the first to third display devices 101, 102, and 103 may acquire an image of a user. For example, the first display device 101 may acquire a first user image 511 that captures the first user. The first display device 101 may detect the first user in the first user image 511, obtain tracking information of the first user, and generate the first avatar image 521.

The first avatar image 521 is an image that moves in the same manner as the first user of the first user image 511 and may be generated based on avatar information set by the user. The method of setting avatar information will be described in detail with reference to FIG. 6.

Figure 6 is a diagram illustrating an example of a user interface screen for setting avatar information according to an embodiment.

Referring to FIG. 6, the first display device 101 can display a user interface screen (avatar setting screen) that can set avatar information.

As shown in FIG. 6, the first display device 101 may include a menu screen 610 for setting a basic character for an avatar. For example, the first display device 101 may display a list 620 including a plurality of characters with different genders, races, ages, etc.

When receiving an input for selecting one of a plurality of characters, the first display device 101 may display the selected character 630 and display a menu screen 615 for setting items related to the selected character. there is.

The first display device 101 may display a list 640 including items that can decorate the selected character.

When the first display device 101 receives an input for selecting at least one of a plurality of items, the first display device 101 may apply the selected at least one item to the selected character 630 and display it.

The first display device 101 may store the final character as first avatar information for the first user based on the user input of selecting the save button 650 included in the menu screen 615.

However, the user interface screen for setting avatar information shown in FIG. 6 is only an example, and the first display device 101 according to one embodiment can set the first avatar information in various ways.

Referring again to FIG. 5 , the second display device 102 may obtain a second user image 512 that captures a second user. The second display device 102 may detect the second user in the second user image 512 and obtain tracking information of the second user. The second display device 102 may generate a second avatar image 522 based on the acquired tracking information of the second user. The second avatar image 522 may be generated based on avatar information set by the user, and may be an image that moves in the same manner as the second user detected in the second user image 512.

Additionally, the third display device 103 may acquire a third user image 513 that captures a third user. The third display device 103 may detect a third user in the third user image 513 and obtain tracking information of the third user. The third display device 103 may generate a third avatar image 523 based on the acquired tracking information of the third user. The third avatar image 523 may be generated based on avatar information set by the user, and may be an image that moves in the same manner as the third user detected in the third user image 513.

Each of the first to third display devices 101, 102, and 103 may transmit the generated avatar image to other display devices.

Each of the first to third display devices 101, 102, and 103 requests identification information about other display devices connected to the server 200 through a group service, and receives identification information about the other display devices from the server 200. Information can be received.

Each of the first to third display devices 101, 102, and 103 may transmit the generated avatar image to other display devices based on the received identification information about the other display devices.

For example, the first display device 101 transmits the first avatar image 521 to the second and third display devices 102 and 103, and transmits the second avatar image (521) from the second display device 102. 522), and a third avatar image 523 can be received from the third display device 103.

Additionally, the second display device 102 transmits the second avatar image 522 to the first and third display devices 101 and 103, and transmits the first avatar image 521 from the first display device 101. and can receive the third avatar image 523 from the third display device 103.

Additionally, the third display device 103 transmits the third avatar image 523 to the first and second display devices 101 and 102, and transmits the first avatar image 521 from the first display device 101. , and the second avatar image 522 can be received from the second display device 102.

The first display device 101 may generate and display a first group image 531 including a first avatar image 521, a second avatar image 522, and a third avatar image 523. The first group image 531 is divided into three areas, with the second avatar image 522 in the first area, the first avatar image 521 in the second area, and the third avatar image 523 in the third area. ) may be displayed. At this time, the size and position of the areas where the first to third avatar images 521, 522, and 523 are displayed may be set based on user input.

Additionally, the second display device 102 may generate and display a second group image 532 including the first avatar image 521, the second avatar image 522, and the third avatar image 523. . The second group image 532 is divided into three areas, with the first avatar image 521 in the first area, the second avatar image 522 in the second area, and the third avatar image 523 in the third area. ) may be displayed. At this time, the size and position of the areas where the first to third avatar images 521, 522, and 523 are displayed may be set based on user input.

Additionally, the third display device 103 may generate a third group image 533 including the first avatar image 521, the second avatar image 522, and the third avatar image 523. The third group image 533 is divided into three areas, with the first avatar image 521 in the first area, the third avatar image 523 in the second area, and the second avatar image 522 in the third area. ) may be displayed. At this time, the size and position of the areas where the first to third avatar images 521, 522, and 523 are displayed may be set based on user input.

FIG. 7 is a flowchart illustrating a method of operating a plurality of display devices when a group service configuration method according to an embodiment is determined on a device basis.

A plurality of display devices according to an embodiment may include a first display device 101 and a second display device 102. A plurality of display devices may be devices connected to each other through a group service.

Referring to FIG. 7, the first display device 101 may acquire a first user image captured by a camera using a camera (S710). At this time, the camera may be a camera mounted on the first display device 101 or a camera connected to the first display device 101 through wired or wireless communication.

Additionally, the first display device 101 may detect the first user from the first user image and obtain tracking information (first tracking information) for the first user (S720).

Since the method of obtaining tracking information for the first user has been described in detail in step 420 (S420) of FIG. 4, the same description will be omitted.

The first display device 101 may transmit the first tracking information to the second display device 102 (S730).

Additionally, the second display device 102 according to one embodiment may acquire a second user image captured by a user of the second display device (second user) using a camera (S715). At this time, the camera may be a camera mounted on the second display device 102 or a camera connected to the second display device 102 through wired or wireless communication.

Additionally, the second display device 102 may detect the second user from the second user image and obtain tracking information (second tracking information) about the second user (S725).

Since the method of obtaining tracking information for the user has been described in detail in step 425 (S425) of FIG. 4, the same description will be omitted.

The second display device 102 may transmit the second tracking information to the first display device 101 (S735).

The first display device 101 according to one embodiment may generate a first avatar image based on first tracking information and a second avatar image based on second tracking information (S740 ).

For example, the first display device 101 may generate a first avatar image corresponding to the first user image based on first avatar information and first tracking information set by the first user. Additionally, the first display device 101 may obtain second avatar information and generate a second avatar image corresponding to the second user image based on the second avatar information and the second tracking information. At this time, the second avatar information may be obtained by receiving avatar information set by the second user or obtained by setting the first user. However, it is not limited to this.

The first display device 101 may generate and display a first group image including a first avatar image and a second avatar image (S750).

For example, the first display device 101 may generate a first group image by combining the first avatar image and the second avatar image into a virtual background screen. However, it is not limited to this.

Additionally, the second display device 102 according to one embodiment may generate a first avatar image based on first tracking information and may generate a second avatar image based on second tracking information. (S745).

For example, the second display device 102 may obtain first avatar information and generate a first avatar image corresponding to the first user image based on the first avatar information and first tracking information. At this time, the first avatar information may be obtained by receiving avatar information set by the first user or obtained by setting the second user. However, it is not limited to this.

Additionally, the second display device 102 may generate a second avatar image corresponding to the second user image based on the second avatar information and second tracking information set by the second user.

The second display device 102 may generate and display a second group image including the first avatar image and the second avatar image (S755).

For example, the second display device 102 may generate a second group image by combining the first avatar image and the second avatar image into a virtual background screen. However, it is not limited to this.

FIG. 8 is a diagram illustrating operations of a plurality of display devices when the configuration of a group service according to an embodiment is device-based.

Referring to FIG. 8, a plurality of display devices according to an embodiment may include a first display device 101, a second display device 102, and a third display device 103. However, it is not limited to this.

The first display device 101, the second display device 120, and the third display device 103 may be devices connected through a group service provided by the server 200. The first display device 101, the second display device 120, and the third display device 103 may be connected as a group based on their respective identification information and may be able to share user images with each other.

For example, the first to third display devices 101, 102, and 103 transmit each user information to the server 200, and the server 200 transmits the first to third display devices 101 and 102. , 103) Based on each user information, authentication may be performed for the first to third display devices 101, 102, and 103. Accordingly, the first to third display devices 101, 102, and 103 can log in to the group fitness service.

In addition, the server 200 according to one embodiment performs an operation of generating an avatar image based on the capability information of each of the first to third display devices 101, 102, and 103. It may be decided to perform each of the fields 101, 102, and 103. Since this was explained in detail in FIG. 2, detailed description will be omitted.

Each of the first to third display devices 101, 102, and 103 may acquire an image of a user. For example, the first display device 101 may acquire a first user image 811 that captures the first user. The first display device 101 may detect the first user in the first user image 811 and obtain tracking information (first tracking information, 821) about the first user. Additionally, the second display device 102 may acquire a second user image 812. The second display device 102 may detect the second user from the second user image and obtain tracking information (second tracking information, 822) about the second user. Additionally, the third display device 103 may acquire a third user image 813. The third display device 103 may detect a third user from the third user image and obtain tracking information (third tracking information, 823) about the third user.

In addition, although not shown, if the device specifications of the second display device 102 do not meet the criteria for obtaining the second tracking information from the second user image 812, the second display device 102 The second user image 812 may be transmitted to an external electronic device linked to the display device 102. The external electronic device may be a personal electronic device of the second user, and the device specifications of the external electronic device may satisfy the criteria for obtaining second tracking information from the second user image 812. The external electronic device may detect the second user in the second user image and obtain second tracking information 822 for the second user. The external electronic device may transmit second tracking information 822 to the second display device 102.

Each of the first to third display devices 101, 102, and 103 may transmit tracking information to other display devices.

Each of the first to third display devices 101, 102, and 103 requests identification information about other display devices connected to the server 200 through a group service, and receives identification information about the other display devices from the server 200. Information can be received. Each of the first to third display devices 101, 102, and 103 may transmit tracking information to other display devices based on the received identification information about the other display devices.

For example, the first display device 101 transmits first tracking information 821 to the second and third display devices 102 and 103, and receives second tracking information (821) from the second display device 102. 822), and third tracking information 823 can be received from the third display device 103.

Additionally, the second display device 102 transmits the second tracking information 822 to the first and third display devices 101 and 103, and receives the first tracking information 821 from the first display device 101. Third tracking information 823 can be received from the third display device 103.

Additionally, the third display device 103 transmits third tracking information 823 to the first and second display devices 101 and 102, and receives first tracking information 821 from the first display device 101. Can receive second tracking information 822 from the second display device 102.

The first display device 101 generates a first avatar image based on the first tracking information 821 and the first avatar information, and generates a second avatar image based on the second tracking information 822 and the second avatar information. An avatar image can be generated, and a third avatar image can be generated based on the third tracking information 823 and the third avatar information. At this time, the first avatar information may be set by the first user, and the second avatar information and third avatar information may be set by the first user or received from the second and third display devices 102 and 103. You can. However, it is not limited to this.

The first display device 101 may generate and display a first group image 831 by combining the first to third avatar images into a virtual background screen.

The second display device 102 generates a first avatar image based on the first tracking information 821 and the first avatar information, and generates a second avatar image based on the second tracking information 822 and the second avatar information. An avatar image can be generated, and a third avatar image can be generated based on the third tracking information 823 and the third avatar information. At this time, the second avatar information may be set by the second user, and the first avatar information and third avatar information may be set by the second user or received from the first and third display devices 101 and 103. You can. However, it is not limited to this.

The second display device 102 may generate and display a second group image 832 by combining the first to third avatar images into a virtual background screen.

Alternatively, if the second display device 102 cannot perform the function of generating an avatar image or group image based on tracking information, the second display device 102 may be interlocked with the second display device 102. The first to third tracking information (821, 822, 823) can be transmitted to an external electronic device. The external electronic device may generate first to third avatar images based on the first to third tracking information 821, 822, and 823. Additionally, the external electronic device may generate a second group image 832 by combining the first to third avatar images into a virtual background screen. The external electronic device may transmit the first to third avatar images or the second group image 832 to the second display device 102.

The third display device 103 generates a first avatar image based on the first tracking information 821 and the first avatar information, and generates a second avatar image based on the second tracking information 822 and the second avatar information. An avatar image can be generated, and a third avatar image can be generated based on the third tracking information 823 and the third avatar information. At this time, the third avatar information may be set by the third user, and the first avatar information and the second avatar information may be set by the third user or received from the first and second display devices 101 and 102. You can. However, it is not limited to this.

The third display device 103 may generate and display a third group image 833 by combining the first to third avatar images into a virtual background screen.

FIG. 9 is a flowchart illustrating a method of operating a plurality of display devices and a server when a group service configuration method according to an embodiment is determined to be server-based.

A plurality of display devices according to an embodiment may include a first display device 101 and a second display device 102, and the first display device 101 and the second display device 102 may be used as a group service. These may be devices connected to each other.

Referring to FIG. 9, the first display device 101 according to one embodiment may obtain a first user image captured by a camera using a camera (S910). At this time, the camera may be a camera mounted on the first display device 101 or a camera connected to the first display device 101 through wired or wireless communication.

Additionally, the first display device 101 may transmit the first user image to the server 200 (S920).

The second display device 102 may acquire a second user image captured by a camera using a camera (S915). At this time, the camera may be a camera mounted on the second display device 102 or a camera connected to the second display device 102 through wired or wireless communication.

Additionally, the second display device 102 may transmit the second user image to the server 200 (S925).

The server 200 may detect the first user from the first user image and obtain first tracking information about the first user. Additionally, the server 200 may detect the second user from the second user image and obtain second tracking information about the second user (S940).

For example, the server 200 may detect the first user by analyzing the first user's image using an object detection module. The object detection module may include appropriate logic, circuitry, interface, and/or code to detect a specific object included in an image captured by a camera. The object detection module may detect the first user by analyzing the first user image using at least one of a machine learning, neural network, or deep learning algorithm as a rule-based or artificial intelligence algorithm. Additionally, the server 200 may obtain first tracking information using a tracking module. The tracking module may include appropriate logic, circuits, interfaces, and/or code that can track the movement of a specific object included in an image captured by a camera. The tracking module is a rule-based or artificial intelligence algorithm and may use at least one of machine learning, neural network, or deep learning algorithms to obtain movement information of the first user from the first user image.

For example, the tracking module may estimate the pose of the first user, identify a specific body part (e.g., face, hand, etc.) of the first user based on the estimated pose, and Movement can be detected in more detail.

Additionally, the server 200 can detect the second user by analyzing the second user image using an object detection module and obtain movement information of the second user from the second user image using the tracking module. there is.

Meanwhile, the first display device 101 may transmit the first avatar information set by the first user to the server 200 (S950), and the second display device 102 may transmit the second avatar information set by the second user. Information can be transmitted to the server 200 (S955).

The server 200 may generate a first avatar image based on the first tracking information and the first avatar information, and generate a second avatar image based on the second tracking information and the second avatar information (S960) ). At this time, the first avatar image may be an image that moves to correspond to the first user in the first user image, and the second avatar image may be an image that moves to correspond to the second user in the second user image.

The server 200 may generate a group image including a first avatar image and a second avatar image (S970). For example, the server 200 may create a group image by combining the first avatar image and the second avatar image into a virtual background screen.

Alternatively, the server 200 may generate a first group image and a second group image in which the sizes and positions of the first avatar image and the second avatar image are different from each other. For example, in the first group of images, the first avatar image may be displayed larger than the second avatar image, and in the second group of images, the second avatar image may be displayed larger than the first avatar image. However, it is not limited to this.

The server 200 may transmit the group image to the first display device 101 and the second display device 102 (S980, S985). For example, the server 200 may transmit the first group image to the first display device 101 and the second group image to the second display device 102.

The first display device 101 and the second display device 102 can display the group image received from the server 200 (S990, S995). For example, the first display device 101 may display a first group image, and the second display device 102 may display a second group image.

Figure 10 is a flowchart showing a method of operating a plurality of display devices and a server when a group service configuration method according to an embodiment is determined to be server-based.

A plurality of display devices according to an embodiment may include a first display device 101 and a second display device 102, and the first display device 101 and the second display device 102 may be used as a group service. These may be devices connected to each other.

Referring to FIG. 10, the first display device 101 according to an embodiment may obtain a first user image captured by a camera using a camera (S1010).

The first display device 101 may detect the first user from the first user image and obtain first tracking information of the detected first user (S1020).

For example, the first display device 101 may detect the first user by analyzing the first user image using an object detection module. The object detection module may include appropriate logic, circuitry, interface, and/or code to detect a specific object included in an image captured by a camera. The object detection module may detect the first user by analyzing the first user image using at least one of a machine learning, neural network, or deep learning algorithm as a rule-based or artificial intelligence algorithm.

Additionally, the first display device 101 may obtain first tracking information using a tracking module. The tracking module may include appropriate logic, circuitry, interface, and/or code that can track the movement of a specific object included in an image captured by a camera. The tracking module is a rule-based or artificial intelligence algorithm and may use at least one of machine learning, neural network, or deep learning algorithms to obtain movement information of the first user from the first user image.

For example, the tracking module may estimate the pose of the first user, identify a specific body part (e.g., face, hand, etc.) of the first user based on the estimated pose, and Movement can be detected in more detail.

However, the object detection module and tracking module described above are only examples, and the first display device 101 according to one embodiment detects the first user in the first user image in various ways and provides first tracking information. It can be obtained.

The first display device 101 may transmit first tracking information to the server 200 (S1030).

Additionally, the first display device 101 may transmit the first avatar information set by the first user to the server 200 (S1050).

Additionally, the second display device 102 according to one embodiment may acquire a second user image captured by a camera using a camera (S1015).

The second display device 102 may detect the second user from the second user image and obtain second tracking information about the second user (S1025).

The second display device 102 may obtain second tracking information about the second user using the object detection module and tracking module described in step S1020.

The second display device 102 may transmit the second tracking information to the server 200 (S1035).

Additionally, the second display device 102 may transmit second avatar information set by the second user to the server 200 (S1055).

The server 200 according to one embodiment generates a first avatar image based on first tracking information and first avatar information, and generates a second avatar image based on second tracking information and second avatar information. You can do it (S1060). At this time, the first avatar image may be an image that moves to correspond to the first user in the first user image, and the second avatar image may be an image that moves to correspond to the second user in the second user image.

The server 200 may generate a group image including a first avatar image and a second avatar image (S1070). For example, the server 200 may create a group image by combining the first avatar image and the second avatar image into a virtual background screen.

Alternatively, the server 200 may generate a first group image and a second group image in which the sizes and positions of the first avatar image and the second avatar image are different from each other. For example, in the first group of images, the first avatar image may be displayed larger than the second avatar image, and in the second group of images, the second avatar image may be displayed larger than the first avatar image. However, it is not limited to this.

The server 200 may transmit the group image to the first display device 101 and the second display device 102 (S1080, S1085). For example, the server 200 may transmit the first group image to the first display device 101 and the second group image to the second display device 102.

The first display device 101 and the second display device 102 can display the group image received from the server 200 (S1090, S1095). For example, the first display device 101 may display a first group image, and the second display device 102 may display a second group image.

Figure 11 is a block diagram showing the configuration of a display device according to an embodiment.

The display device 100 according to the embodiment of FIG. 11 may be one of the plurality of display devices 101, 102, and 103 of FIGS. 1 to 10.

Referring to FIG. 11 , the display device 100 according to an embodiment may include a camera 110, a processor 120, a memory 130, a display 140, and a communication unit 150.

The camera 110 according to one embodiment may acquire image frames such as still images or moving images. For example, the camera 110 may capture an image of the outside of the display device 100 and may capture a user of the display device 100. At this time, the image captured through the camera 110 may be processed through the processor 120 or a separate image processor.

The camera 110 according to one embodiment may be built into or mounted as a component of the display device 100. Alternatively, the camera 110 is a separate component from the display device 100 and may be connected to the display device 100 through a wired or wireless communication method through an input/output interface or communication unit 150. However, it is not limited to this.

The processor 120 according to one embodiment controls the overall operation of the display device 100 and signal flow between internal components of the display device 100, and performs the function of processing data.

The processor 120 may include single core, dual core, triple core, quad core, and multiple cores thereof. Additionally, the processor 120 may include a plurality of processors. For example, the processor 120 may be implemented as a main processor (not shown) and a sub processor (not shown).

Additionally, the processor 120 may include at least one of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a Video Processing Unit (VPU). Alternatively, depending on the embodiment, it may be implemented in the form of a SoC (System On Chip) integrating at least one of CPU, GPU, and VPU. Alternatively, the processor 120 may further include a Neural Processing Unit (NPU).

The memory 130 according to one embodiment may store various data, programs, or applications for driving and controlling the display device 100.

Additionally, a program stored in memory 130 may include one or more instructions. A program (one or more instructions) or application stored in the memory 130 may be executed by the processor 120.

The processor 120 according to one embodiment executes one or more instructions stored in the memory 130, obtains device capability information of the display device 100, and transmits it to the server 200. You can control it.

Device capacity information according to an embodiment includes type information, capacity information, information about the number of cores, performance information, and memory included in the display device 100 for at least one processor included in the display device 100. It may include at least one of type information, capacity information, and performance information.

The processor 120 may receive information about the configuration method of the group service from the server 200. Alternatively, the processor 120 may determine a group service configuration method based on device capability information and service decision policy information received from the server 200.

If the group service configuration method according to one embodiment is determined based on the device, the processor 120 may detect the user from a user image captured of the user and obtain tracking information. The processor 120 may detect the user by analyzing the user image using an object detection module. The object detection module may include appropriate logic, circuitry, interface, and/or code to detect a specific object included in an image captured by a camera. The object detection module is a rule-based or artificial intelligence algorithm that can detect the user by analyzing the user's image using at least one of machine learning, neural network, or deep learning algorithms.

Additionally, the processor 120 may obtain tracking information using a tracking module. The tracking module may include appropriate logic, circuits, interfaces, and/or code that can track the movement of a specific object included in an image captured by a camera. The tracking module is a rule-based or artificial intelligence algorithm that can obtain user's movement information from the user's image using at least one of machine learning, neural network, or deep learning algorithms.

For example, the tracking module may estimate the user's pose, identify specific body parts of the user (e.g., face, hands, etc.) based on the estimated pose, and further perform movements for the identified specific body parts. It can be detected in detail. However, it is not limited to this.

For example, the processor 120 may obtain tracking information using a neural network trained to obtain tracking information from a user image. However, it is not limited to this.

The neural network according to one embodiment may be a network in which various standards or conditions for obtaining tracking information from user images are learned based on various training data. Here, 'training' refers to the method of analyzing the input data when various collected data are input to the neural network network, and the method of obtaining tracking information from the user's image based on the input data. It can mean training a neural network so that the network can discover or learn on its own. Accordingly, the trained neural network can obtain tracking information from the user image. At this time, the learned neural network may be stored in advance in the display device 100 and may be periodically updated through communication with an external server.

The processor 120 according to one embodiment may generate an avatar image corresponding to a user image. The processor 120 may generate an avatar image based on avatar information and tracking information set by the user. At this time, the avatar image may be an image that moves to correspond to the user in the user image.

The processor 120 according to one embodiment may control the communication unit 150 to transmit the avatar image to another display device. Additionally, the processor 120 may receive another avatar image for another user or information for generating another avatar image (for example, tracking information of another user) from another display device through the communication unit 150.

The processor 120 may generate a group image by combining an avatar image with another user's avatar image (another avatar image).

On the other hand, when the group service configuration method according to one embodiment is determined based on the device, the processor 120 may control the communication unit 150 to transmit the user image or tracking information detected from the user image to the server 200. there is.

The processor 120 may receive a group image including avatar images of a plurality of users from the server 200, and may control the received group image to be displayed on the display 140.

The display 140 according to one embodiment may generate a driving signal by converting an image signal, a data signal, an OSD signal, a control signal, etc. processed by the processor 120, and display an image according to the driving signal.

The display 140 according to one embodiment may be implemented as a PDP, LCD, OLED, flexible display, etc., and may also be implemented as a 3D display. Additionally, the display 140 can be configured as a touch screen and used as an input device in addition to an output device.

The display 140 according to one embodiment may display a user interface screen for setting avatar information. Additionally, the display 140 may display a group image including a plurality of avatar images for a plurality of users. For example, the display 140 may divide one screen into a plurality of areas and individually display different avatar images in each of the divided areas.

Alternatively, the group image according to one embodiment may include an image in the form of a composite of a plurality of avatar images on a virtual background screen. However, it is not limited to this.

The communication unit 150 according to one embodiment may transmit and receive data or signals with an external device or server. For example, the communication unit 150 may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, a LAN module, an Ethernet module, a wired communication module, etc. At this time, each communication module may be implemented in the form of at least one hardware chip.

The Wi-Fi module and Bluetooth module communicate using Wi-Fi and Bluetooth methods, respectively. When using a Wi-Fi module or a Bluetooth module, various connection information such as SSID and session key are first transmitted and received, and various information can be transmitted and received after establishing a communication connection using this. Wireless communication modules include zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc. It may include at least one communication chip that performs communication according to various wireless communication standards.

The communication unit 150 according to one embodiment may transmit device capability information to the server 200 and receive information about a group service configuration method from the server 200.

In addition, the communication unit 150 can transmit a user image taken of a user or tracking information obtained from a user image to the server 200, and send a group image including avatar images of a plurality of users from the server 200. You can receive it.

Additionally, the communication unit 150 may receive identification information about other display devices connected to the group service from the server 200.

Additionally, the communication unit 150 may transmit tracking information or an avatar image obtained from a user's image to another display device, and may receive tracking information about another user or an avatar image of another user from another display device. However, it is not limited to this.

Figure 12 is a block diagram showing the configuration of a display device according to an embodiment.

Referring to FIG. 12, the display device 1200 of FIG. 12 may be another embodiment of the display device 100 of FIG. 11.

Referring to FIG. 12, the display device 1200 according to an embodiment includes a tuner unit 1240, a processor 1210, a display 1220, a communication unit 1250, a detection unit 1230, and an input/output unit ( 1270), a video processing unit 1280, an audio processing unit 1285, an audio output unit 1260, a memory 1290, and a power supply unit 1295.

The camera unit 1232, processor 1210, display 1220, communication unit 1250, and memory 1290 included in the detection unit 1230 of FIG. 12 are the camera 110 and camera 110 of FIG. 11, respectively. , Since it is a configuration corresponding to the processor 120, display 140, communication unit 150, and memory 130, the same description will be omitted.

The tuner unit 1240 according to an embodiment amplifies, mixes, resonates, etc. broadcast signals received by wire or wirelessly to determine which of the many radio wave components the display device 1200 wants to receive. You can select only the frequency of the channel by tuning it. Broadcast signals include audio, video, and additional information (eg, Electronic Program Guide (EPG)).

The tuner unit 1240 can receive broadcast signals from various sources, such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, and Internet broadcasting. The tuner unit 1240 may receive broadcast signals from sources such as analog broadcasting or digital broadcasting.

The detection unit 1230 detects the user's voice, the user's image, or the user's interaction, and may include a microphone 1231, a camera unit 1232, and a light receiver 1233.

The microphone 1231 receives the user's uttered voice. The microphone 1231 may convert the received voice into an electrical signal and output it to the processor 1210. The user voice may include, for example, a voice corresponding to a menu or function of the display device 1200.

The camera unit 1232 may receive images (eg, consecutive frames) corresponding to the user's motion, including gestures, within the camera recognition range. The processor 1210 can use the received motion recognition result to select a menu displayed on the display device 1200 or perform control corresponding to the motion recognition result.

The light receiver 1233 receives optical signals (including control signals) received from an external control device through a light window (not shown) of the bezel of the display 1220. The light receiver 1233 may receive an optical signal corresponding to a user input (eg, touch, press, touch gesture, voice, or motion) from the control device. A control signal may be extracted from the received optical signal under the control of the processor 1210.

The input/output unit 1270 provides video (e.g., video, etc.), audio (e.g., voice, music, etc.), and additional information (e.g., EPG, etc.) from the outside of the display device 1200. receives. Input/output interfaces include HDMI (High-Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, and RGB port. , D-SUB (D-subminiature), DVI (Digital Visual Interface), component jack, or PC port.

The processor 1210 controls the overall operation of the display device 1200 and signal flow between internal components of the display device 1200, and performs the function of processing data.

The processor 1210 stores signals or data input from the outside of the display device 1200, or uses RAM as a storage area corresponding to various tasks performed on the display device 1200. It may include a ROM and a processor storing a control program for control.

The video processing unit 1280 performs processing on video data received by the display device 1200. The video processing unit 1280 can perform various image processing such as decoding, scaling, noise filtering, frame rate conversion, and resolution conversion on video data.

The display 1220 according to one embodiment may generate a driving signal by converting an image signal, a data signal, an OSD signal, a control signal, etc. processed by the processor 1210, and display an image according to the driving signal.

The audio processing unit 1285 according to one embodiment performs processing on audio data. The audio processing unit 1285 may perform various processing such as decoding, amplification, noise filtering, etc. on audio data. Meanwhile, the audio processing unit 1285 may be equipped with a plurality of audio processing modules to process audio corresponding to a plurality of contents.

The audio output unit 1260 outputs audio included in the broadcast signal received through the tuner unit 1240 under the control of the processor 1210. The audio output unit 1260 may output audio (eg, voice, sound) input through the communication unit 1250 or the input/output unit 1870. Additionally, the audio output unit 1260 may output audio stored in the memory 1290 under the control of the processor 1810. The audio output unit 1260 may include at least one of a speaker, a headphone output terminal, or a Sony/Philips Digital Interface (S/PDIF) output terminal.

The power unit 1295 supplies power input from an external power source to the components inside the display device 1200 under the control of the processor 1210. Additionally, the power unit 1295 may supply power output from one or more batteries (not shown) located inside the display device 1200 to internal components under the control of the processor 1210.

The memory 1290 may store various data, programs, or applications for driving and controlling the display device 1200 under the control of the processor 1210. The memory 1290 includes a broadcast reception module (not shown), a channel control module, a volume control module, a communication control module, a voice recognition module, a motion recognition module, an optical reception module, a display control module, an audio control module, an external input control module, and a power supply. It may include a control module, a power control module of an external device connected wirelessly (eg, Bluetooth), a voice database (DB), or a motion database (DB). Not shown modules and database of the memory 990 include broadcast reception control function, channel control function, volume control function, communication control function, voice recognition function, motion recognition function, and light reception control function in the display device 1800. , may be implemented in software form to perform a display control function, an audio control function, an external input control function, a power control function, or a power control function of an external device connected wirelessly (eg, Bluetooth). The processor 1810 can perform each function using these software stored in the memory 1290.

Meanwhile, the block diagrams of the display devices 100 and 1200 shown in FIGS. 11 and 12 are block diagrams for one embodiment. Each component of the block diagram may be integrated, added, or omitted depending on the specifications of the display devices 100 and 1200 that are actually implemented. That is, as needed, two or more components may be combined into one component, or one component may be subdivided into two or more components. In addition, the functions performed by each block are for explaining the embodiments, and the specific operations or devices do not limit the scope of the present invention.

Figure 13 is a block diagram showing the configuration of a server according to an embodiment.

Referring to FIG. 13, the server 200 according to one embodiment may include a communication unit 210, a processor 220, and a memory 230.

The communication unit 210 according to one embodiment may transmit and receive data or signals with a plurality of display devices. For example, the communication unit 210 may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, a LAN module, an Ethernet module, a wired communication module, etc. At this time, each communication module may be implemented in the form of at least one hardware chip.

The Wi-Fi module and Bluetooth module communicate using Wi-Fi and Bluetooth methods, respectively. When using a Wi-Fi module or a Bluetooth module, various connection information such as SSID and session key are first transmitted and received, and various information can be transmitted and received after establishing a communication connection using this. Wireless communication modules include zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc. It may include at least one communication chip that performs communication according to various wireless communication standards.

The communication unit 210 according to one embodiment may receive device capability information from a plurality of display devices connected through a group service. Additionally, the communication unit 210 may transmit information about the group service configuration method determined based on the received device capability information to a plurality of display devices.

Additionally, the communication unit 210 may receive a user image or tracking information obtained from a user image from a plurality of display devices. The communication unit 210 may transmit a group image including avatar images for a plurality of users created based on the user image or tracking information to a plurality of display devices.

The processor 220 according to one embodiment controls the overall operation of the server 200 and signal flow between internal components of the server 200, and performs the function of processing data.

Processor 220 may include single core, dual core, triple core, quad core, and multiple cores thereof. Additionally, the processor 220 may include a plurality of processors. For example, the processor 220 may be implemented as a main processor (not shown) and a sub processor (not shown).

Additionally, the processor 220 may include at least one of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a Video Processing Unit (VPU). Alternatively, depending on the embodiment, it may be implemented in the form of a SoC (System On Chip) integrating at least one of CPU, GPU, and VPU. Alternatively, the processor 220 may further include a Neural Processing Unit (NPU).

The memory 230 according to one embodiment may store various data, programs, or applications for driving and controlling the server 200.

Additionally, a program stored in memory 230 may include one or more instructions. A program (one or more instructions) or application stored in the memory 230 may be executed by the processor 220.

The processor 220 according to an embodiment may execute one or more instructions stored in the memory 230 to determine a group service configuration method based on device capability information and service decision policy information received from the display device. there is.

Device capacity information according to an embodiment includes type information, capacity information, information about the number of cores, performance information, type information about memory included in the display device, capacity information, May include at least one of performance information.

The server 200 may store service decision policy information for determining a service configuration method. Service decision policy information includes device specifications (e.g., processor type information, capacity information, information on the number of cores, performance information, memory) required to perform operations included in group services such as creating an avatar on the device. It may include standards for type information, capacity information, performance information, etc.). However, it is not limited to this.

Service decision policy information may change depending on the number of concurrent users connected to the server or network conditions, etc. Additionally, as the functions provided by the group service change, service decision policy information may also change. However, it is not limited to this.

The processor 220 may compare device capability information and service decision policy information to determine whether to configure the group service based on a device or a server.

If it is determined that the group service will be configured based on the server, the processor 220 may detect the user in the user image received from the display device and obtain tracking information.

The processor 220 may detect the user by analyzing the user image using an object detection module. The object detection module may include appropriate logic, circuitry, interface, and/or code to detect a specific object included in an image captured by a camera. The object detection module is a rule-based or artificial intelligence algorithm that can detect the user by analyzing the user's image using at least one of machine learning, neural network, or deep learning algorithms.

Additionally, the processor 220 may obtain tracking information using a tracking module. The tracking module may include appropriate logic, circuits, interfaces, and/or code that can track the movement of a specific object included in an image captured by a camera. The tracking module is a rule-based or artificial intelligence algorithm that can obtain user's movement information from the user's image using at least one of machine learning, neural network, or deep learning algorithms.

For example, the tracking module may estimate the user's pose, identify specific body parts of the user (e.g., face, hands, etc.) based on the estimated pose, and further perform movements for the identified specific body parts. It can be detected in detail. However, it is not limited to this.

Alternatively, the processor 220 may generate an avatar image based on tracking information and avatar information received from the display device. At this time, the avatar image may be an image that moves to correspond to the user in the user image.

The processor 220 may control the communication unit 210 to generate a group image by combining avatar images of a plurality of users and transmit the group image to a plurality of display devices.

A display device according to an embodiment may include a communication unit, a memory storing one or more display instructions, and at least one processor executing the one or more instructions.

The at least one processor may obtain a first user image captured by the first user by executing the one or more instructions.

The at least one processor may control the communication unit to transmit capability information of the display device to a server by executing the one or more instructions.

The at least one processor may receive information about a group service configuration method determined based on the capability information from the server by executing the one or more instructions.

The at least one processor may acquire the first avatar image and a second avatar image corresponding to another display device user based on the received information by executing the one or more instructions.

The at least one processor may control the display to display a group image including the first avatar image and the second avatar image by executing the one or more instructions.

Capability information of the display device includes at least one of type information, capacity information, information about the number of cores, performance information, type information about the memory, capacity information, and performance information about the at least one processor included in the display device. It can contain one.

The configuration method of the group service is a device-based configuration method in which the display device generates a first avatar image corresponding to the first user image, and the server is connected to the first avatar image and the group service. It may be decided on one of the server-based configuration methods for generating a second avatar image corresponding to the device user.

By executing the one or more instructions, the at least one processor detects the first user in the first user image based on the device-based configuration method of the group service, and executes the first user image. Based on the user's tracking information, the first avatar image can be generated.

The at least one processor may control the communication unit to receive at least one of the second avatar image and information about the second avatar image from the other display device by executing the one or more instructions.

The at least one processor may generate the second avatar image based on receiving information about the second avatar image by executing the one or more instructions.

The at least one processor may execute the one or more instructions to synthesize the first avatar image, the second avatar image, and the virtual background image to generate the group image.

The at least one processor may control the communication unit to request device information about the other display device from the server and receive device information about the other display device from the server by executing the one or more instructions. .

The at least one processor executes the one or more instructions to transmit at least one of the first avatar image or information about the first avatar image to the other display device based on device information about the other display device. The communication unit can be controlled.

The at least one processor executes the one or more instructions, based on which the configuration method of the group service is determined to be the server-based configuration method, and the first user image and the first user detected from the first user video The communication unit can be controlled to transmit at least one of the information about to the server.

The at least one processor may receive the group image including the first avatar image and the second avatar image from the server by executing the one or more instructions.

The at least one processor may receive an input for setting avatar information corresponding to the first avatar image by executing the one or more instructions.

The first avatar image may be an avatar image generated based on the set avatar information.

The display device may further include a camera that captures the first user and obtains an image of the first user.

A method of operating a display device according to an embodiment may include acquiring a first user image that captures a first user.

A method of operating a display device according to an embodiment may include transmitting capability information of the display device to a server.

A method of operating a display device according to an embodiment may include receiving information about a configuration method of a group service determined based on the capability information from the server.

A method of operating a display device according to an embodiment may include obtaining the first avatar image and a second avatar image corresponding to another display device user based on the received information.

A method of operating a display device according to an embodiment may include displaying a group image including the first avatar image and the second avatar image.

The step of acquiring the first avatar image and the second avatar image includes detecting the first user from the first user image based on the configuration method of the group service being determined as the device-based configuration method, It may include generating the first avatar image based on the tracking information of the first user.

Obtaining the first avatar image and the second avatar image may include receiving at least one of the second avatar image and information about the second avatar image from the other display device.

Obtaining the first avatar image and the second avatar image may include generating the second avatar image based on receiving information about the second avatar image.

The method of operating a display device according to an embodiment may further include generating the group image by synthesizing the first avatar image, the second avatar image, and the virtual background image.

A method of operating a display device according to an embodiment includes requesting device information about the other display device from the server based on the configuration method of the group service being determined as the device-based configuration method, and receiving device information from the server. The method may further include receiving device information about the other display device.

A method of operating a display device according to an embodiment includes transmitting at least one of the first avatar image or information about the first avatar image to the other display device based on device information about the other display device. More may be included.

The step of acquiring the first avatar image and the second avatar image includes the first user image and the second avatar image detected from the first user image, based on the configuration method of the group service being determined as the server-based configuration method. 1 It may include transmitting at least one piece of information about the user to the server.

Obtaining the first avatar image and the second avatar image may include receiving the group image including the first avatar image and the second avatar image from the server.

A method of operating a display device according to an embodiment may include receiving an input for selecting avatar information corresponding to the first avatar image.

The first avatar image may be an avatar image generated based on the set avatar information.

A display device and a method of operating the same according to an embodiment configure a group service variably according to device capability information of the display device, so that various devices can use the group service. Accordingly, by allowing group services to be used through a server even if the specifications of the device are not high, the range of services that can be provided can be expanded.

Additionally, the display device and its operating method according to one embodiment can optimize the configuration method of the group service by configuring the group service variably according to network conditions.

In addition, the display device and its operating method according to one embodiment can protect the privacy of individual users by displaying a group image converted from a user image to an avatar image when using a group service.

A method of operating a display device according to an embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and constructed for the present invention or may be known and usable by those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

Additionally, at least one of the methods of operating a display device according to the disclosed embodiments may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers.

A computer program product may include a S/W program and a computer-readable storage medium in which the S/W program is stored. For example, a computer program product may include a product in the form of a S/W program (e.g., a downloadable app) distributed electronically by the manufacturer of an electronic device or through an electronic marketplace (e.g., Google Play Store, App Store). there is. For electronic distribution, at least part of the S/W program may be stored in a storage medium or temporarily created. In this case, the storage medium may be a manufacturer's server, an electronic market server, or a relay server's storage medium that temporarily stores the SW program.

A computer program product, in a system comprised of a server and a client device, may include a storage medium of a server or a storage medium of a client device. Alternatively, if there is a third device (e.g., a smartphone) in communication connection with the server or client device, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include the S/W program itself, which is transmitted from a server to a client device or a third device, or from a third device to a client device.

In this case, one of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of a server, a client device, and a third device may execute the computer program product and perform the methods according to the disclosed embodiments in a distributed manner.

For example, a server (eg, a cloud server or an artificial intelligence server, etc.) may execute a computer program product stored on the server and control a client device connected to the server to perform the method according to the disclosed embodiments.

Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also included in the scope of the present invention. belongs to

Claims (20)

In the display device 100,
Communication Department (150);
display 140;
a memory 130 that stores one or more instructions; and
Comprising at least one processor 120 that executes the one or more instructions,
The at least one processor 120 executes the one or more instructions,
Obtaining a first user image captured by the first user,
Controlling the communication unit to transmit capability information of the display device to a server,
Receiving, from the server, information on a configuration method of a group service determined based on the capability information,
Based on the received information, obtain the first avatar image and a second avatar image corresponding to another display device user,
A display device that controls the display to display a group image including the first avatar image and the second avatar image. According to paragraph 1,
Capability information of the display device includes at least one of type information, capacity information, information about the number of cores, performance information, type information about the memory, capacity information, and performance information about the at least one processor included in the display device. A display device, including one. According to claim 1 or 2,
The configuration method of the group service is,
A device-based configuration method in which the display device generates a first avatar image corresponding to the first user image, and the server generates the first avatar image and a second avatar corresponding to the other device user connected to the group service. A display device determined by one of the server-based configuration methods for generating images. According to paragraph 3,
The at least one processor 120 executes the one or more instructions,
Based on the configuration method of the group service being determined as the device-based configuration method,
Detecting the first user from the first user image and generating the first avatar image based on tracking information of the first user,
Controlling the communication unit to receive at least one of the second avatar image and information about the second avatar image from the other display device,
A display device that generates the second avatar image based on receiving information about the second avatar image. According to paragraph 4,
The at least one processor 120 executes the one or more instructions,
A display device that generates the group image by combining the first avatar image, the second avatar image, and the virtual background image. According to clause 4 or 5,
The at least one processor 120 executes the one or more instructions,
A display device that requests device information about the other display device from the server and controls the communication unit 150 to receive device information about the other display device from the server. According to clause 6,
The at least one processor 120 executes the one or more instructions,
A display device that controls the communication unit 150 to transmit at least one of the first avatar image or information about the first avatar image to the other display device based on device information about the other display device. According to any one of claims 3 to 7,
The at least one processor 120 executes the one or more instructions,
Based on the configuration method of the group service being determined as the server-based configuration method,
Controlling the communication unit 150 to transmit at least one of the first user image and information about the first user detected in the first user image to the server,
A display device that receives the group image including the first avatar image and the second avatar image from the server. According to any one of claims 1 to 8,
The at least one processor 120 executes the one or more instructions,
Receiving an input for selecting avatar information corresponding to the first avatar image,
The first avatar image is an avatar image generated based on the selected avatar information. According to any one of claims 1 to 9,
The display device is,
A display device further comprising a camera 110 that captures the first user and obtains an image of the first user. In a method of operating a display device,
Obtaining a first user image captured by a first user;
Transmitting capability information of the display device to a server;
Receiving, from the server, information about a configuration method of a group service determined based on the capability information;
Based on the received information, obtaining the first avatar image and a second avatar image corresponding to another display device user; and
A method of operating a display device, comprising displaying a group image including the first avatar image and the second avatar image. According to clause 11,
Capability information of the display device includes at least one of type information, capacity information, information about the number of cores, performance information, type information about the memory, capacity information, and performance information about the at least one processor included in the display device. A method of operating a display device, including one. According to claim 11 or 12,
The configuration method of the group service is,
A device-based configuration method in which the display device generates a first avatar image corresponding to the first user image, and the server generates the first avatar image and a second avatar corresponding to the other device user connected to the group service. A method of operating a display device that is determined by one of the server-based configuration methods for generating images. According to clause 13,
The step of acquiring the first avatar image and the second avatar image is,
Based on the configuration method of the group service being determined as the device-based configuration method,
detecting the first user from the first user image and generating the first avatar image based on tracking information of the first user;
Receiving at least one of the second avatar image and information about the second avatar image from the other display device; and
A method of operating a display device comprising: generating the second avatar image based on receiving information about the second avatar image. According to clause 14,
The operation method is,
A method of operating a display device further comprising generating the group image by combining the first avatar image, the second avatar image, and the virtual background image. According to claim 14 or 15,
The operation method is,
Based on the configuration method of the group service being determined as the device-based configuration method,
requesting device information about the other display device from the server; and receiving device information about the other display device from the server. According to clause 16,
The operation method is,
A method of operating a display device, further comprising transmitting at least one of the first avatar image or information about the first avatar image to the other display device based on device information about the other display device. According to any one of claims 13 to 17,
The step of acquiring the first avatar image and the second avatar image is,
Based on the configuration method of the group service being determined as the server-based configuration method,
transmitting at least one of the first user image and information about the first user detected in the first user image to the server; and
A method of operating a display device, comprising receiving the group image including the first avatar image and the second avatar image from the server. According to any one of claims 11 to 18,
The operation method is,
Receiving an input for selecting avatar information corresponding to the first avatar image; and
The first avatar image is an avatar image generated based on the selected avatar information. One or more computer-readable recording media storing a program for performing the method of any one of claims 11 to 19.

Images