KR102177777B1 - Avatar system based on multi layered display - Google Patents

Abstract

A multi-layered display-based avatar system is provided that allows participants of a teleconference to have a real presence. Such a multi-layered display-based avatar system includes a body, a display, and a control unit. The body portion includes a camera for photographing an image and a speaker for outputting audio. The display unit includes a plurality of transparent displays attached to an upper portion of the body unit and overlapping each other. Each of the plurality of transparent displays is defined as a plurality of gate wires extending in a first direction, a plurality of data wires extending in a second direction crossing the first direction, the gate wire, and the data wire. And a pixel, a gate driving circuit for driving the plurality of gate wires, and a data driving circuit for driving the plurality of data wires. The control unit controls the camera, the speaker, and the display unit.

Description

Avatar system based on multi-layer display {AVATAR SYSTEM BASED ON MULTI LAYERED DISPLAY}

The present invention relates to an avatar system for a participant in a teleconference, etc., and to a multi-layer display based avatar system capable of expressing a participant's face in three dimensions in more detail.

In the past, parties that were spatially distant were gathered together to hold a meeting, but in recent years, with the development of communication networks and electronic devices, remote video conferences are widely practiced in which meetings are held at each location without the need to attend meetings.

This teleconference is performed by exchanging video and audio contents with each other in real time at geographically dispersed locations.

For this, remote video conference, Korean Patent Registration No. 10-1771400, “Reminiscence Conference Method Using Meeting Room Objects”, Korean Patent No. 10-1732613, “Modification of Participants' Appearance During Video Video Conference”, Korean Patent Publication Various technologies, such as No. 10-2017-0030463, "Systems and methods for controlling a robotic stand during operation of a video conference", have been disclosed.

However, these conventional techniques display images of participants of the reminiscence conference in a flat image, so that the participants of the conference are less likely to actually attend. Therefore, there is a demand for a video conferencing system that allows participants of a conference to feel actually attended.

Korean Patent Registration No. 10-1771400 Korean Patent Registration No. 10-1732613 Republic of Korea Patent Publication No. 10-2017-0030463

Accordingly, the problem to be solved by the present invention is to provide a multi-layered display-based avatar system that allows participants of a teleconference to have an actual presence.

A multi-layer display-based avatar system according to an exemplary embodiment of the present invention for solving this problem includes a body portion, a display portion, and a controller. The body portion includes a camera for photographing an image and a speaker for outputting audio. The display unit includes a plurality of transparent displays attached to an upper portion of the body unit and overlapping each other. Each of the plurality of transparent displays is defined as a plurality of gate wires extending in a first direction, a plurality of data wires extending in a second direction crossing the first direction, the gate wire, and the data wire. And a pixel, a gate driving circuit for driving the plurality of gate wires, and a data driving circuit for driving the plurality of data wires. The control unit controls the camera, the speaker, and the display unit.

For example, the plurality of transparent displays may be disposed to contact each other.

In addition, among a plurality of transparent displays, the control unit may control the display with a smaller amount of light as it is closer to the surface.

In addition, the display unit may further include an opaque display attached to the rear end of the transparent displays.

Meanwhile, the control unit extracts feature points of a flat image photographed by the camera, calculates a positional relationship of the feature points, grasps a boundary corresponding to the feature points of the planar images, and according to the identified boundary portions, the image And the sliced images may be transmitted to each of the corresponding transparent displays.

Moreover, the control unit can divide the same area within the boundary into multiple layers.

On the other hand, the body portion may further include a microphone for receiving the voice of the other party.

In addition, the body portion may further include a beam projector for displaying materials.

In addition, the body portion may further include a transfer wheel for movement.

In addition, the body portion may further include a transmission/reception unit for connection with an external communication network.

As described above, the avatar system based on a multi-layer display according to the present invention can be implemented to express the faces of participants in a meeting in three dimensions, so that each party can have a feeling of attending the meeting.

The avatar system based on a multi-layer display according to the present invention can impart a three-dimensional effect by giving a sense of depth to a two-dimensional image even without taking a separate three-dimensional image.

In addition, among a plurality of transparent displays, by controlling the display with a smaller amount of light as it is closer to the surface, it is possible to display a uniform image by compensating for the amount of light absorbed by the transparent display.

Fig. 1 is a perspective view of an avatar system based on a multi-layer display according to an exemplary embodiment of the present invention.
FIG. 2 is a schematic perspective view of a display unit included in the multilayer display-based avatar system shown in FIG. 1.
FIG. 3 is a side view of the display unit illustrated in FIG. 2, and is a side view illustrating a case of displaying a spherical shape.
FIG. 4 is a front view of the display unit illustrated in FIG. 2, which is an exemplary front view for explaining a case of displaying a spherical shape.
5 is a circuit diagram illustrating an example of each transparent display constituting the display unit of FIG. 2.
FIG. 6 is a block diagram of an avatar system based on a multi-layer display shown in FIG. 1.
FIG. 7 is a flowchart illustrating a process of dividing a multi-layered sliced image for displaying a flat image captured by a camera on a display by the controller illustrated in FIG. 6.

In the present invention, various modifications may be made and various forms may be applied, and specific embodiments will be illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to a specific form disclosed, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention. In describing each drawing, similar reference numerals are used for similar components. In the accompanying drawings, dimensions of structures may be exaggerated than actual for clarity of the present invention.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or a combination thereof described in the specification, but one or more other features or It is to be understood that the presence or addition of numbers, steps, actions, components, parts, or combinations thereof does not preclude the possibility of preliminary exclusion. In addition, A and B are'connected' and'coupled' means that A and B are directly connected or combined, and other components C are included between A and B, so that A and B are connected or combined. To include that.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not. Further, in the claims for the method invention, the order of each step may be interchanged with each other, as long as each step is not clearly bound to the order.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

Fig. 1 is a perspective view of an avatar system based on a multi-layer display according to an exemplary embodiment of the present invention.

Referring to FIG. 1, a multilayer display based avatar system 100 according to an exemplary embodiment of the present invention includes a body 120, a display 110, and a control unit (not shown). Meanwhile, the body part 120 may further include at least one of a microphone 123, a beam projector 124, a transfer wheel 125, and a transmission/reception unit (not shown) for receiving the voice of the other party. .

In the drawings, the body portion 120 is configured in a hexahedral shape, but may be configured to have a shape such as an upper body of an actual human body.

The body portion 120 includes a camera 121 for photographing an image and a speaker 122 for outputting audio. In the case of a one-on-one meeting, assuming participants A and B, participant A is spatially with B's avatar system 100, and participant B is with A's avatar system 100.

At this time, the avatar system 100 of B captures the face of the participant A and displays it on the display unit 110 of the avatar system of A, and the avatar system 100 of A captures the face of the participant B. Is displayed on the display unit 110 of the avatar system 100. Similarly, the microphone 123 of the avatar system 100 of B receives the voice of A and transmits the voice to the avatar system 100 of A, and the voice is transmitted through the speaker 122 of the avatar system 100 of A. And the microphone 123 of the avatar system 100 of A receives the voice of B and transmits the voice to the avatar system 100 of B, and the voice is played through the speaker 122 of the avatar system 100 of B. do.

Meanwhile, when there are multiple conference participants, a camera and a microphone having separate IDs are provided for each participant, and a face shape and a voice may be given to the avatar system 100 having the same ID.

When it is necessary to display a power point or other data during a meeting, the beam projector 124 displays it on a screen so that the meeting proceeds more smoothly. The beam projector 124 may employ a conventional beam projector.

The transfer wheel 125 imparts mobility to the multi-layer display-based avatar system 100. In the drawing, the transfer wheel 125 employs an actual wheel, but in this embodiment, the transfer wheel 125 includes all devices that enable movement, such as an infinite orbit.

The display unit 110 is attached to the upper portion of the body unit 120. In the present embodiment, since the display unit 110 displays the face of a meeting participant, it is preferable to attach it to the upper portion of the body unit 120.

The display unit 110 includes a plurality of transparent displays overlapping each other. Each of the plurality of transparent displays is defined as a plurality of gate wires extending in a first direction, a plurality of data wires extending in a second direction crossing the first direction, the gate wire, and the data wire. And a pixel, a gate driving circuit for driving the plurality of gate wires, and a data driving circuit for driving the plurality of data wires. This structure will be described in more detail with reference to FIG. 5.

The controller (not shown) controls the microphone 123, the camera 121, the speaker 122, the beam projector 124, and the display unit 110. The control unit (not shown) will be described in more detail with reference to FIG. 6.

FIG. 2 is a schematic perspective view of a display unit included in the multilayer display-based avatar system shown in FIG. 1. FIG. 3 is a side view of the display unit shown in FIG. 2, which is an exemplary side view for explaining a case of displaying a spherical shape, and FIG. 4 is a front view of the display unit shown in FIG. It is a front view for explaining the case.

2 to 4, the display unit 110 includes a plurality of transparent displays d1, d2, d3, ..., dn overlapping each other.

For example, the plurality of transparent displays d1, d2, d3, ..., dn may be implemented as an organic light emitting diode (OLED) display. Alternatively, the plurality of transparent displays d1, d2, d3, ..., dn may be implemented as a liquid crystal display panel (LCD).

Meanwhile, the plurality of transparent displays d1, d2, d3, ..., dn may be spaced apart from each other at a slight interval, but in this embodiment, the plurality of transparent displays d1, d2, d3,. .., dn) can be arranged to be in contact with each other.

As described above, the plurality of transparent displays d1, d2, d3, ..., dn arranged to be in contact with each other may display different images while proceeding in the depth direction to represent a three-dimensional image. That is, the transparent display d1 closest to the front displays an image closest to the viewer, and the transparent display gradually distant from the front displays an image gradually distant from the viewer, thereby displaying a three-dimensional effect.

For example, as shown in Figs. 3 and 4, when displaying a spherical shape, the transparent display d1 closest to the front displays a circle with a radius r1, and then the transparent display d2 displays a radius r2 at a radius r1. Display the donut shape located in the space between, and then the transparent display d3 displays the donut shape located in the space between the radius r2 and the radius r3, and then the transparent display d4 displays the radius r4 at the radius r3. A donut shape positioned in the space between them is displayed.

The control unit (not shown) controls the display unit 110. The control unit (not shown) may control the next transparent display d4 to stop the display when the next transparent display d4 displays the maximum radius r4 at the radius r3, and the transparent display d5 or less may stop the display. (d5) displays an opaque monochromatic image (e.g., black), and the lower transparent displays (d6, d7,...) continuously display an image, and control to display an image from the rear. .

In addition, the control unit (not shown) may control the display with a smaller amount of light as it is closer to the surface among the plurality of transparent displays d1, d2, d3, ..., dn. That is, the displays d1 and dn closest to the surface are controlled to display with the least amount of light, and the displays d2 and dn-1 disposed next are controlled to display with the next less amount of light.

In this way, the control unit (not shown), among the plurality of transparent displays (d1, d2, d3, ..., dn), when controlling to be displayed with a smaller amount of light closer to the surface, uniformity of the screen displayed from the front You can pursue sex. That is, the light generated by the transparent display located deeply from the surface must penetrate relatively many transparent displays and is absorbed by each transparent display, so the control unit (not shown) increases the amount of light of the transparent display located deeply from the surface. Can be controlled.

Meanwhile, the display unit 110 may further include an opaque display (not shown) attached to the rear end of the transparent displays d1, d2, d3, ..., dn. In this way, when an opaque display is attached to the rear end, it is possible to prevent a phenomenon in which the image is penetrated and the rear surface is visible.

In this example, a case of displaying a spherical image has been described, but in actual case, a face photographed by a camera is displayed.

5 is a circuit diagram illustrating an example of each transparent display constituting the display unit of FIG. 2.

2 and 5, each of the transparent displays d1, d2, d3, ..., dn is a gate wiring GL1, GL2,..., and a data wiring DL1, DL2,... , A pixel, a gate driving circuit 10 and a data driving circuit 20.

The plurality of gate wires GL1, GL2,... extend in a first direction. For example, the plurality of gate wires GL1, GL2,... may extend in the x-axis direction.

The data lines DL1, DL2,... extend in a second direction, which is a direction crossing the first direction. For example, the data lines DL1, DL2,... may extend in the y-axis direction. The gate driving circuit 10 drives the plurality of gate wires GL1, GL2,..., and the data driving circuit 20 drives the plurality of data wires DL1, DL2,... do.

The pixel is defined by the gate lines GL1, GL2,... and the data lines DL1, DL2,.... In more detail, the pixel includes two gate wires adjacent to each other among the gate wires GL1, GL2,... and two data wires adjacent to each other among the data wires DL1, DL2,... Is defined as Accordingly, the pixels may be arranged in a matrix shape.

For example, when each of the transparent displays d1, d2, d3, ..., dn is composed of an organic light emitting diode (OLED) display, a switching element (SW) and a light emitting diode (LED) are formed in each pixel. I can.

For example, when a high voltage is applied to the first gate line GL1 through the gate driving circuit 10, the switching elements SW connected to the first gate line GL1 are activated. At this time, the data driving circuit 10 , Controls the amount of light generated by the light emitting diode (LED) through the voltage applied to the data lines (DL1, DL2, ...).

In this way, after the gate driving circuit 10 activates the switching elements SW connected to the first gate line GL1, the switching elements SW connected to the second gate line GL2 are activated, and a data driving cycle The furnace 10 controls the amount of light generated by the light emitting diode (LED) through a voltage applied to the data lines DL1, DL2, ....

Through this process, the entire image is displayed by adjusting the amount of light of the matrix-shaped pixel.

FIG. 6 is a block diagram of an avatar system based on a multi-layer display shown in FIG. 1.

Referring to FIG. 6, the control unit 200 of the multi-layer display-based avatar system includes a transmission/reception unit 210, a display unit 110, a camera 121, a microphone 123, a speaker 122, and a beam projector 124. And it is connected to the transfer wheel 125 to control each.

As described above, the control unit 200 transmits/receives image and audio data received through the camera 121 and the microphone 123 through the transceiving unit 210 to the transceiving unit 210 of another multi-layer display-based avatar system. ), the image and audio data transmitted through the transmission/reception unit 210 of another multi-layer display-based avatar system are received through the transmission/reception unit 210, and the image data is processed by the control unit, and the display unit It is displayed through 110, and voice data is reproduced through the speaker 122. Processing of such image data will be described in more detail through FIG. 7.

Meanwhile, as described above, the image data and voice data may be data acquired through other cameras and microphones, not the camera 121 and microphone 123 attached to the multi-layer display-based avatar system.

In addition, when it is necessary to project and display data such as a power point, the control unit 200 changes the direction toward the screen through the transfer wheel 125, and then controls to perform the projection through the beam projector 124. can do.

FIG. 7 is a flowchart illustrating a process of dividing a sliced image into multiple layers for displaying a flat image captured by a camera on the display unit by the controller illustrated in FIG. 6.

6 and 7, the control unit 200 first extracts feature points of a plane image captured by the camera 121 and received (step S110). For example, when the multi-layer-based stereoscopic image forming apparatus according to the present invention is used in an avatar device for a remote conference, the multi-layer-based stereoscopic image forming apparatus may display face images of conference participants. When the flat image is a face, a part of white is arranged in an oval shape outside the circular black color, and when a similar shape exists at a location spaced apart by a certain distance, it can be recognized as a feature point and recognized as an eye.

Thereafter, the positional relationship of the feature points is calculated (step S120). For example, extracted feature points, that is, feature points corresponding to the eyes, are extracted, and their positional relationship is calculated to grasp nose, mouth, and the like. That is, a part extending from the center between the two eyes to the bottom is recognized as a nose, and a feature located at the lower part of the nose is recognized as a mouth.

In this way, after specifying the image corresponding to the feature points of each part, such as eyes, nose, mouth, and ears, from the planar images, the boundary is identified (step S130). As the neighbors of the feature points are expanded, the point where the sudden change occurs is determined as the boundary.

Thereafter, the image is sliced according to the identified boundary portions (step S140). At this time, in order to give a more three-dimensional effect, the control unit 200 may divide the same area within the boundary into multiple layers. That is, even in the case of the face of the face, it can be sliced to retreat toward the boundary of the outer face rather than the position of the nose.

Thereafter, the control unit 200 may transmit respective images corresponding to the sliced images to another multi-layer display-based avatar system through the transmission/reception unit 210 and display a three-dimensional image of the face through the transparent display. . Meanwhile, in this embodiment, the photographed image is sliced before being transmitted and transmitted to another multi-layer display-based avatar system. However, differently, after transmitting the photographed flat image and slicing it in the received multi-layer display-based avatar system, , It is obvious to those skilled in the art that it may be displayed through the display unit.

As described above, the avatar system based on a multi-layer display according to the present invention can be implemented to express the faces of participants in a meeting in three dimensions, so that each party can have a feeling of attending the meeting.

Meanwhile, the above-described multi-layer display-based avatar system has been described in an example applied to a teleconference, but may be applied in various fields differently. For example, a multi-layer display-based avatar system can be applied to an artificial intelligence assistant. In other words, artificial intelligence systems currently used in homes are applied by adopting a shape completely different from that of humans, but by using a number of transparent displays that display human faces three-dimensionally, they display more human-like shapes and facial expressions. You can also implement an intimate AI assistant.

In the detailed description of the present invention described above, it has been described with reference to preferred embodiments of the present invention, but those skilled in the art or those of ordinary skill in the art will have the spirit of the present invention described in the claims to be described later. And it will be appreciated that various modifications and changes can be made to the present invention within a range not departing from the technical field.

100: Avatar system based on multi-layer display
110: display unit 120: body
121: camera 122: speaker
123: microphone 124: beam projector
125: transfer wheel
10: gate driving circuit 20: data driving circuit
200: control unit 210: transmission/reception unit
DL1, DL2: data line GL1, GL2: gate line
SW: switching element

Claims (5)

A body portion including a camera for photographing an image and a speaker for outputting sound;
Defined as a plurality of gate wires attached to the upper part of the body, each extending in a first direction, a plurality of data wires extending in a second direction crossing the first direction, the gate wire and the data wire A display unit in which a plurality of transparent displays including a pixel to be formed, a gate driving circuit for driving the plurality of gate lines, and a data driving circuit for driving the plurality of data lines are overlapped to each other; And
A control unit controlling the camera, the speaker, and the display unit;
Including,
The control unit controls the display of the plurality of transparent displays in a smaller amount of light as they are closer to the surface.
delete The method of claim 1,
The control unit,
Extracting feature points of a flat image photographed by the camera,
Calculate the positional relationship of the feature points,
Grasp the boundary corresponding to the feature points of flat images,
Slice the image according to the identified boundaries,
A multi-layer display-based avatar system, characterized in that the sliced images are transmitted to each of the corresponding transparent displays.
The method of claim 3,
The control unit,
A multi-layer display-based avatar system, characterized in that the same area within the boundary is divided into multiple layers.
The method of claim 1,
The body part,
A microphone for receiving the voice of the other party;
A beam projector for displaying materials;
Transfer wheel for movement; And
Transmitting and receiving unit for connection with an external communication network;
A multi-layer display based avatar system, further comprising at least one of.

Images