KR102172985B1 - Display apparatus and method for displaying graphic item using the same - Google Patents

Abstract

A method of displaying a graphic element is disclosed. The present graphic element display method includes storing a graphic element overlapping a reference face area, detecting a user's face area included in an image, and depending on the direction in which the detected face area faces based on the direction the reference face area faces. Deforming the graphic element, and displaying the deformed graphic element by overlapping the face region in the image.

Description

Display device and its graphic element display method {DISPLAY APPARATUS AND METHOD FOR DISPLAYING GRAPHIC ITEM USING THE SAME}

The present invention relates to a display device and a method for displaying graphic elements thereof, and more particularly, to a display device for displaying a graphic element by overlapping an image and a method for displaying graphic elements thereof.

With the recent development of electronic technology, display devices such as mobile phones and TVs provide a function of overlapping various graphic elements on the face of the other party when performing a video call with the other party. For example, various graphic elements such as masks and mustaches are overlapped on the other's face area to induce user interest.

However, since the conventional display device overlaps graphic elements on the premise that the other party is facing the front, there is a problem that the overlapped graphic elements are not natural when the other party's face is rotated.

Therefore, even when the other person's face is rotated, it is required to find a plan for displaying by naturally overlapping graphic elements on the rotated face.

The present invention is in accordance with the above-described necessity, and an object of the present invention is to provide a display device capable of deforming a graphic element according to a direction in which a rotated face area faces and overlapping the deformed graphic element on a face area, and a graphic element display method thereof. Is in.

According to an embodiment of the present invention for achieving the above object, a method of displaying a graphic element of a display device includes storing a graphic element overlapping a reference face area, detecting a user's face area included in the image. And transforming the graphic element according to a direction in which the detected face area is directed based on a direction in which the reference face area is facing, and displaying the modified graphic element by overlapping the face area in the image. do.

Here, the graphic element display method according to the present embodiment includes the steps of capturing the image according to a user command, detecting a user's face area included in the captured image, and inputting to overlap the captured image. The further step of storing the overlapped position information of the graphic element and the input graphic element in the face area of the user included in the captured image may be further included.

In the transforming step, the user's face region included in the captured image may be set as the reference face region, and the shape of the graphic element may be changed.

The transforming may include setting first and second reference points for each of the reference face region and the detected face region based on a preset face component, and the second reference point based on the first reference point. It may include determining a relative position of the reference point, and deforming the graphic element based on the determined relative position.

In addition, the transforming may include dividing the reference face region into a plurality of regions based on the first reference point, dividing the detected face region into a plurality of regions based on the second reference point, and Comprising the size of each of the plurality of regions divided based on the first reference point and the plurality of regions divided based on the second reference point, and changing the shape of the graphic element based on the comparison result; and can do.

In the displaying, the reference position of the graphic element whose shape has been changed may be matched with the second reference point to overlap the detected face area and displayed.

Meanwhile, the display device according to an embodiment of the present invention includes a storage unit for storing graphic elements overlapping a reference face area, a display unit for displaying an image, a detection unit for detecting a user's face area included in the image, and And a controller configured to transform the graphic element according to a direction in which the detected face area faces based on a direction in which the reference face area faces, and to display the transformed graphic element by overlapping the face area in the image.

Here, the display unit captures and displays the image according to a user command, the detection unit detects a user's face area included in the captured image, and the control unit inputs the captured image to overlap The storage unit may be controlled to store the overlapped position information of the graphic element and the input graphic element in the face area of the user included in the captured image.

In addition, the controller may set the user's face region detected in the captured image as the reference face region.

In addition, the control unit sets first and second reference points for each of the reference face region and the detected face region based on a preset face component, and the second reference point is relative to the first reference point. The position may be determined, and the graphic element may be deformed based on the determined relative position.

In addition, the control unit divides the reference face region into a plurality of regions based on the first reference point, divides the detected face region into a plurality of regions based on the second reference point, and divides the first reference point into a plurality of regions. The size of each of the plurality of regions divided based on the reference point and the plurality of regions divided based on the second reference point may be compared, and the shape of each region of the graphic element may be changed based on the comparison result.

In addition, the controller may control to match the reference position of the graphic element whose shape has been changed with the second reference point to overlap and display the detected face area.

According to various embodiments of the present disclosure as described above, even when the other person's face is rotated, natural graphic elements may be overlapped and displayed. Therefore, the user's interest can be further increased.

1 is a block diagram illustrating a configuration of a display device according to an embodiment of the present invention;
2 is a view for explaining a reference face area and graphic elements according to an embodiment of the present invention;
3 to 7 are diagrams for explaining a method of transforming a graphic element according to an embodiment of the present invention;
8 is a flowchart illustrating an example of receiving a graphic element from a user according to an embodiment of the present invention;
9 is a block diagram illustrating a detailed configuration of a display device according to an embodiment of the present invention;
10 to 17 are diagrams for explaining a method of overlapping and displaying graphic elements according to an embodiment of the present invention, and
18 is a flowchart illustrating a method of displaying a graphic element according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

1 is a block diagram illustrating a configuration of a display device according to an embodiment of the present invention. Referring to FIG. 1, the display apparatus 100 includes a storage unit 110, a display unit 120, a detection unit 130, and a control unit 140.

The storage unit 110 stores graphic elements overlapping the reference face area. Refer to FIG. 2 for a more detailed description.

The graphic element may include various types of items that may cause interest or fun to the user by overlapping with the user's face area included in the image. For example, as shown in (a) of FIG. 2, the graphic element 210 may include various types of graphic elements that may overlap the user's face area, such as a mask, beard, horn, wound, and glasses.

In addition, the reference face region may be a face region capable of realizing a natural image when graphic elements are overlapped and displayed. In general, since the graphic element is matched to and stored in a face area facing the front direction, the reference face area 220 may be a face area facing the front direction as shown in FIG. 2B.

Meanwhile, the graphic element may have a size corresponding to the size of the reference face area. That is, when the graphic element overlaps the reference face region, the graphic element may have a size suitable for the size of the reference face region so that it may naturally overlap.

In addition, the storage unit 110 may store location information in which the graphic element overlaps the face area. Specifically, since the positions at which the graphic elements overlap the face area are different according to the type of the graphic element, the storage unit 110 may store information on the overlapping positions on the face area for each graphic element.

In this case, the storage unit 110 may store information on a point where the reference position of the graphic element is located within the face area. Here, the reference position of the graphic element may vary depending on the type of the graphic element.

For example, when the graphic element is the mask 211 as shown in (c) of FIG. 2, the storage unit 110 includes a horizontal line connecting two eyes in which the reference position 211-1 of the mask is included in the reference face area. Information on a position where the mask overlaps may be stored so that 221 and the vertical line 222 connecting the nose included in the reference face region meet each other at a point 223.

As another example, as shown in (d) of FIG. 2, when the graphic element is the mustache 212, the storage unit 110 may have a mustache such that the reference position 211-2 of the mustache is located at the center 224 between the nose and the mouth. Information about this overlapping position can be stored.

To this end, the storage unit 110 includes a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (for example, SD or XD memory). Etc.), RAM, and ROM may include at least one type of storage medium.

The display unit 120 may display an image. Here, the image may be a still image or a moving image. In particular, in the case of a video, the image may be each image frame of a video call image received from another display device (not shown) by performing a video call with another display device (not shown).

Also, the image may be pre-stored in the display device 100 or may be received from the outside. For example, when received from the outside, the display device 100 may perform a video call with another display device (not shown) to receive and output a video call image from another display device (not shown) through a network. .

To this end, the display unit 120 may be implemented as a Liquid Crystal Display Panel (LCD), Organic Light Emitting Diodes (OLED), or the like, but is not limited thereto.

Meanwhile, the display unit 120 may be implemented in the form of a touch screen forming a layer structure with a touch pad. In this case, the display unit 120 can be used as an input device to receive input of various user commands other than the output device. Here, the touch screen may be configured to detect not only a touch input location and area, but also a touch input pressure.

In this case, the display unit 120 may display various GUIs for receiving user commands.

The detection unit 130 detects a face area included in the image. For example, the detection unit 130 may use a feature-based face detection method. Specifically, the detection unit 130 may detect feature points (e.g., facial elements such as eyes, nose, mouth, etc., texture, flesh color), etc. in the face, and detect a face area included in the image using the detected feature points. have. Among the various features of a face, skin color, in particular, is less sensitive to movement, rotation, and size change of the face, and thus is a method that is widely used in the field of face recognition.

As another example, the detection unit 130 may detect a face region in an image using a support vector machine (SVM). SVM-based methods are a method of sub-sampling different areas from an image, learning about the face area and non-face area (non-face area) through a learner, and then finding a face in the input image.

As another example, the detection unit 130 may detect a face area within an image using a face template-based detection method. The face template-based detection method creates several standard patterns for faces and then stores the patterns for face detection. Then, the patterns are compared with the image one by one in the search window of the image, and a face is detected.

Since the methods used by the detection unit 130 to detect the face area are already known, detailed descriptions are omitted. However, even with the above-described exceptions, it goes without saying that the detection unit 130 may detect a face area within an image using various known methods.

In this way, the detection unit 130 may detect a face region within an image and an eye, a nose, a mouth, and an ear within the face region through various methods. In addition, the detection unit 130 may detect the size and rotation state of the face area.

The controller 140 controls the overall operation of the display device 100.

In particular, the controller 140 may control the graphic element to be deformed according to the direction in which the detected face region faces based on the direction in which the reference face region faces, and to display the transformed graphic element by overlapping the face region in the image.

The controller 140 sets a first reference point and a second reference point for each of the reference face area and the detected face area based on a preset face component, and determines a relative position of the second reference point based on the first reference point. , The graphic element may be deformed based on the determined relative position. Here, the facial component may be an element that conveys a face such as eyes, nose, and mouth.

Hereinafter, a detailed method of transforming a graphic element will be described with reference to FIGS. 3 to 7.

3 to 7 are diagrams for explaining a method of transforming a graphic element according to an embodiment of the present invention.

First, FIG. 3 is a flowchart illustrating a method of transforming a graphic element according to an embodiment of the present invention. In describing FIG. 3, for convenience of description, reference will be made to FIGS. 4 to 7 together.

The controller 140 may divide the reference face region into a plurality of regions based on the first reference point (S310), and may divide the face region detected based on the second reference point into a plurality of regions (S320).

For example, as shown in (a) of FIG. 4, the control unit 140 connects a horizontal line 412 connecting two eyes included in the reference face area 410 and a nose included in the reference face area 410. A point where the vertical line 413 meets may be set as the first reference point 411. In this case, the first reference point 411 may be the center of both eyes in the reference face area 410.

In addition, the controller 140 may set the second reference point on the detected face area as shown in FIGS. 4B to 4D. 4B to 4D illustrate cases in which the detected face regions 420, 430, 440, and 450 face left, right, upper, and lower directions, respectively.

Specifically, the control unit 140 includes horizontal lines 422, 432, 442, 452 connecting two eyes included in the detected face areas 420, 430, 440, 450 and the detected face areas 420. A point where the vertical lines 423, 433, 443, and 453 connecting the nose meet may be set as the second reference points 421, 431, 441, and 451. In this case, the second reference points 421, 431, 441, and 451 may be the centers of the two eyes in the detected face regions 420, 430, 440, and 450.

Here, the first reference point 411 is set as the center of the two eyes in the reference face area 410, and the face areas 420, 430, 440, 450 in which the second reference points 421, 431, 441, and 451 are detected. ), which is set as the center of the two eyes may vary depending on the type of graphic element selected by the user to overlap the detected face area.

That is, the controller 140 may set the first reference point and the second reference point based on the reference position of the graphic element selected by the user. For example, the controller 140 determines the reference position of the graphic element by using information on the point where the reference position of the graphic element selected by the user is located within the face area, and determines the determined reference position to the first reference point and It can be set as a second reference point.

For example, as described in (c) of FIG. 2, the mask has a point where a horizontal line connecting two eyes and a vertical line connecting the nose meet (ie, the center of the two eyes) as a reference position. Therefore, when the user selects a mask as a graphic element to be overlapped, the controller 140 sets the centers of the two eyes in the reference face area 410 as the first reference point 411, and the detected face areas 420, 430, and At 440 and 450), the centers of the two eyes are set as the second reference points 421, 431, 441, and 451.

Accordingly, when another graphic element is selected, the controller 140 may set a second reference point corresponding to the type of the selected graphic element. For example, as shown in (d) of FIG. 2, when the mustache is selected, the controller 140 may set the center between the nose and the mouth as a first reference point and a second reference point.

Meanwhile, hereinafter, a case where the graphic element to be overlapped with the detected face area is a mask will be described as an example.

Meanwhile, before setting a reference point for each of the reference face region and the detected face region, the controller 140 may scale the reference face region or the detected face region so that the size of each face region becomes the same. For example, when the size of the detected face area is larger than the reference face area, the controller 140 may downscale the detected face area to have the same size as the reference face area. Alternatively, the controller 140 may upscale the reference face region to have the same size as the detected face region.

In addition, the controller 140 divides the reference face area 410 in the horizontal direction based on the first reference point 413 and divides the reference face area 410 in the vertical direction based on the first reference point 413, The reference face area 410 may be divided into four.

Further, the controller 140 divides the detected face regions 420, 430, 440, 450 horizontally based on the second reference points 421, 431, 441, and 451, and divides the second reference points 421, 431, The detected face regions 420, 430, 440, 450 based on 441 and 451 may be divided in a vertical direction, and the detected face regions 420, 430, 440 and 450 may be divided into four.

Meanwhile, in FIG. 4, for convenience of explanation, each divided area of the reference face area 410 is indicated as 1,2, 3, and 4, and each divided area of the detected face area 420, 430, 440, 450 is indicated. Marked as 1',2',3',4'.

Then, the controller 140 determines the direction in which the detected face area is rotated (S330). That is, the controller 140 determines the direction in which the detected face area is rotated based on the reference face area.

That is, as shown in (a) of FIG. 5, when the control unit 140 rotates the reference face region 410 around any of the x-axis and y-axis, the reference face region 410 is detected face regions 420 and 430. , 440, 450 may be in a direction, and the direction in which the detected face regions 420, 430, 440, and 450 are rotated with respect to the reference face region 410 may be determined.

In this case, the controller 140 may determine the direction in which the detected face area is rotated using the relative positional information of the second reference point based on the first reference point. Specifically, when the second reference point is located above or below the first reference point, the control unit 140 may determine that the detected face area is rotated around the x-axis, and the second reference point is to the left of the first reference point or When positioned on the right, it may be determined that the detected face area is rotated around the y-axis.

For example, as shown in (b) of FIG. 5, when the second reference point 421 is located on the left side of the first reference point 411, the control unit 140 determines that the detected face area 420 is on the y-axis. The rotation direction of the face area 420 detected as being rotated around the center may be determined.

In addition, as shown in (c) of FIG. 5, when the second reference point 431 is located on the right side with respect to the first reference point 411, the controller 140 determines that the detected face area 420 is centered on the y-axis. The rotation direction of the face area 430 detected as being rotated may be determined.

In addition, as shown in (d) of FIG. 5, when the second reference point 441 is located above the first reference point 411, the controller 140 determines that the detected face area 440 is centered on the x-axis. The direction of rotation of the face region 440 detected as being rotated may be determined.

In addition, as shown in (e) of FIG. 5, when the second reference point 451 is located below g with respect to the first reference point 411, the controller 140 determines that the detected face area 450 is centered on the x-axis. The rotation direction of the face area 450 detected as being rotated may be determined.

Meanwhile, in order to change the size of the graphic element, the control unit 140 may divide the graphic element into a plurality of areas. In this case, the controller 140 may divide the graphic element into a plurality of areas based on the reference position of the graphic element.

For example, as shown in (a) of FIG. 6, the controller 140 divides the mask 460 in a horizontal direction based on the reference position 461 and vertically divides the mask 460 based on the reference position 461. By dividing in the direction, the mask 460 can be divided into four equal parts. Meanwhile, for convenience of description, each divided area of the mask 460 is indicated as 1 ″, 2 ″, 3 ″, 4 ″.

Further, the controller 140 may compare the sizes of each of the plurality of regions divided based on the first reference point and the plurality of regions divided based on the second reference point, and change the size of the graphic element based on the comparison result. .

Specifically, the control unit 140 compares the size of each divided area matched between the reference face area and the detected face area, and enlarges or expands each divided area of the detected face area based on each divided area of the reference face area. Determine the degree of reduction. Here, each of the divided regions to be matched means each divided region positioned in the same direction with respect to each of the first reference point and the second reference point. That is, in FIG. 4, each of the divided areas 1', 2', 3', and 4'of the detected face areas 420, 430, 440, and 450 are divided areas 1, 2, and 3 of the reference face area 410. It can be seen as matching each of ,4.

In addition, the control unit 140 may change the shape of the graphic element by increasing or decreasing the divided area of the matching graphic element according to the determined degree of enlargement or reduction. Here, the divided areas of matching graphic elements means each divided area located in the same direction based on each of the first reference point and the reference position. That is, in FIGS. 4 and 6, each divided area 1 ″, 2 ″, 3 ″, 4 ″ of the graphic element 460 is divided into 1, 2, 3, and 3 of the reference face area 410, It can be seen as matching each of 4.

Specifically, the controller 140 determines the direction in which the detected face area is rotated in order to change the shape of the graphic element (S340-Y).

Accordingly, if it is determined that the detected face area is rotated around the y-axis (S340-Y), the controller 140 compares the size of each divided area divided based on the vertical line (S351). That is, the controller 140 includes the size of the area 1 + area 3 divided based on the vertical line in the reference face area 410 and the area 1 divided based on the vertical line in the detected face areas 420, 440, 440, 450 Compare the size of'+ area 3'. In addition, the controller 140 includes the size of the area 2 + area 4 divided based on the vertical line in the reference face area 410 and the area 2 divided based on the vertical line in the detected face areas 420, 440, 440, 450. Compare the size of'+area 4'.

Accordingly, if the control unit 140 determines that the size of the detected face area area 1'+ area 3'is smaller than the size of the reference face area area 1 + area 3 (S352-Y), the graphic element area 1 The shape of the graphic element can be changed by reducing ``+ area 3'' and increasing area 2'' + area 4'' (S353).

That is, when the detected face area 420 looks to the left as shown in FIG. 4B, the size of area 1'+ area 3'of the detected face area 420 is the size of the reference face area 410. It becomes smaller than area 1 + area 3. In this case, the control unit 140 reduces the area 1'' + area 3'' of the graphic element and increases the area 2'' + area 4'' as shown in (b) of FIG. 6, so that the shape of the graphic element 470 ) Can be created.

In this case, the controller 140 increases or decreases the area divided based on the vertical line in the graphic element, based on the extent to which the size of the area divided based on the vertical line in the detected face area 420 is enlarged and reduced. I can.

For example, in FIG. 4, the size of area 1'+ area 3'of the detected face area 420 is reduced by a% based on the size of area 1 + area 3 of the reference face area 410, and detection It is assumed that the size of area 2'+ area 4'of the face area 420 is enlarged by b% based on the size of area 2'+ area 4'of the reference face area 410. In this case, the controller 140 may reduce area 1 ″ + area 3 ″ of the graphic element by a% and increase area 2 ″ + area 4 ″ by b%.

On the other hand, if the control unit 140 determines that the size of the detected face area area 1'+ area 3'is larger than the size of the reference face area area 1 + area 3 (S352-N), the graphic element area 1 The shape of the graphic element may be changed by increasing ``+ area 3'' and reducing area 2'' + area 4'' (S354).

That is, when the detected face area 430 looks to the right as shown in FIG. 4C, the size of area 1'+ area 3'of the detected face area 430 is the size of the reference face area 410. It becomes larger than area 1 + area 3. In this case, the control unit 140 increases the area 1'' + area 3'' of the graphic element and decreases the area 2'' + area 4'' as shown in (c) of FIG. 6, so that the shape of the graphic element 480 ) Can be created.

In this case, the control unit 140 increases or decreases the area divided based on the vertical line in the graphic element based on the extent to which the size of the area divided based on the vertical line in the detected face area 430 is enlarged and reduced. I can.

For example, in FIG. 4, the size of area 1'+ area 3'of the detected face area 430 is enlarged by a% based on the size of area 1 + area 3 of the reference face area 410, and detection It is assumed that the size of the area 2'+ area 4'of the face area 430 is reduced by b% based on the size of the area 2'+ area 4'of the reference face area 410. In this case, the controller 140 may increase the area 1 ″ + 3 ″ of the graphic element by a% and decrease the area 2 ″ + the area 4 ″ by b%.

Meanwhile, if it is determined that the detected face region is rotated around the x-axis (S340-Y), the controller 140 compares the size of each divided region divided based on the horizontal line (S361). That is, the control unit 140 includes the size of the area 1 + area 2 divided by the horizontal line in the reference face area 410 and the area 1 divided by the horizontal line in the detected face areas 420, 440, 440, 450 Compare the size of'+ area 2'. In addition, the controller 140 includes the size of the area 3 + area 4 divided based on the horizontal line in the reference face area 410 and the area 3 divided based on the horizontal line in the detected face areas 420, 440, 440, 450. Compare the size of'+area 4'.

Accordingly, if the control unit 140 determines that the size of the detected face area area 1'+ area 2'is smaller than the size of the reference face area area 1 + area 2 (S362-Y), the graphic element area 1 The shape of the graphic element can be changed by reducing ``+ area 2'' and increasing area 2'' + area 4'' (S363).

That is, when the detected face area 440 looks upward as shown in FIG. 4(d), the size of the area 1'+ area 2'of the detected face area 440 is that of the reference face area 410. It becomes smaller than the size of area 1 + area 2. In this case, the controller 140 reduces the area 1'' + area 2'' of the graphic element and increases the area 3'' + area 4'' as shown in (d) of FIG. 6, so that the graphic element 490 ) Can be created.

In this case, the control unit 140 increases or decreases the area divided based on the horizontal line in the graphic element based on the extent to which the size of the area divided based on the horizontal line in the detected face area 440 is enlarged and reduced. I can.

For example, in FIG. 4, the size of area 1'+ area 2'of the detected face area 440 is reduced by a% based on the size of area 1 + area 2 of the reference face area 410, and detection It is assumed that the size of the area 3'+ area 4'of the face area 440 is enlarged by b% based on the size of the area 3'+ area 4'of the reference face area 410. In this case, the controller 140 may reduce the area 1 ″ + area 2 ″ of the graphic element by a% and increase the area 3 ″ + the area 4 ″ by b%.

Meanwhile, if it is determined that the size of the detected face area area 1'+ area 2'is larger than the size of the reference face area area 1 + area 2 (S362-N), the control unit 140 The shape of the graphic element can be changed by increasing ``+ area 2'' and reducing area 2'' + area 4'' (S364).

That is, as shown in (e) of FIG. 4, when the detected face area 450 looks downward, the size of the area 1'+ area 2'of the detected face area 450 is that of the reference face area 410 It becomes larger than the size of area 1 + area 2. In this case, the controller 140 increases the area 1'' + area 2'' of the graphic element and decreases the area 3'' + area 4'' as shown in (e) of FIG. ) Can be created.

In this case, the control unit 140 increases or decreases the area divided based on the horizontal line in the graphic element based on the extent to which the size of the area divided based on the horizontal line in the detected face area 450 is enlarged and reduced. I can.

For example, in FIG. 4, the size of area 1'+ area 2'of the detected face area 450 is enlarged by a% based on the size of area 1 + area 2 of the reference face area 410, and detection It is assumed that the size of area 3'+ area 4'of the face area 450 is reduced by b% based on the size of area 3'+ area 4'of the reference face area 410. In this case, the controller 140 may increase the area 1 ″ + area 2 ″ of the graphic element by a% and decrease the area 3 ″ + the area 4 ″ by b%.

In addition, the controller 140 may control to match the reference position of the graphic element whose shape has been changed with the second reference point to overlap the detected face area and display it. That is, the controller 140 may overlap and display the graphic element on the detected face area so that the reference position of the graphic element whose shape has been changed is positioned at the second reference point in the detected face area.

For example, FIG. 7 shows a case where a mask is overlapped on a face area.

Specifically, (a) of FIG. 7 is a case where the mask 460 is overlapped with the reference face region 410, and (b) to (e) of FIG. 7 are detected face regions 420, 430, 440, and 450. ) Shows the case of overlapping the modified masks 470, 480, 490, and 510, respectively.

In this case, the controller 140 controls the second reference points 421, 431, 441, in the face regions 420, 430, 440, 450 in which the reference positions 471, 481, 491, and 511 of the mask whose shape has been deformed are detected. The masks 470, 480, 490, and 510 whose sizes have been deformed so as to be positioned on the 451 may be displayed by overlapping the detected face regions 420, 430, 440, and 450, respectively.

Meanwhile, in the above-described example, it has been described that the detected face area is rotated around the x-axis or y-axis in the reference face area, but this is only an example. That is, even when the x-axis and y-axis, that is, rotated around both axes, the method described in FIGS. 3 to 7 may be applied in the same manner. That is, the control unit 140 increases or decreases area 1 + area 3 and area 2 + area 4 of the graphic element based on the size of each divided area, and increases area 1 + area 2 and area 3 + area 4 of the graphic element. By reducing or decreasing, the shape of the graphic element can be changed.

In addition, in the above-described example, a case in which a mask is selected as a graphic element to be overlapped, that is, a reference position of the graphic element is the same as the first reference point. However, even for a graphic element whose reference position is not the same as the first reference point, the control unit 140 may change its shape to correspond to the detected face area.

For example, as shown in Fig. 2(d), the mustache is such a case. In this case, the control unit 140 sets the center between the nose and the mouth in the reference face area as a first reference point, and sets the center between the nose and the mouth in the detected face area as the first reference point, and is the same method as described above. You can change the shape of the graphic element by using.

However, when the detected face area faces upward and downward, the controller 140 does not divide or increase or decrease the upper or lower part of the graphic element as shown in FIGS. 6(c) and (e). You can increase or decrease the whole.

That is, if it is determined that the size of the detected face area area 1'+ area 2'is smaller than the size of the reference face area area 1 + area 2, the controller 140 may increase the entire graphic element in the vertical direction. . In this case, the controller 140 may increase the entire graphic element in the vertical direction based on the degree to which the size of the area 1'+ area 2'is reduced.

In addition, if it is determined that the size of the detected face area area 1'+ area 2'is larger than the size of the reference face area area 1 + area 2, the controller 140 may reduce the entire graphic element in the vertical direction. . In this case, the controller 140 may reduce the entire graphic element in the vertical direction based on the degree to which the size of the area 1'+ area 2'is reduced.

Further, in the above-described example, it has been described that the shape of a previously stored graphic element is changed, but this is only an example. That is, the controller 140 may receive a graphic element by a user, change the shape of the input graphic element, and control the detected face area to overlap and display.

8 is a flowchart illustrating an example of receiving a graphic element from a user according to an embodiment of the present invention.

First, when a user command for executing the graphic element input mode is input (S610), the controller 140 may capture an image according to the user command and display it on the display 120. For example, when a user command for capturing an image being displayed on the display unit 120 is input, the controller 140 displays the captured image so that the captured image is displayed together with the previously displayed image. 120) may be displayed.

In this case, the detection unit 130 detects the user's face area included in the captured image. In this case, the detection unit 130 may detect the user's face region included in the captured image using various methods described in FIG. 1.

In addition, when a user command for terminating the graphic element input mode is input (S620-Y), the controller 140 determines that the input of the graphic element has been completed. In this case, the control unit 140 controls the storage unit 110 to store the input graphic element to overlap the captured image and the location information of the user's face included in the captured image. Can be (S630).

For example, when the display unit 120 is implemented in the form of a touch screen, the user may overlap and draw a graphic element on an image captured through touch manipulation. In this case, the controller 140 may control a certain line or surface to be displayed along a point where a user's touch manipulation is performed.

In addition, the controller 140 may store a graphic element input by a user in the storage unit 110. In this case, the controller 140 may match the positional information in which the graphic element input by the user overlaps the face area, and store it in the storage unit 110 by matching each graphic element.

Specifically, the controller 140 may set a reference point of a graphic element input by a user, and store location information on a point where the reference point of the graphic element overlaps in the face area in the captured image.

For example, when a graphic element input by a user includes a center between two eyes, the controller 140 sets a point passing through the center between two eyes in the graphic element as a reference point, and the reference point is the center of the two eyes. It is possible to store the location information for the graphic element input to be located at. As another example, when the graphic element input by the user includes the center between the nose and the mouth, the controller 140 sets the center between the nose and the mouth in the graphic element as a reference point, and the reference point is between the nose and the mouth. Information on the input graphic elements can be stored to be located in the center of the.

Then, the controller 140 displays the graphic element input by the user by overlapping the face area detected in the image (S640).

To this end, the controller 140 may set the user's face area included in the captured image as the reference face area. In addition, the controller 140 may control a graphic element to be deformed according to a direction in which the detected face region faces based on a direction in which the newly set reference region faces, and to overlap and display the transformed graphic element on the face region in the image. .

Here, the graphic element may be a graphic element input by a user.

Specifically, the controller 140 sets first and second reference points for each of the reference face area and the detected face area based on a preset face component, and determines the relative position of the second reference point based on the first reference point. It is determined, and the graphic element can be transformed based on the determined relative position.

In this case, the controller 140 divides the reference face region into a plurality of regions based on the first reference point, divides the face region detected based on the second reference point into a plurality of regions, and divides the face region based on the first reference point. The sizes of each of the plurality of areas and the plurality of areas divided based on the second reference point may be compared, and the shape of the graphic element may be changed based on the comparison result.

In addition, the controller 140 may control to match the reference position of the graphic element whose shape has been changed with the second reference point to overlap the detected face area and display it.

That is, when a graphic element is input by the user, the controller 140 sets the face area detected in the captured image for inputting the graphic element as the reference face area, and determines the direction in which the face area detected in the captured image faces. The graphic element may be transformed according to the direction in which the face region detected as a reference faces.

In this case, since the method of changing the shape of the graphic element is the same as in the case where the reference face region is the face region detected in the captured image in FIGS. 3 to 7, a detailed description will be omitted.

9 is a block diagram illustrating a detailed configuration of a display device according to an embodiment of the present invention. According to FIG. 9, the display device 100 includes a storage unit 110, a display unit 120, a detection unit 130, a control unit 140, an input unit 150, a GUI processing unit 160, a communication unit 170, and a detection unit. A unit 180, an audio processing unit 191, a video processing unit 192, a speaker 193, a USB port 194, a camera 195, and a microphone 196 are included. Among the components shown in FIG. 9, detailed descriptions of parts that overlap with the components shown in FIG. 1 will be omitted.

Meanwhile, the operation of the control unit 140 described above may be performed by a program stored in the storage unit 110. The storage unit 110 may store various data such as an O/S (Operating System) software module for driving the display device 100, various applications, various data input or set during application execution, and contents.

The input unit 150 is a component for receiving various user commands. In particular, the input unit 150 includes a user command for overlapping a graphic element on an image displayed on the display unit 110, a user command for selecting a graphic element, a user command for executing a graphic element input mode, and a graphic element input mode. Various user commands, such as a user command for terminating a video call and a user command related to a video call, may be input.

To this end, the input unit 150 may be implemented as a key button, a keypad, or the like, or may be implemented to receive a command transmitted from a remote control device (not shown) such as a remote control.

In particular, when the display unit 120 is implemented in the form of a touch screen, the display unit 120 can be used as the input unit 150 to receive input of various user commands other than the output device. In this case, the display 120 may display a GUI for receiving a user command.

The GUI processing unit 160 generates various GUIs. For example, when the display unit 120 is implemented in the form of a touch screen, the GUI processing unit 160 may generate a GUI for receiving various user commands and provide it to the display unit 120.

The communication unit 170 is a component for performing communication with an external device (not shown) through a network. For example, the communication unit 170 may include wired/wireless LAN, WAN, Ethernet, Bluetooth, Zigbee, USB (Universal Serial Bus), IEEE 1394, Wi-Fi or PLC (Power Line Communication), etc. Using, it is possible to communicate with other display devices (not shown) and external servers (not shown).

To this end, the communication unit 170 may include a chip or an external input port corresponding to each communication method.

In particular, the communication unit 170 may perform communication with an external display device (not shown) to transmit and receive various data for performing a video call. Specifically, the communication unit 170 receives a video call image and a user voice from an external display device (not shown), and a video call image captured by the camera 195 and a user voice collected from the microphone 196 are transmitted to the external display device. Can be transmitted to (not shown).

Here, the communication unit 170 may transmit/receive data for a video call with an external display device (not shown) using a voice over broadband (VoBB) or legacy service. Specifically, VoBB includes voice over cable modem (VoCM), voice over DSL (VoDSL), voice over Internet protocol (VoIP), fixed wireless access (FWA), fiber to the home (FTTH), or voice over ATM (VoATM). May include services of. Meanwhile, the legacy service may include an integrated service digital network (ISDN), a plain old telephone service (POTS), a cellular or 3G service, and the like.

Accordingly, the display device 100 uses a wireless network, a conventional telephone network, a data network such as the Internet, a cable modem system, a cellular network, etc. to transmit a video call image and a user's voice. You can send and receive.

For example, when the display apparatus 100 performs a video call by an external display device (not shown) through the Internet, a video call server of a company providing the video call exists on the Internet. For example, when using a video call through an Internet phone between the display device 100 and an external display device (not shown), an Internet phone call using VoIP (Voice over Internet Protocol) or MoIP (Multimedia over Internet Protocol) over the Internet Servers of companies providing services exist.

In this way, data for a video call can be transmitted and received between the display device 100 and an external display device (not shown) through a server of a telephone service provider. In addition, the display device 100 and an external display device (not shown) may be assigned a phone number through a phone service provider.

Meanwhile, the display device 100 and an external display device (not shown) can transmit and receive a video call image and a user's voice according to standards set therebetween. For example, the H.261 standard for coding a video call image, Protocols such as the H.221 standard and the H.242 standard for setting and canceling calls can be used for communication of video and user voices for video calls. More specifically, in the case of a video communication system using an Internet network, video communication standards such as H.323, H.263 and H264 standards for video coding and G723.1, G.711 and G.729 for voice coding, etc. Protocols implemented in may be used.

However, the communication service for performing a video call is not limited to this configuration. For example, the display device 100 and an external display device (not shown) may use standards such as SIP and RTP/RTCP protocols in addition to the above-described standards, and can use communication services for various known video calls. Yes, of course.

The sensing unit 180 may detect various manipulations such as touch, motion, rotation, tilt, pressure, and approach to the display device 100.

In particular, the sensing unit 180 may include a touch sensor that senses a touch. The touch sensor may be implemented in a capacitive type or a pressure sensitive type. In the capacitive type, when a part of the user's body touches the surface of the display unit 120 by using a dielectric coated on the surface of the display unit 120, the touch coordinates are calculated by sensing microelectricity excited by the user's human body. The pressure-sensitive type includes two electrode plates, and when a user touches the screen, the touch coordinates are calculated by sensing that the upper and lower plates of the touched point touch and current flows. As described above, the touch sensor may be implemented in various forms.

In addition, the sensing unit 180 may include a geomagnetic sensor, a gyro sensor, an acceleration sensor, or the like for detecting a user's motion with respect to the display device 100. Here, the geomagnetic sensor detects the rotation state and movement direction of the display device 100, the gyro sensor detects the rotation state of the display device 100, and the acceleration sensor detects the inclination of the display device 100, etc. It is a sensor to do.

In addition, when a touch on the GUI is sensed through the sensing unit 180, the controller 140 may perform an operation corresponding to the function of the corresponding item. For example, the controller 140 may determine a user command for a selected GUI based on a detection result of the detection unit 180 and control to perform an operation corresponding thereto.

For example, when a user command for performing a video call is input, the controller 140 may capture a video call image through the camera 195 and collect the user's voice through the microphone 196. In addition, the controller 140 may control the communication unit 170 to transmit the captured video call image and the collected user image to an external display device (not shown). In addition, the controller 140 controls the video processing unit 192 to process the video call image received from an external display device (not shown) in a format that can be output by the display unit 120, and displays the processed video call image. The display unit 120 can be controlled. In addition, the controller 140 controls the audio processing unit 191 to process the user's voice received from an external display device (not shown) in a form that can be output by the speaker 193, and the speaker 193 to output the processed user voice. ) Can be controlled.

As another example, when a user command for executing the graphic element input mode is input, the controller 140 may capture an image displayed on the display 120 and display the captured image. In addition, when a user command for terminating the graphic element input mode is input, the controller 140 may store the input graphic element on the captured image.

As another example, when a user command for overlapping graphic elements is input, the controller 140 may display a list of graphic elements stored in the storage unit 110. In addition, the controller 140 may display the graphic element selected through the list by overlapping the detected face area.

On the other hand, when the rotation from the portrait mode to the landscape mode is detected through the detection unit 180, the controller 140 controls to display a screen appropriately displayed for the portrait mode by rescaling or changing it to be suitable for the landscape mode. I can.

For example, when the display device 100 is rotated in the landscape mode while performing a video call through the portrait mode, the controller 140 may process the video call image to fit the landscape mode. By controlling 192, the display 120 may be controlled to output a video call image corresponding to the landscape mode.

The audio processing unit 191 is a component that processes audio data. The audio processing unit 191 may perform various processing such as decoding, amplification, noise filtering, and the like for audio data.

The video processing unit 192 is a component that processes video data. The video processing unit 192 may perform various image processing such as decoding, scaling, noise filtering, frame rate conversion, and resolution conversion on video data.

The speaker 193 is a component that outputs not only various audio data processed by the audio processing unit 180, but also various notification sounds or voice messages. In particular, the speaker 193 may output a user voice received from an external display device (not shown).

The USB port 194 may perform communication with various external devices or perform charging through a USB cable.

The camera 195 is a component for capturing a still image or a moving image according to the user's control. The camera 193 may be implemented in plural, such as a front camera and a rear camera. In particular, the camera 195 may take a video call image to perform a video call with an external display device (not shown).

The microphone 196 is a component for receiving a user's voice or other sound and converting it into audio data. In particular, the microphone 196 may collect a user's voice in order to perform a video call with an external display device (not shown).

In addition, the display apparatus 100 may further include various external input ports for connecting to various external terminals such as a headset and a mouse.

Meanwhile, the control unit 140 generally controls the operation of the display apparatus 100 using various programs stored in the storage unit 110.

For example, the control unit 140 may execute an application stored in the storage unit 110 to configure and display an execution screen thereof, or may play back various contents stored in the storage unit 110.

Specifically, the control unit 140 includes the RAM 141, the ROM 142, the main CPU 143, the graphic processing unit 144, the first to n interfaces 145-1 to 145-n, and the bus 146. Include.

The RAM 141, the ROM 142, the main CPU 143, the graphic processing unit 144, the first to n interfaces 145-1 to 145-n, and the like may be connected to each other through the bus 146.

The first to nth interfaces 145-1 to 145-n are connected to the above-described various components. One of the interfaces may be a network interface that is connected to an external device through a network.

The main CPU 143 accesses the storage unit 110 and performs booting using the O/S stored in the storage unit 110. Further, various operations are performed using various programs, contents, data, etc. stored in the storage unit 110.

The ROM 142 stores an instruction set for booting a system, and the like. When the turn-on command is input and power is supplied, the main CPU 143 copies the O/S stored in the storage unit 110 to the RAM 141 according to the command stored in the ROM 142, and executes the O/S. Boot the system. When booting is complete, the main CPU 143 copies various application programs stored in the storage unit 110 to the RAM 141 and executes the application programs copied to the RAM 141 to perform various operations.

The graphic processing unit 144 generates a screen including various objects such as icons, images, texts, etc. using an operation unit (not shown) and a rendering unit (not shown). The calculation unit (not shown) calculates attribute values such as coordinate values, shapes, sizes, colors, etc. to be displayed for each object according to the layout of the screen. The rendering unit (not shown) generates screens of various layouts including objects based on the attribute values calculated by the calculation unit (not shown). The screen generated by the rendering unit (not shown) is displayed in the display area of the display unit 120.

Meanwhile, FIG. 9 shows an example of a detailed configuration included in the display apparatus 100. Depending on the embodiment, some of the components illustrated in FIG. 9 may be omitted or changed, and other components may be further included. It can also be added. For example, a GPS receiver (not shown) for receiving a GPS signal from a GPS (Grobal Positioning System) satellite to calculate the current position of the display device 100, and a DMB for receiving and processing a Digital Multimedia Broadcasting (DMB) signal. It may further include a receiver (not shown).

10 to 16 are diagrams for explaining a method of overlapping and displaying graphic elements according to an embodiment of the present invention. In particular, FIGS. 10 to 16 illustrate a method of displaying a graphic element overlapping each image frame constituting a video call image while the display device 100 performs a video call with an external display device (not shown). It is a drawing for. However, this is only an example for convenience of description, and it goes without saying that the display apparatus 100 may display various images by overlapping graphic elements.

First, as shown in FIG. 10, the display device 100 may display a video call image 710 received from an external display device (not shown) and a video call image 720 captured by the camera 195.

Meanwhile, the display device 100 may display a GUI 730 for receiving a user command according to a preset event. Here, the preset event may be variously implemented, such as a user manipulation of touching the display screen, a user manipulation of pressing a specific button provided on the display apparatus 100, and the like.

The GUI 730 may include an item 731 for receiving a user command for overlapping a graphic element and an item 732 for executing a graphic element input mode.

Here, the graphic element input mode means a mode in which graphic elements can be input from a user.

When the item 732 for executing the graphic element input mode is selected, as shown in FIG. 11, the display device 100 captures the displayed video call image and transfers the captured image 740 to the existing video call image 710. , 720) can be displayed together. In this case, the display device 100 includes items 751, 752, and 753 for selecting various drawing tools capable of drawing graphic elements on the captured image 740, and a user command for overlapping the input graphic elements. An item 754 for receiving an input and an item 755 for receiving a user command for terminating the graphic element input mode may be displayed.

Accordingly, the user can input a graphic element on the captured image through touch manipulation. In this case, the user can select the pencil tool item 751 and input graphic elements with a pencil, or select the brush tool item 752 to input graphic elements with a brush, and select the eraser tool item 753 You can also delete entered graphic elements. However, this is only an example, and of course, various drawing tool items not shown may be used.

Meanwhile, as shown in FIG. 12, the display device 100 may overlap and display a graphic element 741 input from a user on the captured image.

In addition, when the item 754 for receiving a user command for overlapping the input graphic element is selected, the display apparatus 100 may display the input graphic element by overlapping the video call image. For example, as shown in FIG. 13, the display apparatus 100 may overlap and display a mask 741 input from a user on a face area in the video call image 710.

In this case, the display apparatus 100 may set the face area included in the captured image as the reference face area, and may overlap and display the graphic elements using the method described in FIGS. 3 to 7.

In this case, the display device 100 may store the input graphic element. In this case, the display device 100 may also store information on the location where the graphic element is input. For example, the display device 100 may store the mask 741 input in FIG. 12. In addition, the display apparatus 100 sets a reference point on the mask 741 and is positioned at the center between the two eyes, and may store location information to which the mask 741 is input.

Meanwhile, when the item 755 for receiving a user command for terminating the graphic element input mode is selected, the display device 100 removes the captured image 740 and the graphic element 741 as shown in FIG. 14. , A video call image (710. 720) may be displayed.

In this case, the display device 100 may store the input graphic element 741. In this case, the display device 100 may also store information on a location where the graphic element 741 is input.

In this way, the user can directly input graphic elements to be overlapped on the video call image. Accordingly, the input graphic element may be provided to the user by overlapping the video call image.

Meanwhile, when performing a video call, the other party may look in a direction other than the front. Even in this case, in the present invention, graphic elements may be overlapped and displayed in accordance with the face area of the other person facing the other direction.

For example, FIGS. 15 to 17 illustrate a case where a graphic element overlaps a face area of a counterpart facing the left direction.

As shown in FIG. 15, the display device 100 may display a video call image 810 received from an external display device (not shown) and a video call image 820 captured by the camera 195. In this case, referring to the video call image 810, it can be confirmed that the other party is looking to the left.

Meanwhile, the display device 100 may display a GUI 830 for receiving a user command according to a preset event. Here, the GUI 830 may include an item 831 for receiving a user command for overlapping a graphic element and an item 832 for executing a graphic element input mode.

When an item 831 for receiving a user command for overlapping a graphic element is selected, as shown in FIG. 16, a list 840 of pre-stored graphic elements may be displayed.

Specifically, the list 840 may include graphic elements 841, 842, 843, and 844 previously stored in the display device 100 and a graphic element 845 input and stored by a user.

Accordingly, when a graphic element is selected, the display apparatus 100 may display the selected graphic element by overlapping the user's face area included in the video call image. For example, as shown in FIG. 16, when the mask 841 is selected on the list 840, the display apparatus 100 may mask 841 on the user's face area included in the video call image 810 as shown in FIG. Can be displayed by overlapping.

In this case, the display apparatus 100 detects a face region in the video call image 810, deforms the mask 841 according to the direction in which the detected face region faces, and uses the deformed mask 841 into the detected face region. Can be displayed by overlapping with. In this case, the display apparatus 100 may change the shape of the mask 841 using the method specifically described with reference to FIGS. 3 to 7.

Accordingly, even when the other person's face is rotated, natural graphic elements can be overlapped and displayed. Therefore, the user's interest can be further increased.

Meanwhile, in FIGS. 16 and 17, it has been described that a graphic element previously stored in the display apparatus 100 is transformed and displayed by overlapping a face region detected in a video call image, but this is only an example. That is, when a graphic element input by the user is selected on the list according to the method described in FIGS. 10 to 13, the graphic element input by the user is transformed according to the direction of the face area detected in the video call image, and It goes without saying that it is possible to display the graphic element by overlapping the face area detected in the video call image.

In this case, the display device 100 uses the face area included in the captured image as the reference face area, and changes the shape of the graphic element in the manner described in FIGS. 3 to 7 to overlap the face area detected in the video call image. Can be displayed.

Accordingly, in addition to the previously stored graphic elements, a graphic element having a desired shape can be input and overlapped with the face area detected in the video call image in real time. Therefore, the user's interest can be further increased.

18 is a flowchart illustrating a method of displaying a graphic element according to an embodiment of the present invention.

First, a graphic element overlapping the reference face area is stored (S910). Here, the graphic element may include various types of items that may cause interest or fun to the user by overlapping the user's face area included in the image. For example, the graphic element may include various types of graphic elements that may overlap the user's face area, such as a mask, beard, horn, wound, and glasses.

In addition, the reference face region may be a face region capable of realizing a natural image when graphic elements are overlapped and displayed. In general, since the graphic element is matched to and stored in a face area facing the front direction, the reference face area may be a face area facing the front direction.

In this case, information on a point where the reference position of the graphic element is located within the face area may be stored. Here, the reference position of the graphic element may vary depending on the type of the graphic element.

Meanwhile, the graphic element may be input and stored by the user. In this case, capturing the image according to a user command, detecting a user's face area included in the captured image, and an input graphic element and an input graphic element overlapping the captured image are the captured image Position information overlapped in the user's face area included in may be stored.

Then, the user's face area included in the image is detected (S920). Here, a face region may be detected using various methods such as a feature-based face detection method, an SVM, and a face template-based detection method.

Thereafter, the graphic element is deformed according to the direction in which the detected face region faces based on the direction in which the reference face region faces (S930), and the transformed graphic element is overlapped with the face region in the image to be displayed (S940).

Meanwhile, when a graphic element input by a user is selected as an overlapping graphic element, the shape of the graphic element may be changed by setting the user's face area included in the captured image as the reference face area.

That is, the pre-stored graphic element transforms the graphic element according to the direction of the face area detected based on the reference face area, and the graphic element input and stored by the user is the direction the user's face area included in the captured image faces. Graphic elements can be transformed according to.

Specifically, first and second reference points for each of the reference face area and the detected face area are set based on a preset face component, the relative position of the second reference point is determined based on the first reference point, and the determined relative Graphical elements can be transformed based on their position.

More specifically, a reference face area is divided into a plurality of areas based on a first reference point, a face area detected based on a second reference point is divided into a plurality of areas, and a plurality of areas divided based on a first reference point And comparing the sizes of each of the plurality of regions divided based on the second reference point, and changing the shape of the graphic element based on the comparison result.

In addition, the reference position of the graphic element whose shape has been changed may be matched with the second reference point to overlap the detected face area and displayed.

According to the present embodiment, a method of deforming a graphic element and overlapping an image has been described with reference to FIGS. 3 to 7, and a specific embodiment has been described with reference to FIGS. 10 to 16.

In addition, a non-transitory computer readable medium in which a program for sequentially performing a graphic element display method according to the present invention is stored may be provided.

The non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, not a medium that stores data for a short moment, such as a register, cache, or memory. Specifically, the above-described various applications or programs may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, and ROM.

In addition, although a bus is not shown in the block diagram showing the display device, communication between components in the display device may be performed through a bus. In addition, the display device may further include a processor such as a CPU or a microprocessor that performs various steps described above.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

100: display device 110: storage
120: display unit 130: detection unit
140: control unit

Claims (18)

In the method of displaying a graphic element of a display device,
Selecting a graphic element overlapping the reference face area;
Detecting a user's face area included in the image;
Scaling the reference face area so that the size of the reference face area and the detected face area correspond;
Transforming the graphic element according to the detected face area based on the reference face area; And,
Including; displaying the transformed graphic element by overlapping the face area in the image,
The transforming step,
Dividing the reference face region into a plurality of regions, dividing the detected face region into a plurality of regions, comparing a plurality of regions of the detected face region corresponding to each of the plurality of regions of the reference face region, The graphic element display method of changing the shape of the graphic element based on the comparison result. The method of claim 1,
Capturing the image according to a user command;
Detecting a user's face area included in the captured image; And,
And storing the graphic element input so as to overlap the captured image and location information in which the input graphic element overlaps in the face area of the user included in the captured image. Display method. The method of claim 2,
The transforming step,
And changing a shape of the graphic element by setting the user's face area included in the captured image as the reference face area. The method of claim 1 or 3,
The transforming step,
Setting first and second reference points for each of the reference face region and the detected face region based on a preset face component;
Determining a relative position of the second reference point based on the first reference point; And
And transforming the graphic element based on the determined relative position. The method of claim 4,
The transforming step,
Dividing the reference face area into a plurality of areas based on the first reference point;
Dividing the detected face area into a plurality of areas based on the second reference point;
Comprising the size of each of the plurality of regions divided based on the first reference point and the plurality of regions divided based on the second reference point, and changing the shape of the graphic element based on the comparison result; and Graphic element display method, characterized in that. The method of claim 5,
The displaying step,
And displaying a reference position of the changed graphic element with the second reference point and overlapping the detected face area. In the display device,
A display unit;
A detection unit that detects a face region of a user included in the image; And,
Graphic elements overlapping the reference face area are selected by the user,
Scaling the reference face area so that the size of the reference face area and the detected face area correspond,
Transforming the graphic element according to the detected face area based on the reference face area,
Includes; a control unit for controlling to display the transformed graphic element by overlapping the face area in the image,
The control unit,
Dividing the reference face region into a plurality of regions, dividing the detected face region into a plurality of regions, comparing a plurality of regions of the detected face region corresponding to each of the plurality of regions of the reference face region, A display device configured to change the shape of the graphic element based on the comparison result. The method of claim 7,
The display unit captures and displays the image according to a user command,
The detection unit detects a user's face area included in the captured image,
The control unit controls the storage unit to store the graphic element input so as to overlap the captured image and the location information where the input graphic element overlaps in the face area of the user included in the captured image. Display device. The method of claim 8,
The control unit,
A display device, comprising: setting a user's face region detected in the captured image as the reference face region. The method according to claim 7 or 9,
The control unit,
First and second reference points are set for each of the reference face area and the detected face area based on a preset face component, and a relative position of the second reference point is determined based on the first reference point, and the The display device, characterized in that the graphic element is deformed based on the determined relative position. The method of claim 10,
The control unit,
The reference face region is divided into a plurality of regions based on the first reference point, the detected face region is divided into a plurality of regions based on the second reference point, and a plurality of divided regions are divided based on the first reference point. A display apparatus comprising: comparing sizes of a region and a plurality of regions divided based on the second reference point, and changing a shape of each region of the graphic element based on the comparison result. The method of claim 11,
The control unit,
And controlling a reference position of the graphic element whose shape has been changed to match the second reference point to overlap and display the detected face area. The method of claim 1,
The transforming step,
The graphic element display method, wherein the graphic element is transformed according to the size of the detected face area based on the size of the reference face area. The method of claim 1,
The transforming step,
A method of displaying a graphic element, wherein the graphic element is transformed according to a direction in which the detected face area is directed based on a direction in which the reference face area is directed. The method of claim 14,
The direction in which the detected face area is directed is at least one of a left direction, a right direction, an upward direction, and a right direction. The method of claim 7,
The control unit,
The display device, wherein the graphic element is transformed according to the size of the detected face area based on the size of the reference face area. The method of claim 7,
The control unit,
The display device, wherein the graphic element is transformed according to a direction in which the detected face area is directed based on a direction in which the reference face area is directed. The method of claim 17,
The direction in which the detected face area faces is at least one of a left direction, a right direction, an upward direction, and a right direction.

Images