KR102596487B1 - Display Control System, Method and Computer Readable Record Medium Thereof - Google Patents

Abstract

A display control system, method, and computer-readable recording medium are disclosed. A display control system according to an embodiment of the present invention includes a display unit that displays an image or video, a photographing unit that captures a user and images surrounding the user, a face area detection unit that detects the user's face area, and a size of the face area. and a calculation unit that calculates reference coordinates and a control unit that performs PTZ control on the image or video displayed on the display unit in response to changes in the size of the face area and the reference coordinates over time.

Description

Display Control System, Method and Computer Readable Record Medium Thereof}

The present invention relates to a display control system, method, and computer-readable recording medium, and more specifically, to a display control system, method, and computer-readable recording medium that perform PTZ control on a screen displayed according to a change in the user's gaze. It's about.

Recently, technology that allows media and humans to interact easily with each other (HCI, Human Computer Interface) is rapidly developing. In particular, due to the evolution of performance and lower prices of devices for capturing or playing video, image processing systems that can edit or process video have begun to become available on desktops.

As a result, technology for computer recognition or processing of images containing faces that reveal the user's uniqueness, status, intention, and expression has become a major challenge in the vision-based HCI field.

In relation to this, research on display control methods that reflect the user's movements is also being actively conducted, and this research aims to provide users with more realistic images.

1 is a diagram illustrating a user viewing an image or video using a display device. The display device that reflects the user's movements captures the user using a camera installed at a position facing the user along with the display device (monitor) shown in FIG. 1, analyzes the captured image, and displays the display device. You can control the image displayed.

Aside from their feasibility and effectiveness, the technologies developed to date require complex computational processes, which increases the computational burden on the processor and may cause problems with rapid data processing.

The purpose of the present invention is to provide a display control system, method, and computer-readable recording medium that can display a more realistic and three-dimensional screen by controlling the display screen in response to changes in the user's gaze.

A display control system according to an embodiment of the present invention includes a display unit that displays an image or video, a photographing unit that captures a user and images surrounding the user, a face area detection unit that detects the user's face area, and a size of the face area. and a calculation unit that calculates reference coordinates and a control unit that performs PTZ control on the image or video displayed on the display unit in response to changes in the size of the face area and the reference coordinates over time.

Additionally, the controller may perform zoom-in or zoom-out control when the size of the face area changes between the first and second viewpoints.

In addition, the control unit may perform pan control when the x-coordinate of the reference coordinates at the first and second time points changes, and the control unit may control the reference coordinates at the first and second time points. If the y coordinate of the coordinate changes, tilt control can be performed.

Meanwhile, a display control system according to another embodiment of the present invention is an image display control system including a camera that captures an image and a user terminal device that is wired or wirelessly connected to the camera and receives the image from the camera, The user terminal device includes a display unit that displays an image received from the camera, a photographing unit that captures images of the user and the user's surroundings, a face area detection unit that detects the user's face area, and calculation of the size and reference coordinates of the face area. and a control unit that performs PTZ control on the camera in response to changes over time in the size of the face area and the reference coordinates.

Additionally, the controller may perform zoom-in or zoom-out control when the size of the face area changes between the first and second viewpoints.

In addition, the control unit may perform pan control when the x-coordinate of the reference coordinates at the first and second time points changes, and the control unit may control the reference coordinates at the first and second time points. If the y coordinate of the coordinate changes, tilt control can be performed.

Meanwhile, a display control method according to an embodiment of the present invention is an image display control method using a user terminal device including a display unit that displays an image or video, comprising the steps of photographing a user and an image surrounding the user, Detecting a face area, calculating the size and reference coordinates of the face area, and PTZ control for the image or video displayed on the display unit in response to changes in the size of the face area and the reference coordinates over time. It includes steps to perform.

Additionally, in the PTZ control performance step, when the size of the face area changes at the first and second viewpoints, zoom-in or zoom-out control can be performed. there is.

In addition, in the PTZ control execution step, when the x-coordinate of the reference coordinate at the first and second viewpoints changes, pan control may be performed, and in the PTZ control execution step, the And when the y-coordinate of the reference coordinate at the second viewpoint changes, tilt control can be performed.

Additionally, a computer-readable recording medium on which a program for executing the image display control method according to the present invention is recorded may be provided.

The present invention can provide a display control system, method, and computer-readable recording medium that can display a more realistic and three-dimensional screen by controlling the display screen in response to changes in the user's gaze.

1 is a diagram illustrating a user viewing an image or video using a display device.
Figure 2 is a diagram schematically showing the configuration of a display control system according to an embodiment of the present invention.
3 to 5 are diagrams illustrating screen control performed in response to a change in the user's gaze on the display screen.
Figure 6 is a diagram illustrating a user using a user terminal connected to a camera located in a remote location.
Figure 7 is a diagram schematically showing the configuration of a display control system according to another embodiment of the present invention.
Figure 8 is a flowchart schematically showing the flow of a display control method according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the attached drawings. In addition, in this specification and drawings, the same symbols are used for components having substantially the same configuration, thereby omitting redundant description.

The terms used herein are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “include” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof. Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by these terms. The above terms are used only for the purpose of distinguishing one component from another.

Figure 2 is a diagram schematically showing the configuration of a display control system according to an embodiment of the present invention. Referring to FIG. 2, the display control system 100 according to an embodiment of the present invention includes a display unit 110, a photographing unit 120, a face area detection unit 130, a calculation unit 140, and a control unit 150. Includes.

The display unit 110 displays an image or video. The display control system 100 according to the present invention includes a display unit 110 capable of displaying the image or video, and is capable of receiving data from the outside through wired or wireless communication, and the data is stored in the image. Or it may include video. The display unit 110 can display images or videos included in the data.

Alternatively, the display control system 100 may include a memory device (not shown) that stores an image or video, and the display unit 110 may display the image or video stored in the memory device.

The photographing unit 120 captures images of the user and surroundings of the user. When the display control system 100 includes a mobile communication terminal such as a smartphone, the mobile communication terminal may be equipped with a camera capable of taking images and/or video on the front and back of the terminal, respectively. At this time, the photographing unit 120 may include a front camera of the mobile communication terminal.

Alternatively, when the display control system 100 includes a desktop or a laptop, the photographing unit 120 may be a web camera that is connected to the desktop and can capture a user using the desktop. , or it may be a camera provided on the monitor of the laptop.

The face area detection unit 130 detects the user's face area. The image captured by the photographing unit 120 is provided to the face area detection unit 130, and the face area detection unit 130 detects only the user's face area from the image captured by the photographing unit 120.

Detecting a face area can be related to face recognition or facial recognition, and there are a variety of algorithms related to face recognition, including principal component analysis using unique faces, linear discriminant analysis, and elastic swarm graph matching analysis using the Fisherface algorithm. It can be used. Meanwhile, the face recognition method used in the display control system 100 according to the present invention is not limited to any one method, and any algorithm capable of detecting the human face area included in the image can be used.

The user's face area detected by the face area detector 130 may include the eyes, nose, mouth, and chin excluding the forehead, or may even include the user's forehead area. Alternatively, the area including the user's neck may be detected as the face area.

However, it can be understood that the user's face area detected by the face area detector 130 always includes the same elements over time. When the face area is set to include the user's eyes, nose, mouth, chin, and forehead, the user's face area detected by the face area detector 130 always includes the eyes, nose, and mouth, despite the passage of time. , chin and forehead.

The calculation unit 140 calculates the size and reference coordinates of the face area. The image captured by the photographing unit 120 may consist of m×n pixels, and the size of the face area may be calculated using the number of pixels corresponding to the face area.

The reference coordinate may correspond to the average coordinate value of all pixels included in the face area, or may correspond to the intermediate coordinate value of all pixels. The coordinate value or the intermediate coordinate value may be calculated using the x-coordinate value and y-coordinate value of pixels included in the face area.

The control unit 150 performs PTZ control on the image or video displayed on the display unit in response to changes over time in the size of the face area and the reference coordinates.

The image captured by the photographing unit 120 includes a fixed number of frames per second, and each frame may be composed of m×n pixels. The face area detection unit 130 detects the user's face area from each of the frames, and the calculation unit 140 calculates the size and reference coordinates of the face area from the user's face area detected by the face area detection unit 130. do.

Changes in the size of the face area and the reference coordinates over time can be understood as changes due to changes in the frame, and the control unit 150 controls the PTZ for the image or video displayed through the display unit 110 in response to these changes. perform control.

The PTZ control refers to Pan, Tilt, and Zoom control. Pan control refers to the x-axis direction, Tilt control refers to the y-axis direction, and Zoom control refers to the z-axis direction. Hereinafter, the PTZ control operation of the control unit 150 will be described in more detail with reference to FIGS. 3 to 5.

3 to 5 exemplarily show screen control performed in response to a change in the user's gaze on the display screen.

First, FIG. 3 exemplarily shows a case in which zoom control is performed by the control unit 150 when the user moves away from the photographing unit 120 described with reference to FIG. 2 .

Figures 3(a) and 3(b) show the user's face area detected from images taken at the first time point (t1) and the second time point (t2), respectively, and Figures 3(c) and 3( d) represents a change in the image or video displayed through the display unit 110 corresponding to a change in the size of the face area in FIGS. 3(a) and 3(b), respectively. Here, the first time point (t1) can be understood as temporally preceding the second time point (t2).

Referring to FIGS. 3(a) and 3(b), the size of the user's face area detected at the second time point (t2) is smaller than the size of the user's face area detected at the first time point (t1). , the reference coordinates of the face area are the same.

In this case, the control unit 150 may determine that the user has moved away from the photographing unit 120. That is, the control unit 150 may determine that the user's gaze has moved away from the photographing unit 120, and the control unit 150 may zoom in on the image or video displayed on the display unit 110 in response. Perform (Zoom) control.

Referring to FIGS. 3(c) and 3(d), the image or video displayed at the second viewpoint t2 includes a wider area than the image or video displayed at the first viewpoint t1. In FIG. 3(d), the dashed line area (A) refers to the image or area of the video displayed at the first viewpoint (t1).

Figure 3 shows as an example that zoom-out control is performed when the size of the user's face area becomes small. Conversely, when the size of the user's face area becomes large, the control unit 150 It is determined that the user has moved in a direction closer to the photographing unit 120, and accordingly, zoom-in control can be performed on the image or video displayed on the display unit 110.

FIG. 4 exemplarily shows a case in which pan control is performed by the control unit 150 when the user moves to the right with respect to the photographing unit 120.

Figures 4(a) and 4(b) show the user's face area detected from images captured at the first time point (t1) and the second time point (t2), respectively, and Figures 4(c) and 4( d) represents a change in the image or video displayed through the display unit 110 corresponding to a change in the reference coordinate of the face area in FIGS. 4(a) and 4(b), respectively. Here, the first time point t1 can be understood as being temporally ahead of the second time point t2, and can be understood to mean a time point different from the first time point and the second time point described as an example in FIG. 3.

Referring to FIGS. 4(a) and 4(b), the x coordinate of the reference coordinate of the user's face area detected at the second time point t2 is the user's face area detected at the first time point t1. is located to the right of the x-coordinate of the reference coordinate, and the y-coordinate of the reference coordinate and the size of the face area are the same at the first viewpoint (t1) and the second viewpoint (t2). Here, the reference coordinates of the face area may be the average coordinates or intermediate coordinates of pixels included in the face area, as described with reference to FIG. 2, and exist within the face area.

In this case, the control unit 150 may determine that the user has moved to the right relative to the photographing unit 120. That is, the control unit 150 may determine that the position of the user's eyes has moved to the right based on the photographing unit 120 and is looking to the left, and the control unit 150 may display the display unit 110 in response. Perform pan control for the displayed image or video.

Referring to FIGS. 4(c) and 4(d), the image or video displayed at the second viewpoint t2 further includes a left portion of the image or video displayed at the first viewpoint t1, and a right portion includes a smaller area than at the first time point t1. That is, the control unit 150 may control the image or video display so that an area that was not displayed at a previous viewpoint is newly displayed in response to a change in the actual user's gaze, and an area displayed at a previous viewpoint is no longer displayed.

Figure 4 shows as an example that pan-left control is performed when the user's face area moves to the right. Conversely, when the user's face area moves to the left, the control unit 150 It is determined that the position of the user's eyes has moved to the left relative to the capturing unit 120 and is looking to the right, and accordingly, a pan-right function is applied to the image or video displayed on the display unit 110. Control can be performed.

FIG. 5 exemplarily shows a case in which tilt control is performed by the control unit 150 when the user moves downward with respect to the photographing unit 120.

Figures 5(a) and 5(b) show the user's face area detected from images captured at the first time point (t1) and the second time point (t2), respectively, and Figures 5(c) and 5( d) represents a change in the image or video displayed through the display unit 110 corresponding to a change in the reference coordinate of the face area in FIGS. 5(a) and 5(b), respectively. Here, the first time point (t1) can be understood as being temporally ahead of the second time point (t2), and can be understood to mean a time point different from the first and second time points described as examples in FIGS. 3 and 4. You can.

Referring to FIGS. 5(a) and 5(b), the y coordinate of the reference coordinate of the user's face area detected at the second time point t2 is the user's face area detected at the first time point t1. is located below the y coordinate of the reference coordinate, and the x coordinate of the reference coordinate and the size of the face area are the same at the first viewpoint (t1) and the second viewpoint (t2). Here, the reference coordinates of the face area may be the average coordinates or intermediate coordinates of pixels included in the face area, as described with reference to FIG. 2, and exist within the face area.

In this case, the control unit 150 may determine that the user has moved downward relative to the photographing unit 120. That is, the control unit 150 may determine that the position of the user's eyes has moved downward relative to the photographing unit 120 and is looking upward, and the control unit 150 may display the display unit 110 in response. Performs tilt control on the displayed image or video.

Referring to FIGS. 5(c) and 5(d), the image or video displayed at the second viewpoint t2 further includes an upper portion of the image or video displayed at the first viewpoint t1, and a lower portion includes a smaller area than at the first time point t1. That is, the control unit 150 may control the image or video display so that an area that was not displayed at a previous viewpoint is newly displayed in response to a change in the actual user's gaze, and an area displayed at a previous viewpoint is no longer displayed.

Figure 5 shows as an example that tilt-up control is performed when the user's face area moves downward, and on the contrary, when the user's face area moves upward, the control unit 150 It is determined that the position of the user's eyes moves upward based on the capturing unit 120 and is looking downward, and accordingly, tilt-down the image or video displayed on the display unit 110. Control can be performed.

The control unit 150 controls the PTZ operation for the image and/or video according to the size of the user's face area calculated by the calculation unit 140 and the change in the reference coordinates of the face area, so that the user can enjoy a more realistic image. You can enjoy.

Meanwhile, in the previous example, the case where the user's gaze simply moves in one direction of the x-axis, y-axis, or z-axis was explained as an example, but this is only for convenience of explanation, and the control unit 150 The movement of can be complexly judged on the x-y-z axes, and PTZ control for the image and/or video can be performed in response.

Additionally, since the image and/or video displayed through the display unit 110 must be PTZ controlled by the control unit 150, only a portion of the image and/or video may be displayed on the display unit 110.

For example, when the size of the image and/or video is M×N, the image and/or video displayed through the display unit 110 may be displayed in a size m×n smaller than M×N.

FIG. 6 is a diagram illustrating a user using a user terminal device connected to a camera located in a remote location.

Referring to FIG. 6, the user terminal device 310 and the camera 210 are included in the display control system 500 according to another embodiment of the present invention. The user terminal device 310 and the camera 210 can exchange data with each other using wired or wireless communication, and images and/or videos captured by the camera 210 can be transmitted to the user terminal device 310. The user terminal device 310 may transmit a control signal to control the operation of the camera 210.

Meanwhile, the user terminal device 310 may include a display screen for displaying images and/or videos transmitted from the camera 210 and a front camera capable of capturing user and surrounding images.

Figure 7 is a diagram schematically showing the configuration of a display control system according to another embodiment of the present invention. The display control system 500 shown in FIG. 7 includes the user terminal device 310 and the camera 210 described with reference to FIG. 6 .

More specifically, the display control system 500 according to another embodiment of the present invention includes a camera 210 that captures an image and a user that is connected to the camera 210 by wire or wirelessly and receives the image from the camera 210. Includes a terminal device 310.

Additionally, the user terminal device 310 includes a display unit 311, a photographing unit 312, a face area detection unit 313, a calculation unit 314, and a control unit 315.

The display unit 311 displays the image received from the camera 210. Here, the image captured and received by the camera 210 can be understood as including a still image. Additionally, the image captured by the camera 210 is received by the user terminal device 310 in real time, and the received image can be displayed in real time through the display unit 311.

The photographing unit 312 captures images of the user using the user terminal device 310 and images surrounding the user. When the user terminal device 310 includes a smartphone, PDA, etc. as shown in FIG. 6, the photographing unit 312 may be a front camera included in the smartphone, PDA, etc.

Alternatively, when the user terminal device 310 includes a desktop or a laptop, the photographing unit 312 may be a web camera that is connected to the desktop and can capture a user using the desktop. Alternatively, it may be a camera provided on the monitor of the laptop.

The face area detection unit 313 detects the user's face area. The image captured by the capturing unit 312 is provided to the face area detection unit 313, and the face area detecting unit 313 detects only the user's face area from the image captured by the capturing unit 312.

There are various algorithms for detecting a human face area in an image, and the display control system 500 according to the present invention is not limited and interpreted by a specific face area detection algorithm.

The user's face area detected by the face area detection unit 313 may include the eyes, nose, mouth, and chin excluding the forehead, or may even include the user's forehead area. Alternatively, the area including the user's neck may be detected as the face area.

However, it can be understood that the user's face area detected by the face area detector 313 always includes the same elements over time. When the face area is set to include the user's eyes, nose, mouth, chin, and forehead, despite the passage of time, the user's face area detected by the face area detector 313 always includes the eyes, nose, and mouth. , chin and forehead.

The calculation unit 314 calculates the size and reference coordinates of the face area. The image captured by the photographing unit 312 may consist of m×n pixels, and the size of the face area may be calculated using the number of pixels corresponding to the face area.

The reference coordinate may correspond to the average coordinate value of all pixels included in the face area, or may correspond to the intermediate coordinate value of all pixels. The coordinate value or the intermediate coordinate value may be calculated using the x-coordinate value and y-coordinate value of pixels included in the face area.

The control unit 315 performs PTZ control on the camera 210 in response to changes in the size of the face area and the reference coordinates over time.

The image captured by the capturing unit 312 includes a fixed number of frames per second, and each frame may be composed of m×n pixels. The face area detection unit 313 detects the user's face area from each of the frames, and the calculation unit 314 calculates the size and reference coordinates of the face area from the user's face area detected by the face area detection unit 313. do.

Changes in the size of the face area and the reference coordinates over time can be understood as changes due to changes in the frame, and the control unit 315 performs PTZ control on the camera 210 in response to these changes.

That is, the camera 210 included in the display control system 500 is capable of pan-tilt-zoom operations, and the control unit 315 controls the user terminal device 310 and the camera 210. ) can be used to transmit a control signal for PTZ control of the camera 210, and the camera 210 operates PTZ in response to the control signal transmitted from the control unit 315. .

Accordingly, the user terminal device 310 may include software that generates and outputs the control signal capable of controlling the operation of the camera 210.

PTZ control for the camera 210 can be understood as being substantially the same as PTZ control for the image and/or video described with reference to FIGS. 3 to 5.

For example, when the size of the user's face area changes at the first viewpoint (t1) and the second viewpoint (t2), the controller 315 zooms in or zooms the camera 210. You can control zoom-out operation.

If it is determined that the size of the user's face area increases over time, the control unit 315 determines that the user's gaze is approaching the image and/or video displayed through the display unit 311, and responds accordingly. Thus, the camera 210 can be controlled to perform a zoom-in operation.

Alternatively, when the x-coordinate of the reference coordinate changes at the first time point (t1) and the second time point (t2), the control unit 315 moves the camera 210 to Pan-Left or Pan-Light. (Pan-Right) operation can be controlled.

When the x-coordinate of the reference coordinate of the user's face area moves to the right over time, the control unit 315 determines that the user's gaze is directed to the left and, in response, causes the camera 210 to pan -Can be controlled to operate left (Pan-Left).

Accordingly, the camera 210 captures an image further to the left than the image from the previous viewpoint, and the captured image is transmitted to the user terminal device 310 in real time and displayed to the user through the display unit 311.

Alternatively, when the y-coordinate of the reference coordinate changes at the first time point (t1) and the second time point (t2), the control unit 315 tilts the camera 210 up or down. (Tilt-Down) operation can be controlled.

When the y-coordinate of the reference coordinate of the user's face area moves upward over time, the control unit 315 determines that the user's gaze is directed downward, and causes the camera 210 to tilt in response. -Can be controlled to operate in tilt-down mode.

Accordingly, the camera 210 captures an image lower than the image at the previous viewpoint, and the captured image is transmitted to the user terminal device 310 in real time and displayed to the user through the display unit 311.

The control unit 315 controls the PTZ operation of the camera 210 according to the size of the user's face area calculated by the calculation unit 314 and the change in the reference coordinates of the face area, and the image captured by the camera 210 is By being displayed through the display unit 311 in real time, the user can enjoy a more realistic image.

Meanwhile, in the previous example, the case where the user's gaze simply moves in one direction of the x-axis, y-axis, or z-axis was explained as an example, but this is only for convenience of explanation, and the control unit 315 is the user's gaze. The movement of can be complexly determined with respect to the x-y-z axes, and PTZ control of the camera 210 can be performed correspondingly.

Figure 8 is a flowchart schematically showing the flow of a display control method according to an embodiment of the present invention.

Referring to FIG. 8, the display control method according to an embodiment of the present invention is an image display control method using a user terminal device including a display unit that displays an image or video, and includes a user and surrounding image capturing step (S110), It includes a face area detection step (S120), a face area size and reference coordinate calculation step (S130), and a PTZ control step (S140).

In the user and surrounding image capturing step (S110), images of the user using the user terminal device and images of the user's surroundings are captured using a camera included in the user terminal device. At this time, the camera can be understood as being included in the front of the user terminal device along with the display unit.

In the face area detection step (S120), the user's face area is detected from images of the user and surroundings obtained using the camera. As explained with reference to FIGS. 2 and 7, there are various algorithms for detecting a human face area in an image, and the display control method according to the present invention is not limited and interpreted by a specific face area detection algorithm.

The user's face area detected in the face area detection step (S120) may include the eyes, nose, mouth, and chin excluding the forehead, or may even include the user's forehead area. Alternatively, the area including the user's neck may be detected as the face area.

However, it can be understood that the user's face area detected in the face area detection step (S120) always includes the same elements over time. When the face area is set to include the user's eyes, nose, mouth, chin, and forehead, despite the passage of time, the user's face area detected in the face area detection step (S120) always includes the eyes, nose, and forehead. Includes the mouth, chin, and forehead.

In the face area size and reference coordinate calculation step (S130), the size of the face area and the reference coordinates of the face area are calculated. An image captured using the camera may consist of m×n pixels, and the size of the face area may be calculated using the number of pixels corresponding to the face area.

The reference coordinate may correspond to the average coordinate value of all pixels included in the face area, or may correspond to the intermediate coordinate value of all pixels. The coordinate value or the intermediate coordinate value may be calculated using the x-coordinate value and y-coordinate value of pixels included in the face area.

In the PTZ control step (S140), PTZ control is performed on the image or video displayed on the display unit in response to changes over time in the size of the face area and the reference coordinates.

The image captured by the camera includes a fixed number of frames per second, and each frame may consist of m×n pixels. In the face area detection step (S120), the user's face area is detected from each of the frames, and in the face area size and reference coordinate calculation step (S130), the user's face area detected in the face area detection step (S120) is detected. Calculate the size and reference coordinates of the face area.

Changes in the size of the face area and the reference coordinates over time can be understood as changes due to changes in the frame, and the PTZ control step (S140) is performed to respond to these changes by performing a PTZ control for the image or video displayed through the display unit. perform control.

Meanwhile, in the PTZ control step (S140), PTZ control is performed on the image and/or video displayed through the display unit in response to a change in the size of the user's face or a change in the user's gaze over time, Specifically, it is as described with reference to FIGS. 3 to 5.

For example, when the size of the user's face area changes at the first viewpoint (t1) and the second viewpoint (t2), the image and/or video displayed through the display unit is changed in the PTZ control step (S140). You can control zoom-in or zoom-out.

If it is determined that the size of the user's face area increases over time, in the PTZ control step (S140), it is determined that the user's gaze is approaching the image and/or video displayed through the display unit, and the image And/or the video can be controlled to zoom in.

Alternatively, when the x-coordinate of the reference coordinate changes at the first time point (t1) and the second time point (t2), the image and/or video displayed through the display unit is pan-left in the PTZ control step (S140). (Pan-Left) or Pan-Right (Pan-Right) control.

If the x-coordinate of the reference coordinate of the user's face area moves to the right with the passage of time, in the PTZ control step (S140), it is determined that the user's gaze is directed to the left, and the image and/or You can control the video pan-left.

Alternatively, when the y coordinate of the reference coordinate changes at the first time point (t1) and the second time point (t2), the image and/or video displayed through the display unit is tilted up in the PTZ control step (S140). (Tilt-Up) or Tilt-Down control.

When the y-coordinate of the reference coordinate of the user's face area moves upward with the passage of time, in the PTZ control step (S140), it is determined that the user's gaze is directed downward, and in response to this, the image and/or You can control the video tilt-down.

In the PTZ control step (S140), the image and/or video displayed through the display unit according to the change in the size of the user's face area and the reference coordinates of the face area calculated in the face area size and reference coordinate calculation step (S130). By controlling the PTZ operation for , the user can enjoy a more realistic video.

Meanwhile, in the previous example, the case where the user's gaze simply moves in one direction of the x-axis, y-axis, or z-axis was explained as an example, but this is only for convenience of explanation, and in the PTZ control step (S140) The movement of the user's gaze can be complexly determined with respect to the x-y-z axes, and PTZ control can be performed for the image and/or video the user is viewing in response.

Meanwhile, the present invention can be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media include all types of recording devices that store data that can be read by a computer system.

Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices, and can also be implemented in the form of a carrier wave (e.g., transmission via the Internet). It includes

Additionally, the computer-readable recording medium can be distributed across computer systems connected to a network, so that computer-readable code can be stored and executed in a distributed manner. And functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers in the technical field to which the present invention pertains.

Unless there is an explicit order or statement to the contrary regarding the steps constituting the method according to the invention, the steps may be performed in any suitable order. The present invention is not necessarily limited by the order of description of the above steps. The use of any examples or illustrative terms (e.g., etc.) in the present invention is merely to describe the present invention in detail, and unless limited by the claims, the scope of the present invention is limited by the examples or illustrative terms. It is not limited. In addition, those skilled in the art will know that various modifications, combinations and changes can be made according to design conditions and factors within the scope of the appended patent claims or their equivalents.

So far, the present invention has been examined focusing on its preferred embodiments. A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative rather than a restrictive perspective. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the equivalent scope should be construed as being included in the present invention.

100, 500: display control system 110, 311: display unit
120, 312: Capturing unit 130, 313: Face area detection unit
140, 314: calculation unit 150, 315: control unit
210: Camera 310: User terminal device

Claims (13)

A display unit that displays an image or video;
a photographing unit that captures images including the user and the surroundings of the user;
a face area detection unit that detects the user's face area from the image;
a calculation unit that calculates the size of the face area and reference coordinates, which are average coordinates or intermediate coordinates of pixels included in the face area; and
a control unit that performs PTZ control on the photographing unit in response to changes over time in the size of the face area and the reference coordinates;
Including,
The calculation unit calculates the size and first reference coordinates of the first face area from the first image captured at the first viewpoint by the photographing unit, and calculates the size and first reference coordinates from the second image photographed at the second viewpoint by the photographing unit. 2 Calculate the size and second reference coordinates of the face area,
The control unit performs PTZ control on the photographing unit according to changes in the size of the first face area and the size of the second face area and changes in the first and second reference coordinates. system. According to paragraph 1,
The display control system wherein the controller performs zoom-in or zoom-out control when the size of the second face area changes from the size of the first face area. According to paragraph 1,
The control unit is a display control system that performs pan control when the x-coordinate of the second reference coordinate changes from the x-coordinate of the first reference coordinate. According to paragraph 1,
The control unit performs tilt control when the y-coordinate of the second reference coordinate changes from the y-coordinate of the first reference coordinate. A camera that takes video; and
a user terminal device connected to the camera by wire or wirelessly to receive images from the camera;
A video display control system including, wherein the user terminal device includes,
a display unit that displays images received from the camera;
a photographing unit that captures images including the user and the surroundings of the user;
a face area detection unit that detects the user's face area from the image;
a calculation unit that calculates the size of the face area and reference coordinates, which are average coordinates or intermediate coordinates of pixels included in the face area; and
a control unit that performs PTZ control on the camera in response to changes over time in the size of the face area and the reference coordinates;
Including,
The calculation unit calculates the size and first reference coordinates of the first face area from the first image captured at the first viewpoint by the photographing unit, and calculates the size and first reference coordinates from the second image photographed at the second viewpoint by the photographing unit. 2 Calculate the size and second reference coordinates of the face area,
The control unit performs PTZ control on the photographing unit according to a change in the size of the first face area and the size of the second face area and a change in the first reference coordinate and the second reference coordinate. system. ◈Claim 6 was abandoned upon payment of the setup registration fee.◈ According to clause 5,
The display control system wherein the controller performs zoom-in or zoom-out control when the size of the second face area changes from the size of the first face area. ◈Claim 7 was abandoned upon payment of the setup registration fee.◈ According to clause 5,
The control unit is a display control system that performs pan control when the x-coordinate of the second reference coordinate changes from the x-coordinate of the first reference coordinate. ◈Claim 8 was abandoned upon payment of the setup registration fee.◈ According to clause 5,
The control unit performs tilt control when the y-coordinate of the second reference coordinate changes from the y-coordinate of the first reference coordinate. A video display control method using a user terminal device including a display unit that displays an image or video, comprising:
Taking an image including a user and the user's surroundings;
detecting the user's face area from the image;
calculating the size of the face area and reference coordinates, which are average coordinates or intermediate coordinates of pixels included in the face area; and
performing PTZ control on the capturing unit that captured the image in response to changes over time in the size of the face area and the reference coordinates;
Including,
The step of calculating the size and reference coordinates of the face area is,
The size and first reference coordinates of the first face area are calculated from the first image captured at the first viewpoint by the photographing unit, and the size of the second face area is calculated from the second image photographed at the second viewpoint by the photographing unit. Comprising: calculating the size and second reference coordinates,
The PTZ control performance step is,
A display comprising; performing PTZ control on the photographing unit according to changes in the size of the first face area and the size of the second face area and changes in the first and second reference coordinates. Control method. ◈Claim 10 was abandoned upon payment of the setup registration fee.◈ According to clause 9,
In the PTZ control performance step, when the size of the second face area changes from the size of the first face area, display control to perform zoom-in or zoom-out control method. ◈Claim 11 was abandoned upon payment of the setup registration fee.◈ According to clause 9,
In the PTZ control execution step, when the x-coordinate of the second reference coordinate changes from the x-coordinate of the first reference coordinate, a display control method of performing pan control. ◈Claim 12 was abandoned upon payment of the setup registration fee.◈ According to clause 9,
In the PTZ control performance step, when the y-coordinate of the second reference coordinate changes from the y-coordinate of the first reference coordinate, a display control method of performing tilt control. delete

Images