WO2022124436A1

WO2022124436A1 - Ai studio for online lecture

Info

Publication number: WO2022124436A1
Application number: PCT/KR2020/017829
Authority: WO
Inventors: 김하나
Original assignee: (주)와이즈업
Priority date: 2020-12-08
Filing date: 2020-12-09
Publication date: 2022-06-16
Also published as: KR102571677B1; KR20220081215A

Abstract

The present invention relates to an AI studio for an online lecture and, more particularly, an AI studio for an online lecture which provides a virtual studio by separating an image of a subject to be photographed during an online lecture from a background by means of a processing device unit, and is capable of controlling the size and position of an output outputted from the virtual studio by means of a control terminal unit.

Description

AI Studio for Online Teaching

The present invention relates to an AI studio for an online lecture, and more specifically, a virtual studio is provided by separating an image of a subject to be photographed during an online lecture in a processing unit, and a virtual studio is provided, and output to the virtual studio It relates to an AI studio for online lectures that can control the size and position of the printed output through the control terminal.

The school education field in 2020 is faced with the challenging reality of the unprecedented online school opening as it faces the critical situation of the coronavirus pandemic.

In this situation, not only schools, but also various academic societies, conferences, and meetings have no choice but to conduct remote lectures and teleconferences using online.

Such a sudden change in the environment served as a catalyst for many parts that were previously conducted face-to-face offline to be switched to non-face-to-face use online.

Existing online lectures are methods of using presentations, and they simply film and relay the presentation screen as it is, and place the presenter and presentation material on different screens, which is inconvenient for those who listen to the presentation to alternate between the presenter and the presentation material. It was cumbersome for listeners to watch the presentation, such as showing only the presentation material on one screen and only hearing the presenter's voice without the presenter's appearance.

Therefore, there was a problem in that the concentration of those who listened to the presentation decreased, which lowered the interest of the class or event, and eventually halved the effect of the class or event.

In addition, when people listening to the presentation and the presenter conduct a discussion or question and answer with each other, only one screen is displayed, so both the listener and the presenter have low concentration, making it difficult to conduct a proper discussion or question-and-answer.

Korean Patent Publication No. 10-2016-0126731 (2016.11.02.)

The present invention devised to solve such a problem relates to an AI studio for online lectures, and more particularly, by separating the subject and the background of the subject in the processing unit for the image captured by the subject conducting the online lecture, virtual It provides a studio and relates to an AI studio for online lectures that can control the size of the output output to the virtual studio through the control terminal.

In addition, the present invention can display the presenter, presentation material, and viewer on a single virtual screen, adjust the size, position and movement of the presenter, presentation material or viewer, and increase the viewer's concentration by dynamically configuring the screen, It aims to provide an AI studio for online lectures.

The AI studio for the online lecture of the present invention for achieving the above object is,

A photographing device unit 110 for photographing a subject to be photographed;

processing unit 120 for receiving and processing the image captured by the photographing unit 110;

A control terminal unit 130 consisting of a terminal for controlling the processing unit 120,

a first monitor unit 140 that receives images of at least one viewer from the processing unit 120 and displays them so that the subject to be photographed can check;

It may be composed of a second monitor unit 150 that receives the currently output screen from the processing unit 120 and displays it so that a subject to be photographed can check it.

In addition, the processing unit 120 may recognize the subject and the background by distinguishing them from the image captured by the photographing unit 110 .

In this case, the photographing device unit 110 may further include a distance measuring means capable of confirming the distance between the photographing subject and the background through sound waves and light rays, respectively.

In addition, the processing unit 120 may apply at least one new layer to the output image.

In this case, the user can adjust the size, position, and movement of the layer applied to the output image of the processing unit 120 through the control terminal unit 130 .

In addition, the AI studio for the online lecture of the present invention may be configured to further include a chroma key screen 200 configured as a background behind the subject to be photographed and at least one or more lights 300 for illuminating the subject to be photographed.

In addition, the processing unit 120 may be configured to further include an external relay means.

The AI studio for the online lecture of the present invention is,

By recognizing the subject of the presenter and the background screen, respectively, the screen of the presentation material and/or the viewer listening to the presentation is output in the background so that the presenter, the presentation material, and the viewer can be viewed together, thereby maximizing the immersion of the presentation, lecture or meeting. have.

In addition, the image of the presenter, presentation material, and viewer can be output by adjusting the size, position, and movement of the subject to be photographed. Through this, the subject to be photographed can select as needed and enlarge or reduce the image of the presenter, the presentation material and the person who listens to the presentation, change the position, or give movement to output the image, so a screen reflecting the intention of the presenter, can pay Through this, it is possible to create a video reflecting the intention of the subject of the presentation or lecture, so that the presentation or lecture can be conducted more effectively.

In addition, through the chroma key screen and at least one or more lights to illuminate the subject, the image designated by the subject can be used as the background rather than the actual environment in which the subject is being filmed, so it is more convenient for subjects who need to proceed with shooting in a limited environment. By providing an environment in which to focus, it is possible to shoot a better image, which has the advantage of obtaining high-quality results in video presentations or recording lectures.

1 is an embodiment of an AI studio for an online lecture of the present invention;

Figure 2 is an embodiment of the subject and background recognition of the processing unit of the present invention

3 is an example of adjusting an output screen of the present invention;

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings. Prior to this, the terms or words used in the present specification and claims should not be construed as being limited to conventional or dictionary meanings, and the inventor should properly understand the concept of the term in order to best describe his invention. Based on the principle that can be defined, it should be interpreted as meaning and concept consistent with the technical idea of the present invention. In addition, unless there are other definitions in the technical and scientific terms used, it has the meaning commonly understood by those of ordinary skill in the art to which this invention belongs, and in the following description and accompanying drawings, the gist of the present invention Descriptions of known functions and configurations that may be unnecessarily obscure will be omitted.

Fig. 1 is an embodiment of an AI studio for an online lecture of the present invention, Fig. 2 is an embodiment of recognizing a subject and a background of the processing unit of the present invention, and Fig. 3 is an embodiment of adjusting an output screen of the present invention.

The AI studio for the online lecture of the present invention, as shown in FIG. 1,

A control terminal comprising a photographing apparatus unit 110 for photographing a subject to be photographed, a processing apparatus unit 120 for receiving and processing the image captured by the photographing apparatus unit 110 , and a terminal for controlling the processing apparatus unit 120 . The unit 130, the first monitor unit 140 for receiving and displaying at least one viewer image received from the processing unit 120 so that the subject to be photographed can check, and the processing unit 120 for the screen currently being output It may be composed of a second monitor unit 150 that is received from and displayed so that the subject to be photographed can check it.

That is, the photographing device of the photographing apparatus unit 110 captures the subject to be photographed, and transmits the corresponding image to the processing apparatus unit 120 .

In this case, as shown in FIG. 2 , the processing unit 120 may distinguish and recognize a subject and a background from the image captured by the image capturing unit 110 . Through this, the processing unit 120 may process the subject and the background, respectively.

To explain more easily, as shown in FIG. 2 , the image photographed through the photographing device 110 is transmitted to the processing device 120 , and the processing device 120 recognizes the image and an object in the image. and background are divided into two categories. Through this recognition, the subject and the background are independently recognized and divided.

In this case, the image processing method for segmentation may use the following method.

First, it is a segmentation method using color information.

In the segmentation method using color information, each pixel value is calculated and processed, and the object can be efficiently extracted based on the characteristic that the pixels of the object have similar color values. Among the division methods using color information, a representative method is the Mean Shift algorithm, which is a method for clustering pixel data having color values. The Mean Shift algorithm is an algorithm that groups pixels with similar colors in an image by region. However, when color information is used, when the color of the object and the background area is similar, a part of the object and the background is processed as an integrated area, and thus, an incomplete extraction result is shown.

In order to solve this problem, a method using color information and depth information of an image may be used. By using the color information of the image and the depth map used in 3D stereoscopic conversion, even if the object and the background area have similar colors, it can be divided into detail according to the degree of depth.

In addition, it is possible to divide a video in which shadows and changes in brightness appear in the background by converting it to an HSV color space and extracting main objects without being affected by changes in brightness.

Also, a segmentation method using the boundary of an object may be used.

In image segmentation, the boundary of an object or region becomes a major clue in segmentation. Since the boundary line of the image can be extracted according to the shape of an object or region, morphological division is possible. The task of extracting a boundary from an image is to convert a color image to grayscale and then use a second derivative function to detect a place where the brightness value changes rapidly in the image.

There are several methods for detecting the existing boundary, such as a Laplacian filter, a Sobel mask, and a Canny edge detector. In the gray scale image, since one brightness value per pixel is processed, the characteristics of the boundary must be distinguished and processed. In this case, after the image is over-segmented using the region extension method, an important region in the image can be divided by determining the thickness, continuity, and straightness of the boundary.

The photographing device unit 110 may further include a distance measuring means capable of confirming the distance between the photographing subject and the background through sound waves and light rays, respectively.

That is, a separate means for measuring a distance, such as a TOF sensor, is further included in the photographing device unit 110, and from the distance information image of the object to be recognized and the video image photographed by the camera obtained through the distance measurement means. By calculating the extracted edge image, 3D point cloud data (3D point cloud) consisting of effective constrained features is acquired, and it is matched with the 3D point cloud data of the entire object to be recognized. It can be recognized by dividing the background.

Also, the processing unit 120 may apply at least one new layer to the output image. That is, the processing unit 120 recognizes the subject and the background screen as the presenter, respectively, outputs the presentation material and/or the screen of the viewer who listens to the presentation in the background, so that the presenter, the presentation material, and the viewer can be viewed together to present, It can maximize the immersion of a meeting or lecture.

That is, as shown in FIG. 3 , in order for the viewer to focus on the presentation material, the subject to be photographed may adjust the output size and position of the presentation material or generate a dynamic movement of the presentation material. In addition, the image of the presenter and the viewer who listens to the presentation may be output by adjusting the position, size, and movement of the subject to be photographed. Through this, the subject to be photographed can select as needed and enlarge or reduce the image of the presenter, the presentation material, or the person who listens to the presentation, adjust the position, or give motion to output the image, thereby creating a screen reflecting the intention of the presenter, the subject of photography. can pay

To explain more easily with an example, as shown in (A) of FIG. 3 , when the subject to be photographed shows the presentation material and proceeds with the presentation, the size of the presentation material other than the subject to be photographed is output large, and the position is also adjusted to the central part. As shown in (B) of FIG. 3 , when conducting a discussion or query with a person listening to the presentation, the image of the person listening to the presentation can be enlarged and the output size of the presentation material can be reduced.

Through this, it is possible to make a more effective presentation or lecture by creating a video reflecting the intention of the subject who is conducting a presentation or lecture.

In addition, the present invention provides the first monitor unit 140 for receiving and displaying at least one viewer's images from the processing unit 120 so that the subject to be photographed can check, and the processing unit 120 for the currently output screen. It may include a second monitor unit 150 that is received from and displayed so that the subject to be photographed can check it.

That is, the subject to be photographed can check the current shooting state in real time through the first monitor unit 140 and the second monitor unit 150, so that the subject to be photographed can refer to it in real time to perform such tasks as lectures, conferences, and presentations. Events can be run more efficiently. In this case, the first monitor unit 140 and the second monitor unit 150 may be configured as virtual monitors.

That is, through the chroma key screen and at least one or more lights to illuminate the subject, the image specified by the subject can be used as the background rather than the actual environment where the subject is, so it is more effective for subjects who need to proceed with shooting in a limited environment. By providing an environment in which to focus, it is possible to shoot a better image, which has the advantage of obtaining high-quality results in video presentations or recording lectures.

In addition, the processing unit 120 may be configured to further include an external relay means. Through the external relay means, the subject can transmit images in real time in response to time transmission using the existing image platform, and the images can be viewed through various devices as long as the image platform or various devices are supported. .

The spirit of the present invention should not be limited to the described embodiments, and not only the claims to be described below, but also all those with equivalent or equivalent modifications to the claims will fall within the scope of the spirit of the present invention.

110: photographing device unit

120: processing unit

130: control terminal unit

140: first monitor unit

150: second monitor unit

200: chroma key screen

300: light

The AI studio for the online lecture of the present invention is,

Claims

a photographing device unit 110 for photographing a subject;

a processing unit 120 for receiving and processing the image captured by the photographing unit 110;

a control terminal unit 130 configured as a terminal for controlling the processing unit 120;

a first monitor unit 140 for receiving and displaying at least one viewer's image from the processing unit 120 so that a subject to be photographed can check; and

a second monitor unit 150 that receives the currently output screen from the processing unit 120 and displays it so that a subject to be photographed can check;

AI studio for online lectures, including
According to claim 1,

The processing unit 120 is

AI studio for online lecture, characterized in that the photographing target and the background are distinguished and recognized in the image photographed through the photographing device unit 110 .
3. The method of claim 2,

The processing unit 120 is

AI studio for online lectures, characterized in that at least one new layer can be applied to the output image.
4. The method of claim 3,

The processing unit 120 is

AI studio for online lecture, characterized in that the user adjusts the size and position of the layer applied to the output image through the control terminal unit (130).
5. The method of claim 4,

The processing unit 120 recognizes the subject of the presenter and the background screen, respectively,

AI studio for online lecture, characterized in that by outputting the screen of the presentation material and the viewer listening to the presentation on the background screen, the presenter, the presentation material and the viewer are displayed together.
6. The method of claim 5,

a chroma key screen 200 configured as a background behind the subject; and

AI studio for online lecture, characterized in that it further comprises at least one or more lights 300 for illuminating the subject to be photographed.