TWI826293B - Method for automatically adjusting the layout of video conference and electronic device using the same - Google Patents

Abstract

A method for automatically adjusting the layout of a video conference and an electronic device using the same are provided. The method for automatically adjusting the layout of the video conference includes the following steps. In a video conference, whether a sharing window is a presentation is detected. If the sharing window is the presentation, whether there is a free area suitable for setting a top-level message object in the presentation is detected. If there is the free area, the topmost message object is set according to the free area. If the is no free area, at least one coverable object in the presentation is analyzed, at least one participant window is detected and the topmost message object is set according to the coverable object and the participant window.

Description

Methods for automatically adjusting video conference layout and electronic devices using it set

The present disclosure relates to a control method and an electronic device using the same, and in particular, to a method of automatically adjusting the layout of a video conference and an electronic device using the same.

In response to factors such as globalization and the epidemic, the demand for online meetings has increased significantly. Researchers have observed that there are still some inconveniences in today's use of video conferencing. For example, when the presenter plays the presentation in full screen, there will often be situations where the top-level message window or message object blocks the presentation. In order to view the content of the presentation, the presenter needs to constantly move these message windows or message objects.

This disclosure relates to a method for automatically adjusting the layout of a video conference and an electronic device using the same, which can automatically analyze whether there is enough top-level information to set. Unused areas of objects, or find less important obscurable objects or attendee windows in the presentation. After analyzing idle areas, obscurable objects or participant windows, the top message object can be automatically moved to avoid obscuring important content in the presentation.

According to one aspect of the present disclosure, an electronic device that can automatically adjust the layout of a video conference is provided. The electronic device that can automatically adjust the video conference layout includes a sharing window detection module, a presentation area analysis module, an object analysis module, a participant window analysis module and an adjustment module. The shared window detection module is used in a video conference to detect whether a shared window is a presentation. The presentation area analysis module is used to detect whether there is enough free area for setting a top-level message object in the presentation when the sharing window is a presentation. The object analysis module is used to analyze at least one occludeable object in the briefing when the idle area cannot be detected in the briefing. The participant window analysis module is used to detect at least one participant window when the presentation cannot detect an idle area. The adjustment module is used to set the top-level message object based on the idle area when the presentation cannot detect the idle area; the adjustment module is used to set the top-level message object based on the idle area when the presentation cannot detect the idle area. The top-level message object.

According to another aspect of the present disclosure, a method for automatically adjusting the layout of a video conference is proposed. The method to automatically adjust the video conference layout includes the following steps. In a video conference, detect whether a shared window is a presentation. If the sharing window is a presentation, check whether there is enough free area for setting a top-level message object in the presentation. If an idle area is detected in the presentation, the top-level message object is set based on the idle area. If no idle areas can be detected in the briefing, analyze at least one of the The shielding object detects at least one participant window, and sets the top message object based on the shieldable object and the participant window.

In order to have a better understanding of the above and other aspects of the present disclosure, embodiments are given below and described in detail with reference to the accompanying drawings:

100,200,300: Electronic devices

110: Share window detection module

111:Window analysis unit

113:Task analysis unit

120: Briefing area analysis module

121:Content Analysis Unit

123: Regional judgment unit

130:Object analysis module

131:Image analysis unit

133: Key analysis unit

140:Participant window analysis module

141: Face detection unit

142:Window definition unit

150: Adjustment module

900:Internet

Ai: idle area

AP:application name

Bn: Key possibilities

BX: content block

DF: content definition file

FC: face

HL: hyperlink text

IM:image

MS: Top-level message object

OBn: Object that can be obscured

PP:Picture

PT:Presentation

S110,S111,S112,S113,S114,S115,S116,S120,S121,S122,S123,S124,S125,S126,S127,S128,S130,S131,S132,S133,S134,S140,S141,S142,S150 , S160: Steps

SDp:page

SGn: Split object

TM: Task Management Form

TT: window title

TX: visual text

Wk: attendee window

WS:Share window

Figure 1 is a schematic diagram of a video conference according to an embodiment.

Figure 2 illustrates a block diagram of an electronic device that can automatically adjust the layout of a video conference according to an embodiment.

Figure 3 illustrates a flow chart of a method for automatically adjusting the layout of a video conference according to an embodiment.

Figure 4 shows a detailed flow chart of step S110.

Figure 5 illustrates step S110.

Figure 6 shows a detailed flow chart of step S120.

Figure 7 illustrates step S120.

Figure 8 illustrates a detailed flow chart of step S130.

Figure 9 illustrates step S130.

Figure 10 shows a detailed flow chart of step S140.

Figure 11 illustrates step S140.

Figure 12 illustrates the top-level message object on different pages of the presentation.

Please refer to Figure 1, which illustrates a schematic diagram of a video conference according to an embodiment. During a video conference, the presenter can share the presentation PT through the electronic device 100, and through the network 900, the listeners can view the content of the presentation PT on the electronic devices 200 and 300. However, when the presenter plays the full-screen presentation PT, there may be some top-level message objects MS on the electronic device 100 that block the content of the presentation PT. The top-level message object MS is, for example, a lens, microphone and other control buttons, or a participant text message window. These top-level message objects MS are still objects that need to be displayed for the presenter. Every time the user switches the page of the presentation PT, the user may need to constantly move these top-level message objects MS in order to view the important information in the presentation PT. content.

Please refer to FIG. 2 , which illustrates a block diagram of an electronic device 100 that can automatically adjust the layout of a video conference according to an embodiment. The electronic device 100 includes a shared window detection module 110, a presentation area analysis module 120, an object analysis module 130, a participant window analysis module 140 and an adjustment module 150. The shared window detection module 110 includes a window analysis unit 111 and a task analysis unit 113. The presentation area analysis module 120 includes a content analysis unit 121 and an area determination unit 123. The object analysis module 130 includes an image analysis unit 131 and a focus analysis unit 133 . The participant window analysis module 140 includes a face detection unit 141 and a window definition unit 142. The sharing window detection module 110, the presentation area analysis module 120, the object analysis module 130, the participant window analysis module 140 and the adjustment module 150 are used to execute various processing procedures, control procedures and judgment procedures, such as a circuit , a chip, a circuit board or a storage device storing program code.

In this embodiment, the electronic device 100 can automatically analyze whether there is enough idle area Ai to set up the top-level message object MS, or find out less important obscurable objects OBn or participant windows Wk from the presentation PT. After analyzing the idle area Ai, the obscurable object OBn or the participant window Wk, the top message object MS can be automatically moved to avoid obscuring important content in the presentation PT. The following is a detailed description of the operation of each component with a flow chart.

Please refer to FIG. 3 , which illustrates a flow chart of a method for automatically adjusting the layout of a video conference according to an embodiment. In step S110, as shown in Figure 2, the shared window detection module 110 detects whether a shared window WS is a presentation during the video conference. If the sharing window WS is a presentation, step S120 is entered; if the sharing window WS is not a presentation, the process ends. Please refer to Figures 4 and 5. Figure 4 illustrates a detailed flow chart of step S110, and Figure 5 illustrates step S110. Step S110 includes steps S111 to S116.

In step S111, as shown in FIG. 5, the window analysis unit 111 of the shared window detection module 110 detects the shared window WS. During the video conference, the presenter may share the presentation window, document editing window or PDF window.

Next, in step S112, as shown in FIG. 5, the window analysis unit 111 of the shared window detection module 110 obtains the window title TT of the shared window WS. As shown in Figure 5, the window title TT is, for example, "Power Point" or "Word".

Then, in step S113, as shown in Figure 5, the task analysis unit 113 of the shared window detection module 110 searches for an application name AP in a task management form TM based on the window title TT. The application name AP is, for example, "Power Point Show" or "Word (32-bit)".

Next, in step S114, as shown in Figure 5, the task analysis unit 113 of the shared window detection module 110 analyzes whether the shared window WS is a presentation based on the application name AP. For example, the task analysis unit 113 determines whether the application name AP includes default content (such as "Power Pint", "Keynote", "Workspace", or "Canva"). The default content can be adjusted as the presentation software evolves. If the application name AP contains the default content, it can be determined that the sharing window WS is a presentation (eg, step S115); if the application name AP does not contain the default content, it can be determined that the sharing window WS is not a presentation (eg, step S116).

As shown in Figure 3, when it is determined that the sharing window WS is the presentation PT, the process proceeds to step S120. In step S120, the presentation area analysis module 120 detects whether there is enough free area Ai for setting the top-level message object MS in the presentation PT. If the idle area Ai cannot be detected in the presentation PT, proceed to step S130; if the idle area Ai can be detected in the presentation PT, proceed to step S150. Please refer to Figures 6 and 7. Figure 6 illustrates a detailed flow chart of step S120, and Figure 7 illustrates step S120. Step S120 includes steps S121 to S128.

In step S121, as shown in Figure 7, the content analysis unit 121 of the presentation area analysis module 120 analyzes several content blocks BX based on the content definition file DF of the presentation PT. The content block BX is, for example, a block diagram, a statistical chart or a text block. The file extension of the content definition file DF is, for example, ".xml".

Then, in step S122, as shown in FIG. 7, the content analysis unit 121 of the presentation area analysis module 120 obtains the position and size of each content block BX according to the content definition file DF. Each content block BX has its own location and size. These content blocks BX may overlap or even be completely covered.

Next, in step S123, as shown in Figure 7, the area determination unit 123 of the presentation area analysis module 120 determines whether there is enough space between the content blocks BX to set the top layer based on the position and size of the content blocks BX. The idle area Ai of the message object MS. The area determination unit 123 determines whether the range outside the association of the content block BX can accommodate the top-level message object MS. If there is enough free area Ai for setting the top-level message object MS between these content blocks BX, it is determined that the presentation PT can detect the free area Ai for setting the top-level message object MS (step S128); if these content areas If there is not enough idle area Ai for setting the uppermost message object MS between the blocks BX, step S124 is entered.

In step S124, as shown in FIG. 7, the content analysis unit 121 of the presentation area analysis module 120 decomposes the content block BX into at least one visual text TX, at least one picture PP or at least one hyperlink text HL. In this step, the visual text TX, image PP and hyperlink text HL are not limited to the outer frame of the content block BX, but are directly analyzed based on the edges of the text and the edge of the image to obtain smaller blocks.

Next, in step S125, as shown in Figure 7, the content analysis unit 121 of the presentation area analysis module 120 obtains the position and size of the visual text TX, the picture PP and the hyperlink text HL. Each visual text TX, picture PP and hyperlink text HL have their own position and size. These visual text TX, picture PP and hyperlink text HL will not overlap.

Then, in step S126, as shown in Figure 7, the area determination unit 123 of the presentation area analysis module 120 determines the visual text TX, the picture PP and the hyperlink text HL according to the position and size of the visual text TX, the picture PP and the hyperlink text HL. Whether there is enough free area Ai between the hyperlink text HL to set the top-level message object MS. The area determination unit 123 determines whether the range outside the union of the visual text TX, the picture PP and the hyperlink text HL can accommodate the top-level message object MS. If there is enough free area Ai for setting the top-level message object MS between the visible text TX, the picture PP and the hyperlink text HL, it is determined that the presentation PT can detect the free area Ai for setting the top-level message object MS (step S128 ); if there is not enough free area Ai for setting the top-level message object MS between the visible text TX, the picture PP and the hyperlink text HL, it is determined that the presentation PT cannot detect the free area Ai for setting the top-level message object MS. (Step S127).

As shown in Figure 3, when the presentation PT cannot detect the idle area Ai in which the top-level message object MS can be set, the process continues to step S130.

In step S130. The object analysis module 130 analyzes at least one obscurable object OBn in the presentation PT. Please refer to Figures 8 and 9. Figure 8 illustrates a detailed flow chart of step S130, and Figure 9 illustrates step S130. Step S130 includes steps S131 to S134.

In step S131, the image analysis unit 131 of the object analysis module 130 converts the presentation PT into an image IM. In this step, the entire presentation PT is converted into an image IM using screen capture, regardless of the composition of the presentation PT.

Next, in step S132, as shown in FIG. 9, the image analysis unit 131 of the object analysis module 130 performs a semantic segmentation process on the image IM to obtain several segmented objects SGn. The image IM can be cut into segmented objects SGn using a semantic segmentation program. The segmented object SGn is cut directly according to the pixel changes of the image IM, and is usually not exactly the same as the above-mentioned content block BX, visual text TX, picture PP or hyperlink text HL.

Then, in step S133, as shown in FIG. 9, the focus analysis unit 133 of the object analysis module 130 analyzes the focus possibility of each segmented object SGn. For example, the focus analysis unit 133 analyzes the focus possibility Bn of each segmented object SGn according to the following formula (1).

Bn=(Ws/Sn+Wp/Pn+Wr/Rn)/(Ws+Wp+Wr)............(1)

Among them, Ws is the weight of the size factor of the segmented object SGn, Sn is the relative relationship between the size of the segmented object SGn and the image IM, Wp is the weight of the center of gravity offset of the segmented object SGn, and Pn is the center of gravity offset of the segmented object SGn. Shift value, Wr is the pixel edge ratio weight of the divided object SGn, Rn is the pixel edge ratio of the divided object SGn.

Sn is the relative relationship between the size of the segmented object SGn and the image IM, which uses the composition method of thirds (also known as tic-tac-toe composition, Rule of Third) as the reference basis for analysis. For example, Sn is calculated according to the following formula (2): Sn=Zn/Z if Zn≧Z Sn=Z/Zn if Zn<Z........................ ..................(2)

Among them, Zn is the size of the segmented object SGn, and Z is 1/9 times the size of the image IM.

Since the more important content is usually placed in the center of the presentation PT, Pn can be calculated according to the following formula (3) to obtain the offset value of the center of gravity of each segmented object SGn and the center of gravity of the entire image IM: Pn=Max(( (Xcn-Xh) ² +(Ycn-Yh) ² )1/2,Pm)............(3)

Among them, Xcn and Ycn are the positions of the center of gravity of the segmented object SGn, Xh and Yh are the positions of the center of gravity of the image IM, and Pm is the length distance between two adjacent pixels to avoid zero values in Pn.

In addition, more important content usually has a complete outline, so Rn can be calculated according to the following equation (4) to obtain the proportion of the outline of each segmented object SGn coinciding with the outline of the entire input image.

Rn=Max(Ton/TL,Rm)........................................ ...(4)

Among them, TL is the number of outline pixels of the segmented object SGn, Ton is the number of pixels where the outline of the segmented object SGn coincides with the outline of the image IM, and Rm is 1/TL to avoid zero values in Rn.

Through the above method, the key possibility Bn of each segmented object SGn can be analyzed.

Then, in step S134, as shown in FIG. 9 , the focus analysis unit 133 of the object analysis module 130 outputs at least one obscurable object OBn based on these focus possibilities Bn. For example, the segmented object SGn whose focus possibility Bn is lower than a preset value can be regarded as the occludeable object OBn. Alternatively, the segmented objects SGn whose focus possibilities Bn are ranked after a predetermined percentage may be regarded as the occludeable objects OBn.

As shown in Figure 3, after the occludeable object OBn in the presentation PT is analyzed, the process continues to step S140.

In step S140, the participant window analysis module 140 detects at least one participant window Wk. Please refer to Figures 10 and 11. Figure 10 illustrates a detailed flow chart of step S140, and Figure 11 illustrates step S140. Step S140 includes steps S141~S142.

In step S141, as shown in Figure 11, the face detection unit 141 of the participant window analysis module 140 uses a face detection program to detect at least one face FC.

Next, in step S142, as shown in FIG. 11, the window definition unit 142 of the participant window analysis module 140 obtains the participant window Wk based on the face FC. The window definition unit 142 searches for the window where the face FC is located, and regards this window as the participant window Wk. The participant window Wk usually only displays the appearance or representative image of the participant, so it can also be obscured by the top information object MS.

As shown in Figure 3, when it is determined through step S120 that the presentation PT contains an idle area Ai where the top-level message object MS can be set, the process proceeds to step S150; when it is analyzed through steps S130 and S140 that the coverable objects OBn and After the participant window Wk is opened, the process continues to step S160.

In step S150, the adjustment module 150 sets the top-level message object MS according to the idle area Ai. As long as a suitable idle area Ai can be found, the uppermost message object MS will be set in the idle area Ai first.

In step S160, the adjustment module 150 sets the top-level message object MS according to the occludeable object OBn and the participant window Wk. In this step, the adjustment module 150 sets the window control items of the obscurable object OBn and the participant window Wk to transparent, so that the top message object MS can cover the obscurable object OBn and the participant window Wk to increase the settings. The flexibility of the top-level message object MS.

According to the above embodiment, the electronic device 100 can automatically analyze whether there is enough idle area Ai to set the top-level message object MS, or find out less important obscurable objects OBn or participant windows Wk from the presentation PT. After analyzing the idle area Ai, the obscurable object OBn or the participant window Wk, the top message object MS can be automatically moved to avoid obscuring important content in the presentation PT.

Please refer to Figure 12, which illustrates the situation of the top-level message object MS on different pages SDp of the presentation PT. As shown in Figure 12, each page SDp in the presentation PT can be processed according to the above process, so that the top-level message object MS can be set at the best position in each page SDp of the presentation PT.

In summary, although the present disclosure has been disclosed in the above embodiments, they are not used to limit the present disclosure. Those with ordinary knowledge in the technical field to which this disclosure belongs can make various modifications and modifications without departing from the spirit and scope of this disclosure. Therefore, the protection scope of the present disclosure shall be subject to the scope of the appended patent application.

S110, S120, S130, S140, S150, S160: steps

Claims (8)

An electronic device that can automatically adjust the video conference layout, including: a shared window detection module used to detect whether a shared window is a presentation in a video conference; a presentation area analysis module used to detect the When the sharing window is the presentation, it is detected whether there is enough free area for setting a top-level message object in the presentation. The presentation area analysis module includes: a content analysis unit for based on a content definition file of the presentation, Analyze a plurality of content blocks and obtain the position and size of each content block; and a region judgment unit used to judge whether there is a gap between the content blocks based on the position and size of the content blocks. The idle area is sufficient to set the top-level message object; an object analysis module is used to analyze at least one occludeable object in the briefing when the briefing cannot detect the idle area. The object analysis module includes: an An image analysis unit is used to convert the presentation into an image, and perform a semantic segmentation process on the image to obtain a plurality of segmented objects; and a key analysis unit is used to analyze the key possibility of each segmented object, based on The key possibilities output the at least one obscurable object; a participant window analysis module used to detect at least one participant window when the presentation cannot detect the idle area; and An adjustment module is used to set the top-level message object based on the idle area when the briefing detects the idle area; and is used to set the top-level message object based on the at least one maskable area when the briefing cannot detect the idle area. object and the at least one participant window, and the top-level message object is set, wherein the adjustment module sets the window control item of the at least one occludeable object and the at least one participant window to transparent, so that the top-level message object capable of covering the at least one obscurable object and the at least one participant window. The electronic device that can automatically adjust the layout of a video conference as described in claim 1, wherein the sharing window detection module includes: a window analysis unit for detecting the sharing window and obtaining a window title of the sharing window; and a task analysis unit for searching an application name in a task management form based on the window title, and analyzing whether the shared window is the presentation based on the application name. The electronic device that can automatically adjust the layout of a video conference as described in claim 1, wherein the content analysis unit is used to convert the idle area between the content blocks when there is not enough space to set the top-level message object. Decompose some content blocks into at least one visual text, at least one picture or at least one hyperlink text, and obtain the position and size of the at least one visual text, the at least one picture and the at least one hyperlink text; and the area determination unit It is further used to determine the at least one visual text, the at least one picture and the position and size of the at least one hyperlink text. Whether there is enough free area between the at least one picture and the at least one hyperlink text to set the top-level message object. The electronic device that can automatically adjust the layout of a video conference as described in claim 1, wherein the participant window analysis module includes: a face detection unit for detecting at least one face using a face detection program; and A window definition unit is used to obtain the at least one participant window based on the at least one face. A method for automatically adjusting the layout of a video conference, including: in a video conference, detecting whether a shared window is a presentation; if the sharing window is a presentation, detecting whether there is enough to set a top-level message object in the presentation an idle area; if the idle area is detected in the presentation, set the top-level message object based on the idle area; if the idle area cannot be detected in the presentation, analyze at least one blockable area in the presentation object; if the idle area cannot be detected in the presentation, detect at least one participant window; and if the idle area cannot be detected in the presentation, based on the at least one occludeable object and the at least one participant Window, set the top-level message object, analyze a plurality of content blocks based on a content definition file of the presentation, and obtain the position and size of each content block and the position based on the content block The placement and size are determined to determine whether there is enough free area between the content blocks to set the top-level message object; wherein the presentation is converted into an image, and a semantic segmentation process is performed on the image to obtain a plurality of segmented objects. Analyzing the focus possibilities of each segmented object and outputting the at least one occludeable object according to the focus possibilities; and wherein the window control items of the at least one occludeable object and the at least one participant window are set to transparent, so as to The topmost message object can cover the at least one obscurable object and the at least one participant window. The method of automatically adjusting the layout of a video conference as described in request item 5, wherein in the video conference, the steps of detecting whether the shared window is a presentation include: detecting the shared window; obtaining a window title of the shared window; Based on the window title, an application name is searched in a task management form; and based on the application name, it is analyzed whether the shared window is the presentation. The method for automatically adjusting the layout of a video conference as described in request item 5, wherein if there is not enough idle area between the content blocks to set the top-level message object, the content blocks are decomposed into at least one Visual text, at least one image or at least one hyperlink text; Obtain the position and size of the at least one visual text, the at least one picture, and the at least one hyperlink text; and determine the at least one visual text, the at least one picture, and the at least one hyperlink text based on the position and size of the at least one visual text, the at least one picture, and the at least one hyperlink text. Whether there is enough free area between the visual text, the at least one picture and the at least one hyperlink text to set the top-level message object. The method for automatically adjusting the layout of a video conference as described in claim 5, wherein the step of detecting the at least one participant window includes: using a face detection program to detect at least one face; and obtaining the information based on the at least one face. The at least one participant window.

Images