TWI661722B - Video processing method and video processing system - Google Patents

Abstract

An image processing method includes an image input program, an image frame distinguishing program, an image recognition program, and an image configuration program. The image input program inputs a main image frame and a first additional image frame. The image frame distinguishing program defines at least a first area and a second area of the main image frame. The image recognition program assigns a first image weight to a first region corresponding to the main image frame, and assigns a second image weight to a second region corresponding to the main image frame. The image configuration program covers at least a part of the first image area or the second area of the main image frame according to the first image weight and the second image weight.

Description

Image processing method and image processing system

The present invention relates to an image processing method and system, and more particularly to an image processing method and an image processing system capable of automatically adjusting the configuration of an image frame.

With the rapid development of computer technology, display technology and network technology, people can share a lot of information in real time through the Internet, from simple text messages or image information to even composite information. In terms of display technology, a large-size, high-resolution display screen can simultaneously display an image information, a text information, and a graphic information in a display screen. In terms of network technology, with the increase of bandwidth and Internet connection speed, real-time information transfer has also led to the sharing or operation mode of webcasting.

For this reason, online live broadcast software, such as Open Broadcaster Software (OBS), has been developed to integrate various information and stream it to online to provide viewers with a link to watch.

The current online live broadcast software allows users to set the required screen configuration by themselves. Please refer to Figure 1. This example is an example of the online game player's live sharing game process. Figure 1 is displayed on the player (user One of the displays on a display screen) is to be streamed to an image frame 20 on the line. The image frame 20 includes a main image frame 21 and a live image 22. The main video screen 21 is a main game screen and includes a function list 211 and a dialog box 212.

The user can set the position of the live image 22 on the main image screen 21 according to his personal preference. Here, the live image 22 is arranged at the lower right corner of the main image screen 21.

After the game starts, as the game scene changes, the configuration position of the function list 211 or dialog box 212 may be adjusted, and therefore it may be Will be covered by the live image 22, as shown in FIG. 1, part of the function list 211 is covered by the live image 22. As a result, the player will not be able to obtain the game information clearly, and must move the position of the real-time image 22 again, resulting in inconvenient operations and reduced game smoothness.

In order to improve the above situation, online live broadcast software capable of setting multiple sets of configuration positions has also been developed accordingly. Although there are multiple sets of configuration settings available for gamers to use, if the situation of the scene configuration changes frequently during the game, the player must switch the configuration from time to time, which is also a very inconvenient way of use.

Therefore, how to enable the content of the display screen to automatically adjust the configuration position according to the demand under the condition of live live online is one of the important issues at present, and it is still the goal that the industry must strive to achieve.

In view of this, it is an object of the present invention to provide an image processing method capable of automatically adjusting an arrangement position of an image frame on a display surface of a display.

To achieve the above object, the present invention provides an image processing method, which is capable of automatically adjusting the arrangement position of a plurality of image frames on one display surface of a display. The image processing method includes an image input program, an image frame discrimination program, an image recognition program, and an image configuration program. The image input program inputs a main image frame and a first additional image frame. The main image frame and the first additional image frame are used for displaying on a display surface of a display. The image frame distinguishing program defines at least a first area and a second area of the main image frame. The image recognition program assigns a first image weight to a first region corresponding to the main image frame, and assigns a second image weight to a second region corresponding to the main image frame. The image configuration program covers at least a part of the first image area or the second area of the main image frame according to the first image weight and the second image weight.

According to an embodiment of the present invention, when the weight of the first image is greater than the weight of the second image, the content presented by the main image frame that can represent the first region is more important than the content presented by the main image frame of the second region. Therefore, the first additional image frame covers at least part of the second area of the main image frame.

According to an embodiment of the present invention, the image processing method further includes an image frame adjustment program, which adjusts the size of the first additional image frame according to the size of the second area of the main image frame to make it fit the second area. size.

According to an embodiment of the present invention, the image processing method further includes an image comparison program and a message prompting program. The image comparison program compares the main image frame with a preset image frame and generates a comparison value. The message prompting program generates a prompting signal based on the comparison value. The prompt signal is, for example, a text or a voice.

According to an embodiment of the present invention, the image processing method further includes an image temporary storage program, an image comparison program, and a message prompting program. The image temporary storage program sequentially stores the main screen images. The image comparison program compares the contents of these main image frames and generates a comparison value. The message prompting program generates a prompting signal based on the comparison value.

According to the above, an image processing method for automatically adjusting the position of an image frame on the display surface of a display according to the present invention is to partition the main image frame for image recognition, and according to the importance of the image frame in each area Degree or complexity, and each area is given an image weight, and finally the additional image frame is arranged in an area with a lower image weight (for example, a less important area). In this way, when the main image screen is changing, the additional image screen can automatically change the arrangement position as required, without affecting the user's observation of the main image screen.

20‧‧‧Image screen

21‧‧‧ main image screen

211‧‧‧Feature List

212‧‧‧Dialog Box

22‧‧‧Live

30, 50‧‧‧ image processing system

31, 51‧‧‧Image capture module

32, 52‧‧‧Image area division module

33, 53‧‧‧Image recognition module

34, 54‧‧‧Image integration module

35, 55‧‧‧Image output module

36, 56‧‧‧Image temporary storage module

37, 57‧‧‧Image comparison module

38, 58‧‧‧ Message reminder module

A1 ~ A9‧‧‧area

P01 ~ P08‧‧‧Image processing method flow

P11 ~ P18‧‧‧Image processing method flow

V01, V11‧‧‧ main image screen

V02, V12‧‧‧‧ The first additional image screen

V03, V13‧‧‧‧Second additional image screen

V04, V14‧‧‧‧Integrated image screen

V05, V15‧‧‧ Prompt message

W01 ~ W09‧‧‧Image weight

Figure 1 shows an image of a conventional online live broadcast software transmitted to an online video.

FIG. 2 is a flowchart showing an image processing method according to the first embodiment of the present invention.

FIG. 3 is a block diagram of an image processing system according to the image processing method according to the first embodiment of the present invention.

FIG. 4 is a schematic diagram showing the division of a main image frame by the image area division module.

FIG. 5 is another flowchart showing the image processing method of the first embodiment.

FIG. 6 is another block diagram showing the image processing system of the first embodiment.

FIG. 7 is another flowchart showing the image processing method of the first embodiment.

FIG. 8 is a block diagram showing an image processing system according to a second embodiment of the present invention.

FIG. 9 is a flowchart illustrating an image processing method according to a second embodiment of the present invention.

The content of the present invention will be explained below through embodiments. The embodiments of the present invention are not intended to limit the present invention to be implemented in any specific environment, application or special manner as described in the embodiments. Therefore, the description of the embodiments is only for the purpose of explaining the present invention, rather than limiting the present invention. It should be noted that in the following embodiments and drawings, components not directly related to the present invention have been omitted and not shown; and the dimensional relationship between the components in the drawings is only for easy understanding, and is not used to limit the actual proportion.

Referring to FIG. 2, an image processing method according to a first embodiment of the present invention includes an image input program P01, an image frame discrimination program P02, an image recognition program P03, and an image configuration program P04. Please refer to FIG. 3 again, which is an image processing system 30 that is used in conjunction with the first embodiment of the present invention. The image processing system 30 includes an image capture module 31, an image area division module 32, an image recognition module 33, an image integration module 34, and an image output module 35. Hereinafter, please refer to FIG. 2 and FIG. 3 at the same time to describe the image processing method according to the first embodiment of the present invention.

The image input program P01 inputs a main image frame V01, a first additional image frame V02, and a second additional image frame V03 to the image capturing module 31. In this embodiment, the main video frame V01 is a game screen, which can be obtained through online transmission for an online game, or from a computer or game console for a local game; first The additional image frame V02 is a real-time image captured and output by a photographic lens; the second additional image frame V03 is a picture, such as a logo, which can be pre-designed and stored in the local computer.

The image screen division program P02 is connected by the image area division module 32. The main image frame V01 output by the image capturing module 31 is received, and nine areas A1 to A9 are defined (as shown in FIG. 4). In this embodiment, the main image frame V01 defines the areas A1 to A9 in an equal nine-grid manner. In other embodiments, the number and size of the area divisions may be different according to requirements.

The image recognition program P03 receives the main image picture V01 of the area defined by the image area division module 32 by the image recognition module 33, performs image recognition on the nine areas of the main image picture V01, and assigns each area according to the recognition result. Image weights W01 ~ W09. In other words, the image recognition module 33 will assign the first area A1 to a first image weight W01, the second area A2 to a second image weight W02, the third area A3 to a third image weight W03, and the fourth area A4. A fourth image weight W04, a fifth area A5 corresponds to a fifth image weight W05, a sixth area A6 corresponds to a sixth image weight W06, a seventh area A7 corresponds to a seventh image weight W07, and an eighth area A8 corresponds to a first The eight image weight W08 and the ninth area A9 correspond to a ninth image weight W09.

Here, the definition of image weight represents the importance of the image frame. For example, areas where text and numbers appear may indicate that there may be dialog boxes or instant messages. Therefore, higher image weights will be given. The single-tone areas represent areas that may have less important information, and therefore will be given a lower image weight. In addition, the location of each zone will also affect the image weight. For example, the center of the image frame is generally the main or important game scene. Therefore, the image weight of the central area will also be higher than the image weight of the surrounding areas and corner areas.

The image configuration program P04 is configured by the image integration module 34 to allocate the first additional image frame V02 and the second additional image frame V03 to nine areas A1 to A9 according to the first image weight W01 to the ninth image weight W09. An area and cover the main image frame V01 corresponding to the area to generate an integrated image frame V04. In this embodiment, the image weight of the third region A3 and the ninth region A9 is the lowest. For this reason, the first additional image frame V02 is arranged in the third region A3, and the second additional image frame V03 is It is located in the ninth area A9. In other words, the additional image frame will be preferentially arranged in the area with lower image weight. In addition, the first additional video frame V02 and the second additional video frame V03 may also have an important degree. Differently, this embodiment is described by taking the first additional video frame V02 as more important than the second additional video frame as an example. Finally, the image integration module 34 outputs the integrated image picture V04 to the image output module 35, and the image output module 35 outputs it to a display surface (not shown) of a display for operation by the game player. And stream it to the web for viewing by connected viewers.

In the above embodiments, when the scales of the first additional video frame V02 and the second additional video frame V03 are smaller than the sizes of the third area A3 and the ninth area A9, respectively, the sizes thereof need not be adjusted. When the scales of the first additional image frame V02 and the second additional image frame V03 are larger than the sizes of the third area A3 and the ninth area A9, respectively, please refer to FIG. 5 again. In the image recognition program P03, And an image picture adjustment program P05 is added between the image arrangement program P04, which can appropriately adjust the size of the first additional image picture V02 and the second additional image picture V03 through the image integration module 34 so that it conforms to the third area A3 And the size of the ninth region A9.

As mentioned above, in order to respond to changes in the scene at any time, the above-mentioned image processing method can be automatically executed after a preset time to achieve the purpose of automatically changing the configuration of the image screen in real time. The preset time can be set by the user or set by the software developer in the software. The shortest execution time can be an image frame time.

Hereinafter, please refer to FIG. 6 and FIG. 7 to further explain the image processing method according to the first embodiment of the present invention. The image processing method further includes an image temporary storage program P06, an image comparison program P07, and a message prompting program P08. In the process of live broadcast, sometimes the screen is interrupted due to network disconnection or player suspension. If there is no prompt at this time, the viewers will not be able to obtain the current status and be confused. Therefore, in this embodiment, in addition to outputting the main image screen V01 to the image area division module 32, the image capturing module 31 also outputs the main image screen V01 to an image temporary in the image temporary storage program P06. The storage module 36 (as shown in FIG. 6) stores the main image screen V01 in order within the capability range (capacity of the temporary storage memory) of the image temporary storage module 36.

The image comparison program P07 is based on an image comparison module 37 The main image picture V01 stored in the temporary storage module 36 is compared with a preset image picture, and a comparison value is generated accordingly. Generally speaking, when the signal is interrupted, the computer screen will display a black screen, so the default image screen can be a black screen. However, the picture state that may be generated when the signal is interrupted may also have other forms, such as a snowflake picture. Therefore, the image processing system 30 may include a plurality of preset picture pictures for comparison.

The message prompting program P08 outputs a prompting message V05 by a message prompting module 38 according to the comparison value generated by the image comparison module 37. The comparison value represents the similarity between the main image frame V01 and the preset image frame, and the prompt message V05 (such as sound, text, or specific image) is activated when the comparison value falls within a certain range. In this embodiment, the prompt message V05 is a text message output by the message prompt module 38 to the image output module 35 to notify the viewer that a signal interruption currently occurs.

Please refer to FIG. 8 and FIG. 9 again, which respectively show a block diagram of an image processing system 50 and a flowchart of an image processing method according to a second embodiment of the present invention. As shown in FIG. 8, the image processing system 50 includes an image capture module 51, an image area division module 52, an image recognition module 53, an image integration module 54, an image output module 55, An image temporary storage module 56, an image comparison module 57, and a message prompting module 58.

The image processing method includes an image input program P11, an image screen discrimination program P12, an image recognition program P13, an image screen adjustment program P15, an image configuration program P14, an image temporary storage program P16, and an image comparison program P17. And a message reminder program P18.

The image capturing module 51 receives a main image frame V11, a first additional image frame V12, and a second additional image frame V13, which are the same as the main image frame V01, the first additional image frame V02, and The second additional image frame V03 is the same, and will not be repeated here.

Different from the first embodiment, the image temporary storage program P16 is sequentially received and stored by the image temporary storage module 56 to store one integrated image frame V14 output by the image integration module 54. The image comparison program P17 is compared by the image comparison module 57 based on the plurality of integrated image frames V14 stored by the image temporary storage module 56 to generate a comparison value. The message reminder program P18 is produced by the message reminder module 58 based on the image comparison module 57 A comparison message is output and a prompt message V15 is output. The prompt message V15 is the same as the first embodiment, and may be a sound, a text, or an image. In this embodiment, the message prompting module 58 outputs a prompting message V15 of a text message to the image outputting module 55.

In summary, according to the present invention, an image processing method and an image processing system for automatically adjusting the arrangement position of an image frame on a display surface of a display are based on a preset time to partition the main image frame for image recognition. According to the importance or complexity of the image frame in each area, an image weight is given to each area, and finally the additional image frame is arranged in an area with a lower image weight (for example, a less important area). In this way, when the main video screen is changing, the additional video screen can automatically change the placement position as the screen changes to avoid the additional video screen from obscuring important main video screen information without affecting the user (gamer). The operation also increases viewers' willingness to watch. In addition, when the image screen is stagnant or there is no image signal, timely display of the notification message can also make the viewer not to be confused.

The invention complies with the requirements of an invention patent, and a patent application is filed according to law. However, the above is only a preferred embodiment of the present invention, and it cannot be used to limit the scope of patent application in this case. For those who are familiar with the skills of this case, equivalent modifications or changes based on the spirit of the invention of this case should be included in the scope of patent application below.

Claims (8)

An image processing method is to automatically adjust the arrangement position of an image frame on a display surface of a display, including: an image input program, which inputs a main image frame and a first additional image frame, and the main image frame And the first additional image frame is used for displaying on the display surface; an image frame distinguishing program is defined on the main image frame at least a first area and a second area; an image recognition program is given a first area The image weight corresponds to the first area of the main image frame, and a second image weight is assigned to the second area of the main image frame. An image allocation procedure is based on the first image weight and the second image. Weight, and the first additional image frame covers at least part of the first region or the second region of the main image frame to generate an integrated image frame; an image comparison program is to compare the main image frame with a pre- It is assumed that the image frames are compared and a comparison value is generated; and a message prompting program generates a prompt message according to the comparison value. An image processing method is to automatically adjust the position of an image picture on a display surface of a display, and includes: an image input program for inputting a main image picture and a first additional image picture, the main image picture and the first An additional image frame is used for displaying on the display surface; an image frame distinguishing program is defined in the main image frame at least a first area and a second area; an image recognition program is assigned a first image weight correspondence Assigning a second image weight to the first area of the main image frame and corresponding to the second area of the main image frame; an image allocation procedure based on the first image weight and the second image weight, and Covering the first additional image frame with at least part of the first region or the second region of the main image frame to generate an integrated image frame; an image temporary storage program for storing the integrated image frame; an image comparison procedure , Comparing the integrated image frames with each other, and generating a comparison value; and a message prompting program, generating a reminder message based on the comparison value. An image processing method is to automatically adjust the position of an image picture on a display surface of a display, and includes: an image input program for inputting a main image picture and a first additional image picture, the main image picture and the first An additional image frame is used for displaying on the display surface; an image frame distinguishing program is defined in the main image frame at least a first area and a second area; an image recognition program is assigned a first image weight correspondence Assigning a second image weight to the first area of the main image frame and corresponding to the second area of the main image frame; an image allocation procedure based on the first image weight and the second image weight, and Covering the first additional image frame with at least part of the first area or the second area of the main image frame to generate an integrated image frame; inputting a second additional image frame; defining a third area on the main image frame ; Assign a third image weight corresponding to the third area of the main image frame; and according to the first image weight, the second image weight, and Third image weights, while the second additional video frame covered by the third region of the main video picture portion. The image processing method according to any one of claims 1, 2 or 3, wherein the first image weight is greater than the second image weight, and the first additional image frame covers at least part of the main image frame. The second area. The image processing method as described in claim 4, further comprising: an image picture adjustment program that adjusts the size of the first additional image picture according to the size of the second area of the main image picture so that it conforms to the second The size of the area. The image processing method according to claim 3, wherein the first image weight and the second image weight are greater than the third image weight. An image processing system that automatically adjusts the position of an image picture on a display surface of a display includes: an image capture module that captures a main image picture and a first additional image picture; an image temporary storage A module that stores the main image screen; an image area division module that defines at least a first region and a second region on the main image screen; an image recognition module that assigns a first image weight corresponding to The first area of the main image frame and a second image weight assigned to the second area of the main image frame; an image integration module based on the first image weight and the second image weight, and Covering the first additional image frame with at least part of the first area or the second area of the main image frame, and generating an integrated image frame based thereon; an image comparison module comparing the main image frame with a The preset image screen is used to generate a comparison value; and a message prompt module is used to generate a prompt message according to the comparison value. An image processing system that automatically adjusts the position of an image frame on a display surface of a display includes: an image capture module that captures a main image frame and a first additional image frame; an image area division A module defines at least a first region and a second region on the main image screen; an image recognition module assigns a first image weight corresponding to the first region of the main image screen, and assigns a first Two image weights correspond to the second area of the main image frame; an image integration module is based on the first image weight and the second image weight to cover the first additional image frame with at least part of the main image The first area or the second area of the screen is used to generate an integrated image picture; an image temporary storage module is used to store the integrated image picture; an image comparison module is used to compare these integrated image pictures, To generate a comparison value; and a message prompt module, which generates a prompt message according to the comparison value.