TW202213992A - Live broadcasting method for real time three-dimensional image display - Google Patents

Abstract

A live broadcast for real time three-dimensional (3D) image display is disclosed in the present disclosure and includes: recording an object to acquire 3D images from the object; performing an image processing of the 3D images by a 3D live broadcast algorithm module; generating a video deployment setup according to at least one video choice of a user terminal vie an AI learning module; and optimizing the 3D image according to the video deployment setup and a usage circumstance of a live broadcast system and displaying the corresponding 3D image to the user terminal in cooperated with the video deployment setup after optimizing the 3D image.

Description

Live broadcast method of real-time 3D image display

The invention relates to a live broadcast method, in particular to a live broadcast method with a real-time three-dimensional image display function.

Since the invention of instant messaging software, after many years of development and technological progress, communication and chat between the two parties have gone from pure text at the beginning to video plus text communication and chat, which has gradually become a part of modern people's lives. In addition to the traditional instant messaging function, after a period of development, many applications have emerged, such as live shopping, online instant meetings, live entertainment and so on. After the chat method of instant messaging evolved from the expression methods of emoticons, selfies, and filters to the social trend of "making everything into pictures and videos", live broadcasting, a new field of self-expression, was born.

The shopping method of online live broadcast is characterized by live broadcast, and the product introduction content is broadcast to the Internet immediately after a short time gap in the background, so user terminals or viewers can see the product introduction content in real time, and can interact with the live broadcast. The master interacts instantly. As shown in FIG. 1, it shows a schematic diagram of live selling goods to the user terminal 11 on the Internet through the software (APP) of the mobile device through the live broadcast host 12. This transaction and communication method is different from traditional TV shopping and online shopping. E-shopping and TV-shopping are pre-recorded and then played at a specific time, so shopping is not instant; while traditional online stores use a hanging network to place product introduction pictures on the Internet, and then cooperate with the online store operator's real-time service to complete product transactions.

The visibility and instant interaction of live broadcasts are gradually being sought after by modern people. Therefore, when this method is applied to online shopping, it can save a lot of time for people to purchase items outside, and in the process of shopping , full of entertainment fun, but also improve the immediacy of shopping. However, in this way of online real-time shopping, although there are hosts or professionals introducing products on the live broadcast, shoppers watch 2D product images on the display screen in a remote indoor environment or a mobile situation. Decide whether to shop or not. Since the 2D product introduction video viewed through the user terminal's mobile phone or smart TV is not viewed at the product site, there will be a gap with the actual item. When the consumer terminal receives the physical product, it is found that It is quite different from the psychological expectation.

The reason why live chat, entertainment, and shopping are popular and popular among people is that in addition to the real-time, vivid picture visibility and the atmosphere of the live broadcaster, viewers can interact with the live broadcaster or leave a message, showing The screen can directly display the text interaction results of the viewer and the live broadcaster. This interactive behavior mode can greatly reduce the communication cost of both parties. In the application example of live shopping, it can shorten the time between shoppers and sellers. distance, and this method is also suitable for handheld mobile devices, such as mobile phones, tablet computers, etc. Therefore, the range of applicable platforms for live broadcasting is wider and more direct than various instant messaging software, TV shopping, and flat-screen online shopping in the past, and it also breaks the boundaries between the past platforms.

However, the current live broadcast system 10, as shown in FIG. 1, still relies on the traditional two-dimensional (2D) video and image display technology. There is still a big difference in understanding between shopping and the real world. For example, in the scene where the live broadcaster or the seller is explaining cosmetics, as shown in Figure 2, since the image 20 broadcasted on the traditional 2D video does not have deep stereoscopic information, the seller has already applied cosmetics on the live broadcast scene. In terms of color and shape, it is difficult for viewers at the far end to feel the bright and three-dimensional color and gloss of the live broadcast due to the limitation of image formation, which greatly reduces the display effect of the product. This is because when such products are displayed, the complete features of the product cannot be expressed without the use of three-dimensional light and shadow with depth information and three-dimensional images. Similar to this application example, there have been many limitations in the current two-dimensional image display.

Therefore, how to solve the flat display limitation of the live broadcast system and improve the efficiency of communication and communication through the improvement of the design has become one of the important issues to be solved by the project.

The technical problem to be solved by the present invention is to provide a live broadcast method capable of displaying three-dimensional images in view of the deficiencies of the prior art, and the live broadcast method will not cause the degradation of the image quality or the delay problem when users watch.

In order to solve the above-mentioned technical problems, one of the technical solutions adopted by the present invention is to provide a live broadcast method with a real-time 3D image display function, which includes: capturing an image of an entity, and obtaining a 3D image of the entity; an arithmetic module for image processing of three-dimensional images; according to at least one video selection condition of a user terminal, through an artificial intelligence learning module, a video allocation combination is generated; and according to the video allocation combination, the three-dimensional image is optimized; and The optimized three-dimensional image is displayed in the user terminal in combination with the video allocation and combination of the user terminal.

One of the beneficial effects of the present invention is that, in the live broadcasting system and the live broadcasting method provided by the present invention, in order to reduce the real-time flow of data, two-dimensional and three-dimensional images are coexisted, and the method of mixed editing is adopted, so that the viewer can focus on the needs. On the product or person concerned, to watch the best and most needed 3D images, and this coexistence of 2D and 3D images can be selected on the user interface to set 2D image viewing or 3D image viewing, That is to say, the user can change the position of the 3D image to be viewed at any time, which can be set in a regional or a full-image 3D display. For the 2D image, the left image or the right image in the 3D left-right image can be selected as the display image of the 2D image. In addition, in addition to the stereoscopic display mode of the user terminal device, a stereoscopic display device can also be placed on the host or the control end of the cloud backend to provide a stereoscopic display preview of the host and interactive stereoscopic display with the user terminal. In addition, the user terminal or the live broadcast host terminal can easily select the horizontal or vertical mode to display the 3D live broadcast through the horizontal and vertical control device of the 3D live broadcast image.

For a further understanding of the features and technical content of the present invention, please refer to the following detailed descriptions and drawings of the present invention. However, the drawings provided are only for reference and description, and are not intended to limit the present invention.

The following is a specific embodiment to illustrate the implementation of the "live streaming method for real-time 3D image display" disclosed in the present invention, and those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the concept of the present invention. In addition, the drawings of the present invention are merely schematic illustrations, and are not drawn according to the actual size, and are stated in advance. The following embodiments will further describe the related technical contents of the present invention in detail, but the disclosed contents are not intended to limit the protection scope of the present invention. In addition, the term "or", as used herein, should include any one or a combination of more of the associated listed items, as the case may be.

[Embodiment of the live broadcast method of the present invention]

3 shows a flow chart of the steps of the live broadcast method for real-time 3D image display of the present invention, FIG. 4A is a schematic diagram of image adjustment of a 3D image according to the present invention, and FIG. A schematic diagram, FIG. 5 is a system block diagram of a live broadcast system with a real-time 3D image display function of the present invention.

As shown in FIG. 3 , and with reference to the component labels in FIG. 5 , an embodiment of the present invention provides a live broadcast method for real-time 3D image display, including the following steps. In step S301, an entity is imaged, and a three-dimensional image of the entity is acquired, and an image capture module 51 is used to photograph the live broadcaster. The image capture module 51 may be a three-dimensional video camera or a two-dimensional video camera etc., any electronic device capable of taking pictures can use the image capturing module 51 of the present invention.

In the preferred embodiment of the present invention, the image capturing module 51 is a three-dimensional camera. To solve the shortage of the information displayed by the three-dimensional image, the shooting and compilation of the three-dimensional image are used in the preferred embodiment of the present invention. The above-mentioned 3D image shooting uses a real-time embedded 3D dual-lens camera to capture 3D images. Alternatively, in different embodiments, a 2D camera may be used to first obtain the 2D image of the live broadcast host, and then convert the 2D image into a 3D image through a 2D to 3D image conversion technology. How to capture the 3D image or how to The technology of converting a 2D image into a 3D image is well known to those skilled in the art, and will not be repeated here. In order to reduce the real-time flow of data, the two-dimensional and three-dimensional images coexist, and the method of mixed editing is adopted, so that the user terminal can focus on the products or characters that need attention when watching, so as to watch the best and most needed 3D images. , and this coexistence of 2D and 3D images allows you to select and set 2D image viewing or 3D image viewing on the user interface, which means that the user terminal can change the position of the 3D image to be viewed at any time, and can set In regional, or full-scale 3D display. In addition, it should be noted here that the amount of data that needs to be transmitted for 3D images is greater than the amount of data that needs to be transmitted for 2D images. During video playback, a smaller amount of data is transmitted than when full-image 3D display is performed. Therefore, the smoothness of playback during live broadcast can be maintained and the generation of data transmission delay (lag) can be reduced.

In step S302 , the image processing of the three-dimensional image is performed by a three-dimensional live broadcast calculation module 52 . When the image capture module 51 of the present invention receives the three-dimensional image 40, it will perform image processing, because the mobile devices used by the user terminals are not the same, and the locations or environments of the user terminals are also different, different mobile devices will There are different data transmission speeds and different locations, so the data transmission speeds are also different. Therefore, in the step of optimizing the 3D image, the 3D image is cut and adjusted, and the combination is arranged and combined according to the video, through the artificial intelligence learning module 54 , to optimize the 3D image.

The live broadcast method of the real-time three-dimensional image display function of the present invention adjusts images according to different data transmission speeds. In the existing stereo vision technology, a whole image is processed, so the amount of data at least twice that of a two-dimensional image is often generated. The most common situation is that the image is dropped (frame), mosaic or lag is serious, which reduces the fun and patience of use. Therefore, the present invention automatically detects the image, and performs foreground, background, and image segmentation for adaptation effects. As shown in FIG. 4A , the three-dimensional image 40 is further subjected to image error tolerance and correction, and detection and separation are performed. A plurality of objects in the 3D image 40 . If the object is a person, perform image processing such as image softening filter, background blur, filter effect, or beautifying skin on the object; if the object is an object, perform image processing such as background peeling, edge enhancement, contrast enhancement or image enlargement on the object ; If the object is both a person and an object, the calculation of color, beauty, and graphic edge optimization is performed on the image. Background peeling is to separate the background in the three-dimensional image 40 from the image, so different backgrounds can be replaced in subsequent steps according to the usage habits of the user terminal. Finally, the optimized and image-processed three-dimensional image 40 is output.

In addition, the live broadcast method of the present invention further includes capturing images of at least one user terminal to obtain a three-dimensional image of the user terminal, and then performing a multi-person interactive stereoscopic display mode. In addition to the three-dimensional display, the live broadcast method of the present invention can also use the one-to-many display mode of the live main broadcast. In addition to the general one-to-one mode, a three-dimensional display mode in which multiple people interact simultaneously and simultaneously can be used. Therefore, the live broadcast of the present invention The method can not only provide intuition, but also can improve the fun of entertainment, in addition to avoiding shopping mistakes, it can improve the pleasure of human life.

Through the image processing of the three-dimensional image of the present invention, the user terminal can see the target display area at a glance, and at the same time, the data load of the network bandwidth can be effectively reduced, and the live broadcast method of the present invention is more suitable for the real-time live broadcast system. The video experience of the user terminal can be improved, so that the phenomenon of video freezing will not occur. In the present invention, the 3D image can be processed by a 3D live broadcast calculation module 52 . The reason why the image processing of the 3D image of the present invention can effectively reduce the data load of the network bandwidth is that the local 3D image data is much smaller than the traditional full-image 3D data, and with the data compression technology, only It will have more bytes to several thousand bytes than the data of general full-image 2D images.

In step S303, according to at least one video selection condition of a user terminal, an artificial intelligence learning module 54 is used to generate a video allocation combination. The at least one video selection condition of the user terminal is the use inertia of the user terminal to a live broadcast system, and the use habits of each user terminal are different. The live broadcast method of the present invention uses an artificial intelligence learning module 54. The usage inertia when using the live broadcast software of the present invention displays the most preferred playback mode of the user terminal. Along with the habit of the user terminal, it is also recorded by the learning mode of the artificial intelligence learning module 54. The next time the user terminal opens the same or similar screen, it can automatically display the area that the user terminal is used to or a stereoscopic image with a full image. show.

Further, each user terminal will have personal usage habits. Therefore, when the user terminal watches the live broadcast, the artificial intelligence learning module 54 of the present invention will record the usage habits of each user terminal when watching the live broadcast. For example, as shown in FIG. 4B , when the user terminal is watching the live broadcast, the artificial intelligence learning module 54 will record the usage pattern of the user terminal, for example, what kind of gift the user terminal likes to send, the user terminal will target that specific gift. Switch between two-dimensional and three-dimensional images of people, objects, or people and objects. The live broadcast method of the present invention records the usage habits of each user terminal when watching the live broadcast, and in the artificial intelligence learning module 54, according to the recorded usage habits of each user terminal for watching the live broadcast, when the user terminal watches the next time During live broadcast, the artificial intelligence learning module 54 will display a specific live broadcast screen to the user terminal according to the recorded usage habits, so that the user terminal can have a better experience in using live broadcast viewing.

In addition, in step S304, the three-dimensional image is optimized according to the video deployment combination and the use environment of the live broadcast system. The optimization of the three-dimensional image may be based on the use environment of the live broadcast system, such as the number of viewers and the speed of the network, to optimize the three-dimensional image. In addition, in the live broadcast method of the present invention, in addition to real-time shooting of people or products, the pre-recorded background background can also be added in real time to increase the use scene of virtual reality, which is convenient for the user terminal to change the use scene. To improve the environment to use pleasant somatosensory. In the live broadcast method of the present invention, in addition to providing different video allocation combinations for each user terminal to generate an optimized three-dimensional image, different background patterns can also be selected according to different user terminals. Moreover, the live broadcast method of the present invention is to avoid wrong or inappropriate video or video streaming to the terminal, so no matter whether the user terminal is chatting or communicating with the live broadcaster, or even enjoying the live broadcaster's program performance, or the live broadcaster selling Products, after the image is captured by the camera into the human body or object, in addition to the real-time 3D editing and compression of the image, it will be automatically checked by artificial intelligence in the background of the live broadcast cloud, or manually checked. After the settings or requirements of the user terminal are correctly corresponding to the background system, the corresponding data stream (Data Streaming) is sent to the mobile phone or terminal interaction device of the user terminal, and the user can see the settings and requirements of the terminal. The corresponding display results.

In step S305, the optimized three-dimensional image is displayed in the user terminal in accordance with at least one video selection condition of the user terminal. The live broadcast method of the present invention not only provides the aforementioned foreground and background cutting and display settings required by the user terminal in the background according to the settings of the user terminal, but also sends the corresponding three-dimensional images according to the settings and requirements of the user terminal. to the device of the user terminal. For example, if the user terminal pays for the purchase of a gift, the live broadcast system will send the stereoscopic display image or video of the gift to the device of the live broadcaster according to the corresponding gift purchased by the user terminal, so that the live broadcaster can display the gift on his device. On the display screen of the device, watch the video or image of the corresponding three-dimensional gift.

Through the live broadcast method of the instant three-dimensional image display function of the present invention, in addition to adjusting the output quality of the image according to different data transmission speeds, it can also provide different experience according to different user terminals, so that each user terminal can obtain better quality A viewing experience or a shopping experience.

[Embodiment of the live broadcast system of the present invention]

FIG. 5 shows a system block diagram of the live broadcast system for real-time 3D image display according to the present invention. As shown in FIG. 5, the live broadcast system 50 for real-time 3D image display of the present invention includes an image capture module 51, a 3D live broadcast calculation module 52, a live broadcast system server 53, an artificial intelligence learning module 54, a 3D decoder 55 and a 3D display 56 .

The image capture module 51 can be a 3D video camera or a 2D video camera, and any electronic device capable of taking pictures can use the image capture module 51 of the present invention. In the preferred embodiment of the present invention, the image capturing module 51 is a three-dimensional camera. To solve the shortage of the information displayed by the three-dimensional image, the shooting and compilation of the three-dimensional image are used in the preferred embodiment of the present invention. The above-mentioned 3D image shooting uses a real-time embedded 3D dual-lens camera to capture 3D images. The camera, because it is an embedded type, is built into the electronic device, and the two 3D cameras are directly connected to the handheld device through interfaces, such as Mobile Industry Process Interface (MIPI), serial interface, etc. , and then through the connection between the 3D live broadcast calculation module 52 and the live broadcast system server 53 of the present invention, the 3D images and videos are sent to the user terminal in real time. For personal computers or handheld devices, if there is no built-in 3D camera, an external monocular camera or binocular stereo camera can be used, through the universal serial bus (USB) interface, or wireless (WIFI) method, When connected to the host, after being connected to the live broadcast system of the present invention, data processing is performed according to the processing method or flow of the 2D to 3D or stereo camera.

Alternatively, in different embodiments, a 2D camera may be used to first obtain the 2D image of the live broadcast host, and then convert the 2D image into a 3D image through a 2D to 3D image conversion technology. How to capture the 3D image or how to The technology of converting a 2D image into a 3D image is well known to those skilled in the art, and will not be repeated here. The image capture module 51 can be installed on the devices of the live broadcaster and the user terminal, and can simultaneously obtain the three-dimensional images of the live broadcaster and the user terminal, so as to facilitate the subsequent one-to-one mode or paired mode between the live broadcaster and at least one user terminal. Multiple stereoscopic display modes.

The 3D live broadcast calculation module 52 is connected to the image capture module 51, which is used to receive and optimize the 3D image. The calculation module 52 can also be installed in the live system server 53, which is not limited herein. Further, as shown in FIG. 6 , and referring to FIG. 5 , the 3D live broadcast calculation module 52 includes an image correction unit 521 , an image separation unit 522 , an image synthesis unit 523 , an image correction unit 524 , and an image allocation unit 525 , an image management unit 526 and a video encoding unit 527 . The image correction unit 521 is connected to the image capture module 51 to receive a 3D image, and is used to perform image correction on the received 3D image. During the process of capturing the 3D image or converting the image from 2D to 3D, some errors will be generated. For the problem of image errors or out-of-sync video and audio, the image correction unit 521 corrects the errors generated when the 3D images are captured or converted. The image separation unit 522 is connected to the image correction unit 521 to segment the foreground, background, people or objects in the corrected 3D image, so that specific image processing can be performed on any person or object in the 3D image, for example, The background in the three-dimensional image is cut, so that a specific background pattern can be added to the background in the subsequent image processing, or, in a specific live action, a specific object or image can be displayed three-dimensionally.

The image synthesizing unit 523 is connected to the image separating unit 522, and the separated three-dimensional images can be image-processed for specific patterns, people, or objects, respectively, and the processed specific patterns, people, or objects can be combined by the image synthesizing unit 523. The unit 523 does not just restore the original three-dimensional image, but can combine individual three-dimensional patterns, people or objects with other two-dimensional images, and can also add a specific background pattern to the background through the image synthesis unit 523 . The image modification unit 524 is connected to the image allocation unit 525, and the image modification unit 524 selects and adjusts the viewing angle of the image according to the image capture of different patterns. The 524 also includes general image edge enhancement, image turning, image bokeh, filter effects, or image area magnification.

The image adjustment unit 525 is connected to the image correction unit 524, and the image adjustment unit 525 adjusts the image output of the 3D image according to different network speeds or performance of the mobile device. For example, when the network speed is fast, the 3D image can be completely output, and when the network speed is slow, the 3D image can be partially displayed, and the rest of the image can be displayed in 2D. The image management unit 526 is connected to the image allocation unit 525 to integrate and output the three-dimensional images after image processing. The video encoding unit 527 is connected to the image management unit 526, and the three-dimensional images are converted into data signals by the video encoding unit 527. The data signal of the image is transmitted to the live broadcast system server 53 in a wired or wireless manner.

The live broadcast system server 53 is wirelessly connected to the 3D live broadcast calculation module 52, which can also be called the live broadcast system cloud. The live broadcast system server 53 is used to receive data signals with three-dimensional images. The content will be sent to the live broadcast system server 53 , and then pushed to the device of the user terminal through the live broadcast system server 53 . The artificial intelligence learning module 54 is connected to the live broadcast system server 53. In a preferred embodiment of the present invention, the artificial intelligence learning module 54 can be set on the live broadcast system server 53, or in different embodiments, the artificial intelligence learning module 54 can be set on different servers or computer hosts, and then connected to the live system server 53 in a wireless or wired manner, which is not limited here. The video screen selection conditions or usage habits of each user terminal will be recorded by the learning mode of the artificial intelligence learning module 54, and a video allocation combination will be output. When the user terminal opens the same or similar screen next time, the artificial intelligence learning module 54 can automatically output the video allocation combination according to the video selection conditions.

Therefore, whether the user terminal chats or communicates with the live broadcaster, enjoys the live broadcaster's program performance, or watches the live broadcaster's sales products, after the image is captured by the camera in real time, the human body or object, in addition to the real-time image 3D editing, Except for compression, all of them will be automatically checked by the artificial intelligence learning module 54 on the background of the live broadcast system server 53. After the check is completed, it will be sent out according to the settings or requirements of the user terminal and correctly corresponding to the background system. The corresponding code stream (Data Streaming) is sent to the mobile phone of the user terminal or the terminal interaction device, and the user terminal can see the display results corresponding to the settings and requirements of the terminal.

The three-dimensional decoder 55 is connected to the live broadcast system server 53, or the three-dimensional decoder 55 can also be installed on the live broadcast system server 53, and is connected to the artificial intelligence learning module 54, and the three-dimensional image can be encoded and decoded through the three-dimensional decoder 55, In the live broadcast system 50 of the present invention, the three-dimensional decoder 55 can be applied in a stereoscopic display mode in which the live broadcaster can interact with multiple user terminals. The live broadcast system 50 will display the stereoscopic display image or video of the gift 73 on the mobile device 72 of the live broadcast host 71 according to the corresponding gift purchased by the user terminal, as shown in FIG. 7A, so that the live broadcast host 71 can move On the display screen 74 of the device 72 , the video or image of the corresponding three-dimensional gift 73 is viewed. The three-dimensional display 56 is arranged on the devices of the live broadcast host and the user terminal, and the devices of the live broadcast host and the user terminal receive the three-dimensional image pushed from the live broadcast system server 53 by means of wireless transmission, and display it on the live broadcast host and the live broadcast host through the three-dimensional display 56. on the device of the user terminal. Through the 3D decoder 55, the live broadcast system 50 of the present invention has a 3D stereo bidirectional encoding and decoding technology. As shown in FIG. 7B, on the mobile device 72, in addition to the live broadcast host 71, the stereo video and images can be sent to the user terminal 75 for receiving and viewing. In addition to stereoscopic video and images, the user terminal 75 can also send stereoscopic images or videos to the live broadcast host 71 through the two-way stereoscopic display technology, so that the live broadcast host 71 can also receive the stereoscopic image information of the user terminal 75 in real time. Quickly determine the needs of the user terminal 75 to achieve the purpose of real-time interaction.

In addition, please refer to FIG. 6 , the user terminal also includes the display function of the 3D image at the main end of the live broadcast, and the device of the user terminal also includes the function of the live broadcast system 50 of the present invention, because when watching the live broadcast, the user terminal It is also possible to live broadcast its own image to the live broadcast host, or the user terminal can automatically or manually switch the conversion between two-dimensional and three-dimensional when receiving the video from the live broadcast host. Therefore, when the user terminal receives a two-dimensional or three-dimensional image, if the received two-dimensional image can be converted into a three-dimensional image, the three-dimensional image of the user terminal will also pass through the image correction unit 521, the image separation unit 522, the image synthesis unit 523, the image The correction unit 524, the image allocation unit 525, and the image management unit 526 and other components perform image processing on the three-dimensional image, so that the three-dimensional image on the user terminal can also be displayed as a panoramic three-dimensional image, or can be used for specific people, objects or people and people. The display of 2D or 3D images can be switched actively or manually on the client terminal. How to perform image processing by the 3D live broadcast calculation module 52 has been introduced in the previous chapters, therefore, the details of the relevant image processing will not be repeated here.

For example, when the data transmission speed of the user terminal is not fast, the 3D live broadcast calculation module 52 will automatically display only part of the 3D image on the 3D display 56 of the user terminal, but will not display the panoramic 3D image. When it is too low, the 3D live broadcast calculation module 52 will even automatically switch the 3D image to the 2D image. In addition, the 3D image of the user terminal can be switched automatically, and the 3D image can also be manually switched, and the user terminal can manually designate a specific person, object, or person and object to display the 3D image. In other words, the 3D image of the present invention is not limited to the 3D image of the whole image, and the 3D image of the present invention can also be a partial 3D image. People and objects) are presented in a three-dimensional display, and the rest of the images are presented in a two-dimensional display. Through such an image presentation method, the amount of data transmission during live broadcast can be reduced, and the probability of generating delay in live broadcast display can be reduced.

[The embodiment of the horizontal or vertical display of the three-dimensional live image of the present invention]

In addition, in a preferred embodiment of the present invention, a horizontal and vertical control device 80 for a three-dimensional live video image can be set on the devices of the live broadcast host and the user terminal. Since the device used by the user terminal may habitually use vertical display for viewing or horizontal live display viewing, the present invention also provides a horizontal and vertical control device 80 for 3D live video. For the display orientation setting of the mobile device, the display orientation setting can be It is automatically detected or manually set on the APP software of the user terminal, and is also applicable to the display terminal of the live broadcaster and the monitoring display of the cloud. Therefore, the horizontal and vertical control device 80 for 3D live video of the present invention includes a direction detector 81 and a control interface 82 .

In the automatic detection of the display direction of the live broadcast, the direction detector 81 is equipped with a signal that can sense the gyroscope or the direction sensor inside the mobile device or the handheld device, and the device of the live broadcast host or the user terminal will give feedback according to the terminal device. The direction signal of , modulates the direction of the live broadcast display. The adjustment of this direction will be matched with the 3D live video or video after turning, and the display conversion of the 3D live video will be carried out. Adjustment of the function display position, and other steering settings related to the 3D stereoscopic live broadcast display. The aforementioned setting also includes the setting of the live display function for the three-dimensional display 56 . The control interface 82 is arranged on the mobile devices or handheld devices of the live broadcaster and the user terminal. Further, the control interface 82 can be a controller such as a button, or the control interface 82 can also be a software user interface, which includes the controller's Features. The live broadcaster or the user terminal can automatically flip the screen live broadcast display through the direction detector 81, or can also flip the screen live broadcast display through the control interface 82 to achieve vertical live display viewing or horizontal live viewing.

The horizontal and vertical control device 80 for 3D live video of the present invention has the function of displaying 3D live video in both the horizontal and vertical directions, and has automatic or manual steering detection and display adjustment. For a stereoscopic display terminal that supports only one direction, the horizontal and vertical control device 80 of the 3D live video of the present invention further includes a live video adjuster 83, the live video adjuster 83 is connected to the direction detector 81, and the live video adjuster 83 judges The step of whether to adjust the 3D live display can be shown in FIG. 9 . In step S901 , the direction detector 81 detects that the orientation of the mobile device is changed, or in step S902 , the user performs live video through the control interface 82 Rotate, in step S903, the live image adjuster 83 determines whether the mobile device supports 3D live display horizontally or vertically, if not, in step S904, the live image adjuster 83 can remind the user terminal that the stereoscopic display after turning is not supported , and through the live image adjuster 83, the two-dimensional flat live broadcast display is displayed on the display of the user terminal until the user terminal is turned to display the stereoscopic live display again, and the user terminal can see the stereoscopic live display. On the contrary, in step S905, the live video adjuster 83 adjusts the optimized three-dimensional live video after the turn.

[Advantageous effects of the embodiment]

One of the beneficial effects of the present invention is that, in the live broadcast system and the live broadcast method provided by the present invention, in order to reduce the real-time flow of data, the two-dimensional and three-dimensional images are coexisted, and the mixed editing method is adopted, so that the viewer can focus on the need to pay attention. In order to watch the best and most needed 3D images on the products or characters, and this coexistence of 2D and 3D images, you can choose to set 2D image viewing or 3D image viewing on the user interface, That is to say, the user can change the position of the 3D image to be viewed at any time, which can be set in a regional or a full-image 3D display. For the 2D image, the left image or the right image in the 3D left-right image can be selected as the display image of the 2D image. In addition, in addition to the stereoscopic display mode of the user terminal device, a stereoscopic display device can also be placed on the host or the control end of the cloud backend to provide a stereoscopic display preview of the host and interactive stereoscopic display with the user terminal. In addition, the user terminal or the live broadcast host terminal can easily select the horizontal or vertical mode to display the 3D live broadcast through the horizontal and vertical control device of the 3D live broadcast image.

The contents disclosed above are only preferred feasible embodiments of the present invention, and are not intended to limit the scope of the present invention. Therefore, any equivalent technical changes made by using the contents of the description and drawings of the present invention are included in the application of the present invention. within the scope of the patent.

10: Live broadcast system 11: User terminal 12: Live Master APP: Software 20: Video S301-S305: Steps 40: 3D Image 50: Live system 51: Image capture module 52: 3D Live Calculation Module 521: Image Correction Unit 522: Image Separation Unit 523: Image synthesis unit 524: Image Correction Unit 525: Video Alignment Unit 526: Image Management Unit 527: Video coding unit 53: Live system server 54: Artificial Intelligence Learning Module 55: 3D decoder 56: 3D Display 71: Live Master 72: Mobile Devices 73: Gift 74: Display screen 75: User terminal 80: Horizontal and vertical control device for 3D live video 81: Orientation detector 82: Control interface 83: Live Image Adjuster S901-S905: Steps

FIG. 1 is a schematic diagram of an existing live broadcast system.

FIG. 2 is a schematic diagram of the execution of the existing live broadcast software.

FIG. 3 is a flow chart of the steps of the live broadcast method of the real-time 3D image display function of the present invention.

FIG. 4A is a schematic diagram of image adjustment of a 3D image according to the present invention.

FIG. 4B is a schematic diagram of the artificial intelligence learning module of the present invention learning the usage habits of the user terminal.

FIG. 5 is a system block diagram of the live broadcast system with the real-time 3D image display function of the present invention.

FIG. 6 is a schematic diagram of image processing of the 3D live broadcast calculation module of the present invention.

FIG. 7A is a schematic diagram of displaying a 3D image by the live broadcasting system applying the present invention.

FIG. 7B is a schematic diagram of the interaction between the live broadcaster and the user terminal by applying the live broadcast system of the present invention.

FIG. 8 is a block diagram of a 3D live video control device according to the present invention.

FIG. 9 is a flow chart of the judging steps of the 2D and 3D live video conversion of the 3D live video control device of the present invention.

S301-S305: Steps

Claims (10)

A live broadcast method for real-time three-dimensional image display, comprising: Image an entity, and acquire a three-dimensional image of the entity; Perform image processing of the three-dimensional image through a three-dimensional live broadcast calculation module; According to at least one video selection condition of a user terminal, a video allocation combination is generated through an artificial intelligence learning module; Optimizing the 3D image according to the video deployment combination and a usage environment of the user terminal; and The optimized three-dimensional image is displayed in the user terminal in accordance with at least one of the video selection conditions of the user terminal; Wherein, the 3D image may be a full-image 3D image or a partial 3D image. The live broadcast method for real-time 3D image display according to claim 1, wherein in the step of capturing an image of an entity, a 2D image is obtained through at least one lens, and then the 2D image is simulated through a 3D image. The module is converted into the three-dimensional image. The live broadcast method for real-time 3D image display according to claim 1, wherein in the step of capturing an image of an entity, the 3D image is directly obtained through a plurality of lenses. The live broadcast method for real-time 3D image display according to claim 1, wherein in the step of performing image processing on the 3D image, image error tolerance and correction are performed on the 3D image, and the 3D image is detected and separated. Multiple objects in the image. The live broadcast method for real-time three-dimensional image display according to claim 4, wherein, if the object is a human, image soft filtering, background blur, filter effect or beautifying the skin is performed on the object. The live broadcast method for real-time three-dimensional image display according to claim 4, wherein, if the object is an object, background peeling, edge enhancement, contrast enhancement or image enlargement is performed on the object. The live broadcast method for real-time 3D image display according to claim 1, wherein at least one of the video selection conditions of the user terminal is the use inertia of the user terminal to a live broadcast system. The live broadcast method for real-time three-dimensional image display according to claim 1, wherein, in the step of optimizing the three-dimensional image, the three-dimensional image is optimized according to the number of viewers and network speed of the live broadcast system . The live broadcast method for real-time 3D image display according to claim 1, further comprising: capturing at least one of the user terminals to obtain the 3D image of the user terminal, and then executing a multi-person interactive stereoscopic display mode . The live broadcast method for real-time 3D image display according to claim 1, wherein, in the step of optimizing the 3D image, cutting and adjusting the 3D image is performed, and according to the video allocation and combination, through the The artificial intelligence learning module optimizes the three-dimensional image.

Images