TWI830661B - Online live interactive site navigation system - Google Patents

Abstract

An online live interactive field navigation system includes an image database and a server. The image database stores at least a panoramic environment image of a field object; the server generates live audio and video data based on the panoramic environment image and audio and video data from a host electronic device, and the live audio and video data has a navigation image and a navigation Voice; the server continues to transmit the live video and audio data to a user electronic device to display the navigation image on the display module of the user electronic device and play the navigation voice on the speaker of the user electronic device; the server Receive a control authorization command sent by the master electronic device, and grant a control authority to the user electronic device so that the user electronic device can control the panoramic environment image of the background image of the live video and audio data.

Description

Online live interactive site navigation system

The present invention relates to online field navigation; in particular, it refers to an online live interactive field navigation system.

In recent years, the epidemic has spread seriously. Many people are worried about increasing the risk of contracting the disease by going out and coming into contact with crowds. Moreover, with the development of Internet technology, people's behavior of visiting venues has gradually evolved into visiting online at home. Take house tours as an example. In order to allow consumers to view the house at any time, real estate agents will display videos or photos of the internal environment of the house on their web pages, including living rooms, kitchens, bedrooms and other indoor environments, allowing consumers to watch online at any time.

However, existing online house tours basically involve consumers using electronic devices alone to view relevant information on web pages. There is a lack of two-way interaction between the industry and consumers, so the industry cannot obtain real-time feedback from consumers for online house tours.

In other words, there are many shortcomings in the presentation of the actual situation of the field through online videos or photos. Due to the information of videos and two-dimensional photos, it is really difficult for users to experience the indoor or outdoor scenes of the field and create a specific sense of space, and the videos or photos are easily distorted by Different shooting angles will miss information about the scene.

In view of this, the purpose of the present invention is to provide an online live broadcast interactive field navigation system that allows the live broadcast host to use the panoramic surround image of the field objects to live broadcast the field objects, and allows the user and the live broadcast host to compare the introduced field objects. Panoramic image interaction of domain objects.

In order to achieve the above object, the present invention provides an online live interactive field navigation system for signal connection between a main control electronic device and at least one user electronic device, wherein the main control electronic device has a camera module, a A radio module and a display module. The main control electronic device obtains an audio-visual data of a live broadcaster through the camera module and the radio module. The audio-visual data includes a real-time image and a real-time voice. The at least one user electronic device The device has a display module and a speaker; the online live interactive field navigation system includes an image database and a server, wherein: the image database stores at least one panoramic environment image of at least one field object; the server The signal connects the image database, the main control electronic device and the at least one user electronic device. The server obtains the at least one panoramic environment image of the at least one field object from the image database and continues to receive the image of the main control electronic device. Audio and video data, and the server generates live audio and video data based on the at least one panoramic environment image and the continuously received audio and video data. The live audio and video data has a navigation image and a navigation voice, and the navigation image includes a foreground image. and a background image, the foreground image includes the real-time image, the background image includes the at least one panoramic environment image, and the navigation voice includes the real-time voice; the server continues to transmit the live video and audio data to the at least one user electronic device , display the navigation image on the display module of the at least one user electronic device and play the navigation voice on the speaker of the at least one user electronic device; wherein, the server is based on a first corresponding to the main control electronic device The control command controls the panoramic environment image in the background image of the live audio and video data generated; Wherein, the server receives a control authorization command sent by the master electronic device, and grants a control authority to the at least one user electronic device according to the control authorization command. When the server receives at least one control authorization command sent by the at least one user electronic device, During the second control command, the server controls the panoramic environment image of the background image of the live video and audio data according to the at least one second control command, and transmits the live video and audio data to the master electronic device and the at least one user electronic device. , to synchronously display the navigation image on the display module of the main control electronic device and the display module of the at least one user electronic device.

The effect of the present invention is that by transmitting the live video and audio data to the user's electronic device, the user can watch the navigation image and the navigation voice with real-time images and panoramic environment images during the live broadcast on the user's electronic device. Both the live broadcast host and the user can It can control the panoramic environment image to achieve the purpose of two-way interaction between the live broadcast host and the user. Panoramic environment images can also allow users to feel the sense of space created by the scene in the field.

1: Online live interactive site navigation system

10:Image database

12:Server

20: Main control electronic device

22:Photography module

24:Radio module

26:Display module

28: Speaker

30: User electronic device

32:Photography module

34:Radio module

36:Display module

38: Speaker

40,40a: User interface

42:Menu area

422,422a: Object menu area

424,424a: User menu area

44:Guide area

46:Manipulate images

46a: Moving images

46b: Enlarge image

46c: Shrink image

46d: mark image

48:Switch space image

50:Switch object image

52:List

522:House images

54,54a,54b: User list

542: User image

544:Switch image

546:Confirm image

56:Navigation image

562:Foreground image

564:Background image

60:User interface

62:Guide area

64:Request to manipulate image

66:Manipulate images

66a: Moving images

66b: Enlarge image

66c: Shrink image

66d: Mark images

70:Switch space image

I:Internet

Figure 1 is an architecture diagram of an online live interactive field navigation system according to a first preferred embodiment of the present invention.

FIG. 2 is a schematic diagram of the user interface displayed by the display module of the main control electronic device according to the first preferred embodiment of the present invention.

FIG. 3 is a schematic diagram of a user list displayed by the display module of the main control electronic device according to the first preferred embodiment of the present invention.

FIG. 4 is a schematic diagram of the user interface displayed by the display module of the main control electronic device according to the first preferred embodiment of the present invention.

FIG. 5 is a schematic diagram of a user interface displayed by a display module of a user electronic device according to the first preferred embodiment of the present invention.

FIG. 6 is a schematic diagram of a user list displayed by the display module of the user electronic device according to the first preferred embodiment of the present invention.

FIG. 7 is a schematic diagram of a user interface displayed by a display module of a user electronic device according to the first preferred embodiment of the present invention.

FIG. 8 is a schematic diagram of a user interface displayed by a display module of a main control electronic device according to a second preferred embodiment of the present invention.

In order to illustrate the present invention more clearly, the preferred embodiments are described in detail below along with the drawings. Please refer to FIG. 1 , which is an online live interactive site navigation system 1 according to a first preferred embodiment of the present invention. It is used for signal connection between a master electronic device 20 and at least one user electronic device 30 . In this embodiment, a house is used as an example to illustrate the area, but it is not limited to this. The area can also be a museum, a historical site, an educational institution, a public institution, or a financial institution, etc. that require navigation.

In this embodiment, the main control electronic device 20 and the at least one user electronic device 30 are mobile devices (mobile phones, tablets, laptops) or personal computers that can connect to the Internet 1. The main control electronic device 20 has a camera module 22, a radio module 24, a display module 26 and a speaker 28. The main control electronic device 20 is operated by the live broadcast host and can communicate with the host through the camera module 22. The radio module 24 obtains an audio-visual data of the live broadcast host. The audio and video data includes a real-time image and a real-time voice. The camera module 22 is used to capture the real-time image of the live broadcast host, and the radio module 24 is used to receive the live broadcast. Live voice of the Lord. Preferably, the main control electronic device 20 removes the background behind the live broadcast host in the real-time image.

The number of the at least one user electronic device 30 is a plurality. Each user electronic device 30 has a camera module 32, a radio module 34, a display module 36 and a speaker 38. The camera module 32 is used for taking pictures. A user's real-time image of the user, the radio module 34 is used to receive a user's real-time voice of the user to form the user's audio and video data. The main control electronic device 20 and each of the user electronic devices 30 respectively execute an application program to connect to the online live interactive field navigation system 1 through the Internet 1.

In this embodiment, the display module 26 of the main control electronic device 20 and the display module 36 of the user electronic device 30 are touch displays as an example, so that the live broadcast host and the user can control the main control electronic device 20 and the user. The electronic device 30 , but is not limited thereto, can also be used to control the electronic device 20 or the user electronic device 30 to connect to an input interface such as a mouse or keyboard for control by the live broadcast host or the user.

The online live interactive site navigation system 1 includes an image database 10 and a server 12 . The image database 10 stores at least one panoramic environment image of at least one field object. In this embodiment, the field object is a house object as an example. The image database 10 stores panoramic environment images of a plurality of house objects, and the number of panoramic environment images of each house object can be one or more. These panoramic environment images The environment images respectively correspond to multiple spaces of the house object (such as living room, kitchen, bedroom, bathroom, etc.). The merchant can upload panoramic environment images to the image database 10 for storage through the Internet 1. The panoramic environment images are panoramic images captured by panoramic cameras.

The server 12 is connected to the image database 10 via a signal, and the signal connection may be wired or wireless, or connected through a local area network or the Internet. The image database 10 can also be built into the server 12 . The server 12 connects the main control electronic device 20 and the user electronic devices 30 through Internet I signals.

Through the above structure, the online live interactive field navigation method of this embodiment can be carried out. The method includes the following steps:

In step S11, the live broadcast host controls the main control electronic device 20 to connect to the server 12. Please cooperate with FIG. 2. A user interface 40 is displayed on the display module 26 of the main control electronic device 20. The user interface 40 includes at least a menu area 42, a navigation area 44, a plurality of control images 46, and a plurality of control images 46. Switching space image 48 and a switching object image 50. In this embodiment, the at least one menu area 42 includes two menu areas, namely an object menu area 422 and a user menu area 424. A list 52 of items of a plurality of house objects is displayed in the object menu area 422, and each item of the house object in the list 52 includes a house image 522 of the house object. The live broadcast host can select an item of the house object to be introduced from the list 52, for example, click on the house image 522. After that, the server 12 obtains at least one panoramic environment image of the corresponding house object from the image database 10. For example, the default panoramic environment image is the panoramic environment image of the living room. These switching space images 48 correspond to different spaces, including living room, kitchen, bedroom, and bathroom.

In addition, the main control electronic device 20 continues to upload the audio and video data to the server 12 , and the server 12 continues to receive the video and audio data from the main control electronic device 20 . The user's electronic device 30 can also transmit the user's audio and video data to the server 12 .

In addition, each user also controls each user's electronic device 30 to connect to the server 12 . As shown in FIG. 2 , after the user electronic devices 30 are connected, a user list 54 is displayed in the user menu area 424 of the menu area 42 of the master electronic device 20 . Each user item in the user list 54 includes each user's User image 542. When the number of users is large, the number of other users will be displayed numerically. For example, "+36" will be displayed to indicate that there are 36 other users. When the live broadcast host clicks any position on the user menu area 424, or each user image 542, or the number of users, a detailed user list 54a as shown in FIG. 3 will be displayed on the main control electronic device 20. Each user item in the user list 54 and 54a can subsequently be used by the live broadcast host to select the user to whom control permissions are to be granted, which will be discussed later. After any of the user items is selected, the user interface of the master electronic device 20 The corresponding user image 542 in the user menu area 424 of 40 is displayed in an eye-catching manner (refer to FIG. 4 ).

In step S12, the server 12 generates live audio and video data based on the panoramic environment image and the continuously received audio and video data. The live audio and video data has a navigation image and a navigation voice. As shown in FIG. 2 , the navigation image 56 includes a foreground image 562 and a background image 564 . The foreground image 562 includes the real-time image, and the background image 564 includes the panoramic environment image. The navigation voice includes the real-time voice.

Step S13, the server 12 continues to transmit the live video and audio data to each user electronic device 30, so as to display the navigation image 56 (refer to FIG. 5) on the display module 36 of each user electronic device 30 and on each user electronic device 30. The speaker 38 of the user electronic device 30 plays the navigation voice.

In this embodiment, a user interface 60 displayed by the display module 36 of the user electronic device 30 has a navigation area 62 , and the navigation area 62 is used to display the navigation image 56 . In addition, the live video and audio data are simultaneously transmitted to the main control electronic device 20 to display the navigation image 56 (refer to FIG. 2 ) in the navigation area 44 of the display module 26 of the main control electronic device 20 and can be selected. The guide voice is played on the speaker 28 of the main control electronic device 20 . The user interface 60 also has an object menu area 422a and a user menu area 424a, wherein the object menu area 422a is displayed in a smaller area. The list 52 of house objects in the object menu area 422a of the user interface 60 also has house images 522 of house objects that can be selected by the user, and the user menu area 424a also has user images 542 that can be selected by the user. In addition, in FIGS. 4 and 5 , the house image 522 of the house object being navigated is displayed in an eye-catching manner.

Step S14, as shown in FIG. 2, the live broadcast host can select any control image 46 on the user interface 40 to operate the background image 564 of the navigation image 56 in the navigation area 44. In this embodiment, the manipulation images 46 include a moving image 46a, an enlarged image 46b, a reduced image 46b, and a moving image 46b. Small image 46c and mark image 46d. After selecting the moving image 46a, the panoramic environment image in the background image 564 can be moved to change the presented perspective. Selecting the enlarged image 46b or the reduced image 46c is to enlarge or reduce the panoramic environment image in the background image 564. By selecting the mark image 46d, the at least one area corresponding to the panoramic environment image in the background image 564 can be specified, and at least one mark can be added, for example, a circle mark, an indicator mark, a hand-drawn mark, etc. can be added.

The live broadcast host can click on the control images 46 in conjunction with the explanation time to introduce the house objects. When the moving image 46a, the enlarged image 46b, or the reduced image 46c is clicked, the main control electronic device 20 sends a corresponding first control command to the server 12 according to the clicked control image 46. When the mark image 46d is clicked, the main control electronic device 20 sends a corresponding mark command to the server 12.

When the server 12 receives the first control command, it controls the panoramic environment image in the background image 564 of the live audio and video data generated according to the first control command corresponding to the main control electronic device 20 . Thereby, the background image 564 controlled by the live broadcast host can be displayed correspondingly on the display module 36 of each user electronic device 30 . When the server 12 receives the marking instruction, the server 12 adds a mark to the specified area corresponding to the panoramic environment image in the background image 564 of the live audio and video data according to the marking instruction. Thereby, the background image 564 marked by the live broadcast host can be displayed correspondingly on the display module 36 of each user electronic device 30 .

When the live broadcast host wants to switch the space for commentary, he can select the switching space image 48 corresponding to the space to be explained on the user interface 40, and the main control electronic device 20 sends a corresponding switching space request to the server 12. The switching space The request specifies one of the spaces, such as "bathroom". The server 12 obtains the panoramic environment image of the corresponding space from the image database, and the server 12 requests the panoramic environment image of the specified space and the continuous environment image based on the switching space. The received audio and video data generates the live video and audio data. In this way, the main control electronics can be synchronized The changed panoramic environment image in the background image 564 of the display module of the device 20 and the user electronic device 30 .

Through the above steps, the live broadcast host can explain the house objects live online and control the panoramic environment image, so that users can better feel the sense of space and presence generated by the scene.

Afterwards, in step S15, the live broadcast host chooses to hand over the control authority of the background image 564 to the selected user.

For example, as shown in FIG. 2 , the live broadcast host can select the user to whom control permission is to be granted from the user list 54 in the user menu area 424 of the menu area 42 , for example, click on the user's user image 542 to grant the user control permission. Alternatively, the live broadcast host can click on the user image 542 to be granted control authority on the user list 54a shown in FIG. 3, or click on the corresponding switch image 544. After clicking the confirmation image 546, the control authority can be granted to the user. Take "User 1" as an example.

Of course, the live broadcast host can also grant control permissions according to the requests of each user. Specifically, the user interface 60 of each user electronic device 30 has a request control image 64. After any user clicks the request control image 64, the corresponding user electronic device 30 sends a request to the server 12. A control authorization request. After receiving a control authorization request corresponding to any of the user electronic devices 30 , the server 12 transmits the control authorization request of the corresponding user electronic device 30 to the main control electronic device 20 . The main control electronic device 20 displays a corresponding prompt on the display module 26 according to the control authorization request, and prompts the user item in the user list 54 in an eye-catching manner, for example, the user image 542 of "User 1" flashes. display. The live broadcast host can choose to grant control permissions to "User 1".

Then, the main control electronic device 20 sends a control authorization command corresponding to the user item (such as "User 1") to the server 12. The control authorization command includes a designated user. Instruction, the specified user instruction is to specify one of the user electronic devices 30, for example, the user electronic device 30 corresponding to "user 1".

In step S16, after receiving the control authorization command sent by the master electronic device 20, the server 12 grants a control authority to the corresponding user electronic device 30 according to the control authorization command. More specifically, the server 12 grants the corresponding user electronic device 30 the control authority according to the designated user instruction of the control authorization instruction. The server 12 waits for the user electronic device 30 granted the control authority to send at least a second control command.

In step S17, the user with control authority controls the user electronic device 30 to send the at least one second control command to control the background image 564 of the live audio and video data.

In this embodiment, the user image 542 of the user who is granted the control authority is displayed in an eye-catching manner in the user interface 60 on the display module 36 of the user electronic device 30 . There are a plurality of control images 66 in the user interface 60 of the user electronic device 30. The control images 66 include a moving image 66a, an enlarged image 66b, a reduced image 66c and a mark image 66d. After selecting the moving image 66a, the panoramic environment image in the background image 564 can be moved to change the presented perspective. Select the enlarged image 66b or the reduced image 66c to enlarge or reduce the panoramic environment image in the background image 564. By selecting the mark image 66d, the at least one area corresponding to the panoramic environment image in the background image 564 can be specified, and at least one mark can be added, for example, a circle mark, an indicator mark, a hand-drawn mark, etc. can be added.

Users with control authority click on the control images 66 as needed. When the moving image 66a, the enlarged image 66b, or the reduced image 66c is clicked, the user electronic device 30 sends a corresponding second control command to the server 12 according to the clicked control image 66.

When the server 12 receives at least a second control command sent by the user electronic device 30 with control authority, the server 12 controls the live video according to the second control command. The panoramic environment image of the background image 564 of the audio data, and the live audio and video data is transmitted to the main control electronic device 20 and the user electronic devices 30, so that the display module 26 of the main control electronic device 20 and the users The display module 36 of the electronic device 30 displays the navigation image 56 simultaneously.

Through this, the user can control the full-shape environment image in the background image 564 by himself, and the live broadcast host can also further explain the area that the user is interested in, so as to achieve the purpose of interaction between the live broadcast host and the user. In addition, the user's real-time voice received by the audio receiver module 34 of the user electronic device 30 can also be transmitted to the main control electronic device 20 through the server 12 and played on the speaker 28 of the main control electronic device 20 .

In addition, in this embodiment, when a user with control authority clicks the mark image 66d, the user electronic device 30 with control authority sends the corresponding mark command to the server 12. The server 12 adds a mark to the designated area of the panoramic environment image in the background image 564 of the live video data according to the mark command. The user can add a circle mark, an indicator mark, or a hand-drawn mark, etc., so that the live broadcast host can better target A description of where the user marked it.

Furthermore, the server 12 captures an area image corresponding to one or more designated areas of the panoramic environment image in the background image 564 of the live audio and video data, and creates a list including the area image. The main control electronic device 20 connects to the server 12 to obtain the list, and the live broadcast host can reply to the user for one or more regional images listed in the list.

In this embodiment, the user can control the user electronic device 30 to input a dialogue message corresponding to one or more regional images. The dialogue message can be a text or voice question asked by the user. After receiving the dialogue message corresponding to the area image from the user electronic device 30, the server 12 appends the dialogue message to the list. During the live broadcast or after the live broadcast ends, the main control electronic device 20 is connected to the server 12, and the live broadcast host can control the main control electronic device 20 to input corresponding to the dialogue message for the regional images in the list and the dialogue message. A reply message, which can be a text or voice reply from the live broadcast host. The server 12 obtains information from the main control electronic device 20 receives the reply message corresponding to the conversation message, and appends the received reply message to the list.

The user can control the user's electronic device 30 to connect to the server 12 during the live broadcast or after the live broadcast, obtain the list, and can view the regional images in the list on the user's electronic device 30 and watch or listen to the live broadcast host's speech. Reply to message.

In this embodiment, the control authority granted by the server 12 is temporary. The control authority can also be canceled by the main control electronic device 20. For example, when the live broadcast host clicks on the user item for which the control authority is to be canceled on the user list 54, the main control electronic device 20 sends a cancellation authorization command to the server 12. After receiving the deauthorization command from the master electronic device 20, the server 12 takes back the control authority of the user electronic device 30 corresponding to the specified user command. In addition to the live broadcast host canceling the control authority, the server 12 can set a valid time. During the valid time, the user electronic device 30 is allowed to control the background image 564. After the valid time, the server 12 cancels the control permission. The user electronic device 30 is prohibited from manipulating the background image 564 .

In order to allow the user to choose to let the live broadcast host introduce a specific space, in this embodiment, the user can control the user electronic device 30 to input a space switching request, and the space switching request specifies one of the spaces. For example, the user interface 60 of the user electronic device 30 has a plurality of switching space images 70, and the switching space images 70 correspond to different spaces, including living rooms, kitchens, bedrooms, and bathrooms.

When the user selects the switching space image 70 corresponding to a specific space on the user interface 60 , the user electronic device 30 sends a corresponding switching space request to the server 12 . The server 12 transmits the switching space request to the main control electronic device 20 , and the main control electronic device 20 displays a corresponding prompt on the display module 26 , for example, in an eye-catching manner on the corresponding switching space image 48 Prompt, for example, display a prompt image on the corresponding switching space image 48 or display the corresponding switching space image 48 in different colors. The live broadcast host clicks the corresponding switch space After the image 48 is displayed, the main control electronic device 20 sends a space switching permission command to the server 12 , and the space switching permission command corresponds to the specified space. After the server 12 receives the command to allow switching spaces from the main control electronic device, the server 12 generates the live audio and video data based on the panoramic environment image of the space specified by the switching space request and the audio and video data that is continuously received, and also That is, the server 12 obtains the panoramic environment image of the corresponding space from the image database 10 to generate live audio and video data. Thereby, the panoramic environment image in the background image 564 of the display modules 26 and 36 of the main control electronic device 20 and the user electronic device 30 can be synchronized. The live broadcast host can explain the space specified by the user.

When the live broadcast host wants to introduce other house objects, he can click the switching object image 50 on the user interface 40 . As shown in Figure 4, a list 52 of house objects is displayed in the menu area 42. After the live broadcast host clicks other house objects in the list 52, the main control electronic device 20 sends a switching object request. The switching object request is a specified One of the house objects. The server 12 obtains the panoramic environment image of the corresponding house object from the image database 10, and the server 12 generates the live video based on the panoramic environment image of the house object designated by the switching object request and the audio and video data continuously received. material. In addition, the live broadcast host can also directly click the house image 522 in the object selection menu area 422 of the user interface 40, so that the main control electronic device 20 sends the switching object request to switch the house object to be navigated.

The user can also request the live broadcast host to introduce other house objects. As shown in Figure 5, the user selects the house image 522 of other house objects in the list 52 of house objects in the object menu area 422a on the user interface 60. The user electronically The device 30 transmits a switching object request, and the switching object request specifies one of the selected house objects. The server 12 sends the switching object request to the main control electronic device 20 . The main control electronic device 20 displays a corresponding prompt on the display module 26, for example, a prompt on the corresponding house object item in the list 52 in an eye-catching manner, for example, a prompt image is displayed on the corresponding house object item. or to The corresponding house object items are displayed in different colors, or the house image 522 is displayed in a flashing manner. After the live broadcast host clicks on the corresponding item of the house object, the main control electronic device 20 sends a command to allow switching of objects to the server 12. The command to allow switching of objects corresponds to the specified one of the house objects. The server 12 obtains the panoramic environment image of the corresponding house object from the image database 10, and the server 12 generates the live video based on the panoramic environment image of the house object designated by the switching object request and the audio and video data continuously received. material. Thereby, the panoramic environment images of the house objects in the background image 564 of the display modules 26 and 36 of the main control electronic device 20 and the user electronic device 30 can be synchronized. The live broadcast host can explain the house objects specified by the user.

In addition, the user can also click the house image 522 of the house object to be switched from the object menu area 422a in the user interface 60, and the user electronic device 30 sends a corresponding switching object request to the server 12 to obtain permission from the live broadcast host. Then, the navigation image 56 of the specified house object is switched to display.

The user clicks any position on the user menu area 424a in the user interface 60, or each user image 542, or the number of users, then the detailed user information as shown in FIG. 6 is displayed on the user electronic device 30. Screenshot of Listing 54b. The user can click on other user images 542 on the detailed user list 54b to further view the public information of the selected user.

In addition, as shown in FIG. 7 , a user with control authority can also control his or her user electronic device 30 to cause the server to replace the real-time image of the live broadcast host in the foreground image 562 of the navigation image 56 with the user's real-time image. In this way, the user uses the camera module 32 of the user electronic device 30 to capture a real-time image of the user's whole body and sends it to the server 12. The server 12 generates a navigation image 56 and sends it to the user electronic device 30 to display the navigation image with the user's real-time image. View image 56 allows users to feel the ratio between the user and the panoramic environment image, and have a more immersive experience. Preferably, the user electronic device 30 removes the background behind the user in the user's real-time image.

Figure 8 shows a user interface 40a displayed by the display module 26 of the main control electronic device 20 according to a second preferred embodiment of the present invention. The difference between this embodiment and the first embodiment is that the live broadcast host can use the main The photography module 22 of the control electronic device 20 captures real-time images of the whole body, allowing the user to feel the ratio of the live broadcast host to the panoramic environment image. Preferably, the main control electronic device 20 removes the background behind the live broadcast host in the real-time image.

In order to facilitate the live broadcast host to control the main control electronic device 20, the server 12 recognizes the body posture of the live broadcast host based on the real-time image of the audio and video data received from the main control electronic device 20, and generates the third body posture based on the recognized body posture. A control command to control the panoramic environment image in the generated background image 564 of the live audio and video data. For example, the server 12 has established a plurality of predetermined control body postures, and each predetermined control body posture corresponds to a first control command. For example, the predetermined control body postures may respectively correspond to movement, enlargement, and reduction. When the server 12 determines that the body posture of the live broadcast host in the real-time image matches a corresponding predetermined control body posture, a corresponding first control command is generated to control the panoramic environment image in the background image 564 .

Alternatively, the server 12 recognizes the voice of the live broadcast host based on the real-time voice of the audio-visual data received from the main control electronic device 20, and generates the first control command based on the recognized voice. For example, the server 12 creates a plurality of predetermined control voice commands, and each of the predetermined control voice commands corresponds to a first control command. For example, the predetermined voice commands can respectively correspond to movement, enlargement, and reduction. When the server 12 determines that the live broadcast host's voice in the real-time voice matches a corresponding predetermined control voice command, a corresponding first control command is generated to control the panoramic environment image in the background image 564 .

According to the above, the online live interactive field navigation system of the present invention can transmit the live audio and video data to the user's electronic device, and the user can watch the live broadcast on the user's electronic device. With real-time images and panoramic environment images, navigation images and navigation voices, both the live broadcast host and the user can control the panoramic environment image, achieving the purpose of two-way interaction between the live broadcast host and the user.

The above are only the best possible embodiments of the present invention. Any equivalent changes made by applying the description and patent scope of the present invention should be included in the patent scope of the present invention.

1: Online live interactive site navigation system

10:Image database

12:Server

20: Main control electronic device

22:Photography module

24:Radio module

26:Display module

28: Speaker

30: User electronic device

32:Photography module

34:Radio module

36:Display module

38: Speaker

I:Internet

Claims (7)

An online live interactive field navigation system is used to signally connect a main control electronic device and at least one user electronic device, wherein the main control electronic device has a camera module, a radio module and a display module, the The main control electronic device obtains an audio-visual data of a broadcaster through the camera module and the radio module. The audio-visual data includes a real-time image and a real-time voice. The at least one user electronic device has a display module and a speaker; The online live interactive field navigation system includes: an image database that stores at least one panoramic environment image of at least one field object; a server that connects signals to the image database, the master electronic device, and the at least one user electronic device. device, the server obtains the at least one panoramic environment image of the at least one field object from the image database and continuously receives the audio and video data of the main control electronic device, and the server relies on the at least one panoramic environment image and the continuously received The audio-visual data generates live audio-visual data. The live audio-visual data has a navigation image and a navigation voice. The navigation image includes a foreground image and a background image. The foreground image includes the real-time image. The background image includes the At least one panoramic environment image, the navigation voice includes the real-time voice; the server continues to transmit the live video and audio data to the at least one user electronic device to display the navigation image on the display module of the at least one user electronic device and playing the navigation voice on the speaker of the at least one user electronic device; wherein the server controls the panoramic environment image in the background image of the live audio and video data generated according to a first control command corresponding to the main control electronic device. ; Wherein, the server receives a control authorization command sent by the master electronic device, and grants the at least one user electronic device a control authority according to the control authorization command. When the server receives at least one control authorization command sent by the at least one user electronic device, When a second control command is issued, the server controls the panoramic environment of the background image of the live video and audio data according to the at least one second control command. image, and transmit the live video and audio data to the main control electronic device and the at least one user electronic device, so that the display module of the main control electronic device and the display module of the at least one user electronic device can simultaneously display the guide. Browse the image; wherein the server receives at least one marking instruction from the at least one user electronic device, and the at least one marking instruction specifies at least one area in the background image corresponding to the panoramic environment image; the server relies on the at least one marking instruction At least one mark is added to the background image of the live audio and video data corresponding to the at least one area of the panoramic environment image; wherein, the server captures an area of the background image of the live audio and video data corresponding to the at least one area of the panoramic environment image. image, and create a list including the image of the area; wherein the server receives a dialogue message corresponding to the image of the area from the at least one user electronic device, and appends the dialogue message to the list; the server receives the image from the area from the at least one user electronic device At least one master electronic device receives a reply message corresponding to the conversation message and appends the reply message to the list. The online live interactive field navigation system as described in claim 1, wherein the number of the at least one user electronic device is plural; wherein after the server receives a control authorization request from any one of the user electronic devices, the corresponding The control authorization request of the user electronic device is transmitted to the master electronic device, and the control authorization instruction received by the server from the master electronic device includes a designated user instruction, and the designated user instruction specifies one of the user electronic devices; The server grants the corresponding user electronic device the control permission according to the designated user instruction. The online live interactive site navigation system described in claim 2, wherein the server recovers the control authority of the user's electronic device corresponding to the designated user instruction after receiving a deauthorization instruction from the master electronic device. The online live interactive field navigation system as described in claim 1, wherein the at least one panoramic environment image stored in the image database is a plurality, and the panoramic environment images correspond to a plurality of spaces of the at least one field object; The server receives a switching space request from the at least one user electronic device, and the switching space request specifies one of the spaces; the server transmits the switching space request to the main control electronic device, and the server receives the switching space request. After a command from the master electronic device to allow switching of spaces, the server generates the live audio and video data based on the panoramic environment image of the space specified by the switching space request and the audio and video data that it continues to receive. The online live interactive field navigation system as described in request item 1, wherein the at least one field object is a plurality, and the image database stores the at least one panoramic environment image of each field object; wherein the server receives A switching object request of the at least one user electronic device, the switching object request specifies one of the field objects; the server transmits the switching object request to the main control electronic device, and the server receives the main control electronic device After the device sends a command to allow switching of objects, the server will generate the live video and audio data based on the panoramic environment image of the field object specified by the switching object request and the audio and video data that it continues to receive. The online live broadcast interactive field navigation system as described in request item 1, wherein the server recognizes the body posture of the live broadcast host based on the real-time image of the audio and video data received from the main control electronic device, and based on the recognized body The gesture generates the first control command. The online live broadcast interactive site navigation system as described in request item 1, wherein the server recognizes the voice of the live broadcast host based on the real-time voice of the audio and video data received from the main control electronic device, and generates a generated voice based on the recognized voice. the first control command.

Images