TWI640900B - System and method of augmented reality for mobile vehicle - Google Patents

Abstract

The present invention provides a system for augmented reality of a mobile vehicle, in which a mobile device in a mobile vehicle is wirelessly connected to a wireless network device in a mobile vehicle, and a mobile vehicle is obtained through the wireless network device. The external real-life image, wherein the moving vehicle external real-life image is taken by a camera mounted on the mobile vehicle. The system also adds augmented reality content to the external real-life image. The augmented reality content is calculated by the local server in the vehicle through the parameters, and is used to filter the appropriate attraction recommendation list and the featured product purchase list.

Description

System and method for augmented reality of mobile vehicles

The present invention relates to an augmented reality application, and more particularly to a system and method for augmented reality for mobile vehicles.

With the development of science and technology, people gradually began to install car navigation and other devices in mobile vehicles, in order to quickly grasp the local driving routes or recommended tourist attractions when traveling abroad. In general, car navigation devices can be built with many preset attractions. When the user selects the target of sightseeing, the user can read the recommended sightseeing information built into the car navigation device through the user interface.

However, the above method is accompanied by a lot of inconveniences, for example, the user cannot manually click on the interface of the car navigation device to obtain the recommended sightseeing information. Conventional car navigation devices are not able to actively provide recommended sightseeing information of the user's current location on the move. Therefore, how to obtain local sightseeing information on the move in an instant and efficiently is one of the goals of the people in the field.

The present invention provides a system for augmented reality of a mobile vehicle, comprising: a camera, a local server, and a user device. The camera is mounted on a mobile vehicle and takes an external image of the moving vehicle. A wireless network device that provides a wireless network. The local server includes: a storage unit and a processor. The storage unit records a plurality of modules. The processor is coupled to the storage unit and accesses and executes the modules stored by the storage unit. The modules include: a recommendation list module and a film and augmented reality integration module. The recommendation list module generates augmented reality content based on the context parameters, and the film and augmented reality integration module generates the augmented reality image by using the augmented reality content and the image converted from the external image. The user device accesses the augmented reality image through the wireless network.

The present invention provides a method for augmenting a reality of a mobile vehicle, comprising: capturing an external image of a mobile vehicle. Augmented reality content is generated based on the context parameters. An augmented reality image is generated using the augmented reality content and the image converted from the external image. And accessing the augmented reality image.

Based on the above, the present invention can transmit an external live image of the mobile vehicle to a local server within the mobile vehicle. The user device in the mobile carrier can be wirelessly connected to the wireless network device and automatically open a web browser, wherein the web browser displays the value added service application in the local server. In addition, the local server can filter the augmented reality content from the local database based on the context parameters, and obtain information such as location information, weather information, and event information to provide more augmented reality images.

The above described features and advantages of the invention will be apparent from the following description.

1 is a schematic diagram of a system 100 for augmented reality of a mobile vehicle in accordance with an embodiment of the present invention. The system 100 is mounted to the mobile carrier 10, which includes, but is not limited to, a camera 110, a wireless network device 120, a user device 130, a local server 140, and a cloud server 170.

The mobile carrier 10 can be a traffic device such as a bus, train, airplane, car, high-speed rail, MRT, ship, passenger, aircraft, and the like. In some large mobile vehicles, passengers in the passenger compartment cannot clearly view the external scenery of each field of view. Passengers in these large mobile vehicles can obtain information on the external environment through the present invention.

The camera 110 is provided with a position at which an external image of the mobile vehicle 10 can be photographed. Generally, the camera 110 is installed at a position where the front image of the vehicle can be photographed, thereby applying the captured image to the route navigation purpose. However, the present invention does not limit the mounting position of the camera 110 to the mobile carrier 10. Camera 110 can be a high speed camera. A high-speed camera can shoot at least 1,000 frames per second compared to a typical camera that shoots 24 frames per second.

The wireless network device 120 is mounted on the mobile carrier 10 for wirelessly connecting the user equipment 130 in the mobile carrier to the wireless network device 120 to obtain data of the local server 140 and value-added services. Application 171 provides services. 2 is a schematic diagram of a wireless network device 120 in accordance with an embodiment of the present invention. Wireless network device 120 may include, but is not limited to, Wi-Fi access point 121, switch 122, Wi-Fi gateway 123, and 4G router 124. The wireless network device 120 can wirelessly connect devices within the mobile vehicle 10, such as the camera 110, the user device 130, and the local server 140, to each other through the Wi-Fi access point 121. The wireless network device 120 can wirelessly connect devices within the mobile carrier 10 to the cloud server 170 external to the mobile vehicle 10 via a 4G router. The present invention defines a wireless communication protocol used by each device in the wireless network device 120. For example, the 4G router 124 may also be a 5G router, or the Wi-Fi access point 121 may be any wireless local area network conforming to the IEEE 802.11 standard. In one embodiment, the wireless network device 120, in response to being connected by the user device 130, causes the user device 130 to automatically open the browser and provide the service from the value-added service application 171 using the browser. The related application of the value-added service application 171 will be described later.

Go back to Figure 1. User device 130 can be wirelessly coupled to wireless network device 120 and coupled to other devices within mobile carrier 10 (e.g., local server 140) by wireless network device 120. The user equipment 130 can be a mobile station, an advanced mobile station (AMS), a server, a client, a desktop computer, a network computer, a workstation, a notebook computer, a tablet, and a personal digital device. A device having a computing power and a wireless communication function, such as a personal digital assistant (PDA), a telephone device, a camera, a television, a palm-type game machine, a music device, and a wireless sensor. In this embodiment, the user device 130 can access the augmented reality image from the local server 140 via the wireless network device 120.

Local server 140 includes, but is not limited to, processor 150 and storage unit 160. The local server 140 can be a mobile station, an advanced mobile station (AMS), a server, a client, a desktop computer, a network computer, a workstation, a notebook computer, a tablet, and a personal digital device. A device having a computing power and a wireless communication function, such as a personal digital assistant (PDA), a telephone device, a camera, a television, a palm-type game machine, a music device, and a wireless sensor.

The storage unit 160 records a plurality of modules, and the processor 150 is coupled to the storage unit 160 and accesses and executes the modules stored in the storage unit 160. The modules include: a movie conversion module 161, a recommendation list module 162, a film and augmented reality integration module 163, a value-added service application 171, a local database 172, a positioning module 181, and a weather module 182. And an activity module 183.

The movie transition module 161 is configured to convert the format of the external image obtained by the camera 110 into a format playable by the user device 130.

The recommendation list module 162 is configured to calculate the augmented reality content to be superimposed on the image according to the context parameter, wherein the context parameter is provided by the positioning module 181, the weather module 182, or the activity module 183.

The positioning module 181 is configured to obtain location information of the location where the mobile vehicle 10 is located. For example, the positioning module 181 can receive the GPS signal through the wireless network device 120 to obtain the latitude and longitude of the mobile vehicle 10. The weather module 182 is configured to obtain weather-related weather information such as temperature information, rainfall probability information, or wind speed information of the location where the mobile vehicle 10 is located. For example, the weather module 182 can be connected to the external network through the wireless network device 120, and the corresponding weather information can be obtained from the external network according to the location of the mobile vehicle 10. Optionally, the weather module 182 can detect current weather information through a detector installed on the mobile vehicle 10. The activity module 183 is used to obtain local activity information of the current location. For example, the activity module 183 can be connected to the external network through the wireless network device 120, and the corresponding local activity information can be obtained from the external network according to the location of the mobile vehicle 10.

The augmented reality integration module 163 uses the augmented reality content calculated by the recommendation list module 162 and the image converted by the movie transition module 161 to generate an augmented reality image.

The local database 172 is used to store attraction information or product information of a plurality of locations. The recommendation list module 162 can access the local database 172 and generate recommended current attraction information or current product information based on the context parameters and the local information database 172 attraction information or product information, wherein the current attraction information or the current product information can be, for example, It is an attraction, business or event not far from the mobile vehicle 10.

The value-added service application 171 includes at least one service-opening control for informing the local server 140 to transmit the augmented reality image generated from the movie and the augmented reality integration module 163 to the user device 130. Specifically, when the user device 130 is connected to the wireless network device 120, the wireless network device 120 can connect the user device 130 to the local server 140, so that the user device 130 can be accessed by an application such as a browser. The value-added service application 171 is turned on, and the service brought by the value-added service application 171 is used. Optionally, the value-added service application 171 can also be actively opened by the user. For example, the user device 130 can install an application associated with the value-added service application 171. When the user device 130 enables the application, the user device 130 searches for and activates the value-added service application 171 via the wireless network device 120.

In addition, the value-added service application 171 may also be a cloud server 170 disposed outside the mobile vehicle 10 instead of being configured in the local server 140. The local server 140 can be connected to the cloud server 170 as needed to access the services provided by the value-added service application 171. The cloud server 170 can store augmented reality content materials provided by multiple vendors and is updated periodically. The local server 140 can also update the attraction information or the product information in the local database 172 via the cloud server 170. It should be noted that the system 100 of FIG. 1 is disposed in the mobile vehicle 10 in order to make the illustration compact and easy to recognize. However, in most cases, the cloud server 170 is not installed in the mobile vehicle 10. As long as the wireless device 120 can wirelessly connect to the cloud server 170, the present invention does not limit the installation location of the cloud server 170.

3A is a schematic diagram of an application 310 associated with a value-added service application 171 presented by a display interface of a user device 130, wherein the various functions of FIG. 3A can be implemented by the previously disclosed system 100, in accordance with an embodiment of the present invention. The application 310 may include, but is not limited to, a value-added service such as a front image 131, a map 133, and a weather 135. In this embodiment, the user can select the front image 131 field through the display interface of the user device 130 to enable the value-added service application 171. After the value-added service application 171 is enabled, the user device 130 displays the augmented reality image output by the movie and the augmented reality integration module 163.

FIG. 3B is a schematic diagram of an augmented reality image 320, which may be implemented by the previously disclosed system 100, in accordance with an embodiment of the present invention. The augmented reality image 320 includes images captured by the camera 110 and related information of the context parameters. For example, the augmented reality image 320 displays the external image captured by the camera 110 on the image interface 321 .

The positioning module 181 obtains the location information of the current location (such as Hualien to Taitung direction), and displays the location information in the field 321 and the field 322. The positioning module 181 may also search for the section information according to the location information, display the attraction information of the section in the field 323 and the field 324, and display the distance information related to the attraction information of the section. Field 325 and field 326. The number of fields for displaying location information, attraction information, and distance information may be determined by information such as the number of recommended attractions at the location of the mobile vehicle 10, and the present invention is not limited thereto.

The weather module 182 obtains current weather information and can adjust the recommended weights of outdoor attractions and indoor attractions based on current weather information. For example, when the rainfall probability is 10%, the weather module 182 may increase the recommended weight value of the outdoor attraction and reduce the indoor attraction recommendation weight value.

The activity module 183 obtains activity information of this section. For example, if Taroko National Park has a forest patrol this month, the recommended weight of this attraction will increase.

After the positioning module 181, the weather module 182, and the activity module 183 obtain their respective calculated attraction weight values, the modules will return a list of attractions with a higher recommended weight (recommended score) to the recommended list. The module 162, the recommendation list module 162 can add the attraction information of the attractions to the augmented reality image 320 generated by the film and the augmented reality integration module 163.

The augmented reality image 320 can also include product information. For example, when the positioning module 181 determines that the location of the mobile vehicle 10 is Hualien, the recommendation list module 162 can add the relevant product information of Hualien to the film and the augmented reality integration module 163 based on the location information. Augmented reality image 320.

In summary, the present invention obtains multi-faceted external real-time images through a camera mounted on a mobile vehicle. The camera can be wirelessly connected to a wireless network device that is mounted in the mobile vehicle to transmit the live image of the mobile vehicle to a local server within the mobile vehicle. The user device in the mobile carrier can be wirelessly connected to the wireless network device and automatically open a web browser, wherein the web browser displays the value added service application in the local server. Another feature of the present invention is that the local server in the mobile vehicle can filter the augmented reality content from the local database based on the context parameters, wherein the context parameters include location information, weather information, activity information, and the like. The local server transmits the live image outside the mobile vehicle to the user equipment in the mobile vehicle, and stacks the augmented reality content onto the live image to provide an interactive application.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.

10‧‧‧Mobile Vehicles

100‧‧‧Augmented Reality System

110‧‧‧ camera 110

120‧‧‧Wireless network device

121‧‧‧Wi-Fi access point

122‧‧‧Switch

123‧‧ Wi-Fi gateway

124‧‧‧4G router

130‧‧‧User equipment

131‧‧ ‧ front image

132‧‧‧Map

133‧‧ ‧weather

140‧‧‧Local Server

150‧‧‧ processor

160‧‧‧storage unit

161‧‧‧Video Shift Module

162‧‧‧Recommended list module

163‧‧‧Video and Augmented Reality Integration Module

170‧‧‧Cloud Server

171‧‧‧ Value-added service application

172‧‧‧Local database

181‧‧‧ Positioning Module

182‧‧‧Weather Module

183‧‧‧ activity module

320‧‧‧Augmented Reality Image

321, 322, 323, 324, 325, 326‧‧‧ display fields

1 is a schematic diagram of a system for augmented reality of a mobile vehicle in accordance with an embodiment of the present invention. 2 is a schematic diagram of a wireless network device in accordance with an embodiment of the present invention. FIG. 3A is a schematic diagram of an application related to a value-added service application presented by a display interface of a user device according to an embodiment of the present invention. 3B is a schematic diagram of an augmented reality image in accordance with an embodiment of the present invention.

Claims (10)

A system for augmented reality of a mobile vehicle, comprising: a camera mounted on a mobile vehicle and photographing an external image of the mobile vehicle; a wireless network device providing a wireless network; and a local server, including a storage unit that records a plurality of modules; and a processor coupled to the storage unit and accessing and executing the modules stored by the storage unit, the modules including: a recommendation list module, based on a context parameter Generating augmented reality content; and integrating a film and augmented reality module to generate an augmented reality image using the augmented reality content and images converted from the external image; and user equipment through the wireless network The road accesses the augmented reality image. The system of claim 1, wherein the modules further comprise: a positioning module for obtaining location information of the location of the mobile vehicle; a weather module for obtaining weather information of the location; and an activity module Obtaining local activity information of the location, wherein the context parameter includes at least one of the location information, the weather information, and the local activity information. The system of claim 1, wherein the modules further comprise: a local database for storing attraction information or product information of a plurality of locations; and a film transfer module for converting the external image into the user The image that can be played by the device; and the value-added service application, to notify the local server to transmit the augmented reality content to the user device, wherein the film and the augmented reality integration module are based on the context parameter The attraction information or the product information generates current attraction information or current product information, and adds the current attraction information or the current product information to the augmented reality content. The system of claim 3, wherein the wireless network device causes the user device to turn on the value-added service application after the user device is connected to the wireless network. The system of claim 3, wherein the local server is connected to the cloud server through the network, and periodically updates the attraction information or the product information through the cloud server. A method for augmenting a reality of a vehicle, comprising: capturing an external image of the mobile vehicle; generating an augmented reality content based on the context parameter; and utilizing the augmented reality content and the image converted by the external image Generating an augmented reality image; and accessing the augmented reality image. The method of claim 6, further comprising: obtaining location information of the location of the mobile vehicle; obtaining weather information of the location; and obtaining local activity information of the location, wherein the context parameter includes the location information At least one of the weather information and information about the local event. The method of claim 6, further comprising: storing attraction information or product information of the plurality of places; and generating current attraction information or current product information based on the situation parameter, the attraction information or the product information, and The current attraction information or the current product information is added to the augmented reality content. The method of claim 8, wherein the wireless network device causes the user device to turn on the value-added service application when the user device is connected to the wireless network provided by the wireless network device. The method of claim 8, further comprising: periodically updating the attraction information or the product information.