TWI805371B - Reality image reconstruction system - Google Patents

Abstract

A reality image reconstruction system which provided, includes: a sensing device for sensing an outdoor environment and a plurality of objects therein to generate environmental parameters of the outdoor environment and object parameters of the plurality of objects; a cloud server for receiving the environmental parameters of the outdoor environment, the object parameters of the plurality of objects, and movement parameters of the sensing device to establish predicted parameters and physical parameters of the plurality of objects in the outdoor environment; a simulation device for establishing a virtual environment synchronized with the outdoor environment according to the environmental parameters of the outdoor environment, the object parameters, the predicted parameters and the physical parameters of the plurality of objects; and an interactive device for interacting with the plurality of objects realistically in the virtual environment according to the environmental parameters of the outdoor environment, the object parameters, the predicted parameters and the physical parameters of the plurality of objects, and the movement parameters of the sensing device.

Description

Reality Image Reconstruction System

The invention relates to an image reconstruction system, in particular to a real-world image reconstruction system for indoor reconstruction of an outdoor environment.

A general virtual engine can only simulate a real outdoor environment into a virtual environment indoors, and the virtual engine creates all the objects in the virtual environment to allow users to immerse themselves in the virtual environment. However, the virtual engine cannot realize the various environmental conditions of the user in the real outdoor environment and the virtual interaction of various objects encountered in the actual experience of the real outdoor environment.

The present invention provides a real-world image reconstruction system that can realize various environmental parameters of the real outdoor environment encountered by users and virtual interaction of various objects encountered in the actual experience of the real outdoor environment.

The real-world image reconstruction system provided by the present invention includes a sensing device, a cloud server, a simulation device, and an interactive device. Among them, the sensing device is used to sense the outdoor environment and multiple objects therein to generate the environmental parameters of the outdoor environment and the object parameters of these objects, and the cloud server is connected to the sensing device and used to receive the environmental parameters of the outdoor environment and the object parameters of these objects And sense the movement parameters of the device to establish the prediction parameters and physical parameters of these objects in the outdoor environment. The simulation device is connected to the cloud server and used to establish the The virtual environment synchronized with the outdoor environment, and the interactive device is connected to the cloud server and used to interact with the objects in the virtual environment according to the environmental parameters of the outdoor environment, the object parameters, prediction parameters and physical parameters of the objects, and the movement parameters of the sensing device Immersive interaction.

In an embodiment of the present invention, the sensing device includes: an optical radar module, an image capture and deep learning module, and a device status sensing module. Among them, the optical radar module is used to sense the outdoor environment to generate the environmental image of the outdoor environment, and the image capture and deep learning module is used to identify the environmental parameters of the outdoor environment from the environmental image of the outdoor environment, and identify these objects to The object parameters of these objects are generated, and the device state sensing module is used to sense the movement parameters of the sensing device.

In an embodiment of the present invention, the environmental parameters of the above-mentioned outdoor environment include weather parameters, time parameters and road condition parameters.

In an embodiment of the present invention, the object parameters of the aforementioned objects include obstacle parameters, vehicle parameters and pedestrian parameters.

In an embodiment of the present invention, the movement parameters of the sensing device include electric drive parameters, power parameters, device status parameters, position parameters, acceleration parameters, and tilt angle parameters.

In an embodiment of the present invention, the above-mentioned prediction parameters and physical parameters of the objects include behavior prediction parameters and relative position relationship parameters.

In an embodiment of the present invention, the aforementioned simulation device is a display screen or a pair of wearable glasses.

In an embodiment of the present invention, the virtual interaction between the interactive device and the objects includes driving control of the sensing device, attitude control and road resistance control of the interactive device.

In one embodiment of the present invention, the above-mentioned sensing device is an automatic guided vehicle (AGV), An autonomous mobile robot (AMR), or an electric vehicle.

In an embodiment of the present invention, the above-mentioned interactive device is a flywheel simulator or an electric drive system.

The real-world image reconstruction system of the present invention uses the sensing device to sense the outdoor environment and the objects therein, and provides movement parameters of various environments that the sensing device will encounter in the outdoor environment, so that the virtual device can truly present the outdoor environment. The virtual environment of environment synchronization can enable the interactive device to truly present the realistic interaction of various objects encountered in the outdoor environment, so as to increase the interactive experience of the user.

In order to make the above and other objects, features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

Please refer to FIG. 1 , which is a schematic diagram of a real-world image reconstruction system provided by an embodiment of the present invention. The real-world image reconstruction system 1 provided by the embodiment of the present invention includes a sensing device 2 , a cloud server 3 , a simulation device 4 , and an interactive device 5 . The cloud server 3 is connected to the sensing device 2 through a wireless network connection, such as 4G, 5G and other wireless communication connections, but the present invention is not limited to the listed connection types. The simulation device 4 and the interactive device 5 are connected to the cloud server 3, and the connection method is a wired and/or wireless network connection, such as Ethernet, wireless area network, 4G, 5G, etc., but the present invention is not limited to such The type of connection listed.

The sensing device 2 is used to sense an outdoor environment and multiple objects therein to generate environmental parameters of the outdoor environment and object parameters of the plurality of objects, wherein the sensing device 2 can be an automatic guided vehicle (AGV), an autonomous mobile robot (AMR), an electric motor Vehicles and other vehicles that can sense and move autonomously, the sensing device 2 at least includes an optical radar module 21, an image capture and deep learning module 22, and a device status sensing module 23, and the optical radar module 21 uses To sense the outdoor environment to generate an environmental image of the outdoor environment, the image capture and deep learning module 22 is used to identify environmental parameters of the outdoor environment and identify multiple objects to generate object parameters of the multiple objects from the environmental image of the outdoor environment, and The device state sensing module 23 is used to sense the movement parameters of the sensing device, wherein all the modules in the sensing device 2 can be formed by hardware, or a combination of hardware and software. In addition, for example, the device status sensing module 23 may include a gyroscope. The outdoor environment is an open space such as a region, city, suburb, etc.; the environmental parameters of the outdoor environment include at least weather parameters such as sunny days, rainy days, etc., time parameters such as time, morning, afternoon, evening, evening, etc., and road condition parameters such as smooth driving , traffic jam, traffic accident, uphill, downhill, etc., but the present invention is not limited to the types of environmental parameters listed; the object parameters of the plurality of objects in the outdoor environment include obstacle parameters such as separation islands, buildings, dead-end alleys, etc., vehicles Parameters such as trucks, box cars, RVs, locomotives, etc., and pedestrian parameters such as adults, children, pets, etc., but the present invention is not limited to the types of object parameters listed. The mobile parameters of the sensing device 2 include electric drive parameters such as abnormal driving warning, driving data collection, power allocation control, etc., power parameters such as power, horsepower, torque, etc., device status parameters such as battery temperature, motor temperature, motor controller temperature, tires, etc. Tire pressure, etc., position parameters such as global positioning system (GPS) coordinates, Beidou satellite navigation system coordinates, etc., acceleration parameters such as average acceleration, clockwise acceleration, etc., and tilt angle parameters such as elevation angle, depression angle, etc., but the present invention is not limited to the above The types of movement parameters listed.

The cloud server 3 is used to receive environmental parameters of the outdoor environment from the sensing device 2, object parameters of the plurality of objects, and movement parameters of the sensing device 2 to establish prediction parameters and physical parameters of the plurality of objects in the outdoor environment. The prediction parameters and physical parameters of the plurality of objects at least include behavior prediction parameters such as the direction of travel of pedestrians, the direction of travel of vehicles, etc., and relative position relationship parameters such as the relative distance between the sensing device 2 and pedestrians such as approaching or far away and/or relative positions such as relative coordinates , the relative distance between the sensing device 2 and the vehicle, such as approaching or far away, and/or the relative position, such as relative coordinates, etc., but the present invention is not limited to the enumerated prediction parameters and physical parameters. In addition, the cloud server 3 can update the movement parameters of the sensing device 2 according to the environmental parameters of the outdoor environment, the object parameters of the plurality of objects, the prediction parameters and physical parameters, and the movement parameters of the sensing device 2, so as to optimize the movement parameters of the sensing device 2, Further let the perception device 2 perform driving assistance such as precise measurement of relative position and velocity between objects, adaptive cruise such as relative position and velocity between moving objects, automatic emergency braking such as using inertial measurement unit, gyroscope, or GPS Information to avoid collisions, forward collision warning such as collecting information through the control area network bus (CAN Bus) to determine the triggering of warning sound/tactile signals, lane keeping/deviation such as calculating the relative position, angle and Speed to maintain/deviate vehicles, blind spot warning/lane change assistance such as creating images around the vehicle through images, traffic sign recognition such as recognizing traffic signals and signs through images, pedestrian and bicycle warning such as recognizing pedestrians and bicycles through images, road Danger reminders such as broken bridges, flooding, landslides, icy roads, tunnel fires, and road condition reminders such as congestion, foggy areas, and poor visibility.

The simulation device 4 is used to establish a virtual environment synchronized with the outdoor environment according to the environmental parameters of the outdoor environment, the object parameters of the plurality of objects, the prediction parameters and the physical parameters. The simulation device 4 can be a display screen or wearable glasses, and the purpose is to allow the user to immerse in the virtual environment to simultaneously realize various environmental parameters of the real outdoor environment that will be encountered, thereby increasing the on-the-spot experience.

The interactive device 5 is used for realistically interacting with the multiple objects in the virtual environment according to the environmental parameters of the outdoor environment, the object parameters, prediction parameters and physical parameters of the multiple objects, and the movement parameters of the sensing device 2 . Among them, the interactive device 5 can be a flywheel simulator or an electric drive system, so that the user can experience various environmental parameters of the real outdoor environment when riding, and the interactive device 5 can interact with multiple objects in a realistic manner, including the driving of the sensing device 2. Control, and attitude control and road resistance control of the interactive device 5, so that the user can experience the realistic interaction of various objects that will be encountered in the real outdoor environment when riding, so that the user can increase the interaction experience.

That is to say, when the user uses the real-world image reconstruction system 1 provided by the present invention, he can experience the real outdoor environment synchronously indoors, realize various environmental parameters of the real outdoor environment that the user will encounter, and actually experience the real outdoor environment. The virtual interaction of various objects encountered in the environment. In one example, when the sensing device 2 is on an uphill road section, the sensing device 2 will perceive the environmental parameters of the real outdoor environment and the movement parameters of the sensing device 2, and the interactive device 5 that the user rides, such as a flywheel simulator, will be adjusted at the same time to simulate the use of The posture of the rider riding uphill and the corresponding resistance of the uphill can be felt by the user, so that the user can experience the realistic feeling.

To sum up, the real-world image reconstruction system of the present invention uses the sensing device to sense the outdoor environment and the objects therein, and provides the movement parameters of the sensing device in various environments that the sensing device will encounter in the outdoor environment, so it can make the simulation The device truly presents a virtual environment synchronized with the outdoor environment to the user, and enables the interactive device to truly present the realistic interaction of various objects encountered in the outdoor environment, so that the user can increase the interactive experience.

Although the present invention has been disclosed above with the embodiments, it is not intended to limit the present invention. Those with ordinary knowledge in the technical field of the present invention can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be defined by the scope of the appended patent application.

1: Real-world image reconstruction system 2: Sensing device 3: Cloud server 4: Simulated equipment 5: Interactive equipment 21: Optical radar module 22: Image capture and deep learning module 23:Equipment status sensing module

FIG. 1 is a schematic diagram of a real-world image reconstruction system provided by an embodiment of the present invention.

1: Real-world image reconstruction system

2: Sensing device

3: Cloud server

4: Simulated equipment

5: Interactive equipment

21: Optical radar module

22: Image capture and deep learning module

23:Equipment status sensing module

Claims (10)

A real-world image reconstruction system, comprising: A sensing device for sensing an outdoor environment and a plurality of objects therein to generate environmental parameters of the outdoor environment and object parameters of the objects; A cloud server, connected to the sensing device, and used to receive environmental parameters of the outdoor environment, object parameters of the objects, and movement parameters of the sensing device to establish prediction parameters and physical parameters of the objects in the outdoor environment; A simulation device, connected to the cloud server, and used to create a virtual environment synchronized with the outdoor environment according to the environmental parameters of the outdoor environment, the object parameters, prediction parameters and physical parameters of the objects; and An interactive device, connected to the cloud server, and used to simulate the objects in the virtual environment according to the environmental parameters of the outdoor environment, the object parameters, prediction parameters and physical parameters of the objects, and the movement parameters of the sensing device interactive. The real-world image reconstruction system as described in Claim 1, wherein the sensing device includes: An optical radar module is used to sense the outdoor environment to generate an environmental image of the outdoor environment; An image capture and deep learning module, used to identify environmental parameters of the outdoor environment from environmental images of the outdoor environment, and identify the objects to generate object parameters of the objects; and A device state sensing module is used for sensing the movement parameters of the sensing device. The real-world image reconstruction system as described in Claim 1, wherein the environmental parameters of the outdoor environment include weather parameters, time parameters and road condition parameters. The real-world image reconstruction system as described in Claim 1, wherein the object parameters of the objects include obstacle parameters, vehicle parameters and pedestrian parameters. The real-world image reconstruction system according to claim 1, wherein the movement parameters of the sensing device include electric drive parameters, power parameters, device status parameters, position parameters, acceleration parameters and tilt angle parameters. The real-life image reconstruction system as described in Claim 1, wherein the predicted parameters and physical parameters of the objects include behavior predicted parameters and relative position relationship parameters. The real-world image reconstruction system as described in claim 1, wherein the simulation device is a display screen or a wearable glasses. The real-world image reconstruction system as described in Claim 1, wherein the realistic interaction between the interactive device and the objects includes driving control of the sensing device, attitude control and road resistance control of the interactive device. The real-world image reconstruction system according to claim 1, wherein the sensing device is an automatic guided vehicle (AGV), an autonomous mobile robot (AMR), or an electric vehicle. The real-world image reconstruction system as described in claim 1, wherein the interactive device is a flywheel simulator or an electric drive system.

Images