TWI727421B - Mixed reality system - Google Patents

Abstract

A mixed reality system is provided to overcome the problem that the conventional virtual reality game has a low fidelity. The mixed reality system includes a carrier configured to produce a feedback signal and to perform a corresponding operation through a control signal, and a motion sensing device coupled with the carrier and configured to generate the control signal and to perform a corresponding action through the feedback signal. The motion sensing device integrates the feedback signal with a virtual image to thereby generate multimedia information.

Description

Mixed reality system

The present invention relates to a reality system, especially a mixed reality system capable of synchronously controlling actual vehicles.

Existing virtual reality (VR) applications have been widely used in various games, such as: shooting virtual reality games combined with physical guns, and racing virtual reality games combined with multi-axis sports platforms.

Please refer to Figure 1, which is a conventional virtual reality game system 9. The conventional virtual reality game system 9 has a hull 91, a posture sensor 92, a wearable device 93 and a main control The attitude sensor 92 is installed on the hull 91 and used to sense the swing state of the hull 91 to generate a sensed data. The sensed data may include the position of the hull 91, Angle and acceleration state; the wearable device 93 is used for the user riding in the hull 91 to wear and play a virtual reality image; the main controller 94 is coupled to the attitude sensor 92 and the wearable device 93. Adjust the playback speed of the virtual reality image according to the sensing data so that the swing speed of the hull 91 is synchronized. An embodiment of the virtual reality game system 9 similar to the prior art has been disclosed in the patent case of the Republic of China Publication No. 201914669 "Pirate Ship Amusement System and Method in Virtual Reality".

The above-mentioned conventional virtual reality game system 9 cannot provide the user to manipulate the image of the hull 91 in the virtual reality image through a unity sensing device to generate a control signal, and use the control signal to control the ship The body 1 performs corresponding actions, and makes the somatosensory device generate corresponding actions based on the sensing data, resulting in a less real sense of play experience of the user and a problem of poor simulation.

In view of this, the conventional virtual reality game system does still need to be improved.

In order to solve the above-mentioned problems, the purpose of the present invention is to provide a mixed reality system capable of synchronously controlling the actual vehicle.

The elements and components described in the full text of the present invention use the quantifiers "one" or "one" for convenience and to provide the general meaning of the scope of the present invention; in the present invention, it should be construed as including one or at least one, and single The concept of also includes the plural, unless it clearly implies other meanings.

The mixed reality system of the present invention includes: a vehicle for generating a feedback signal and generating a corresponding operation by a control signal; and an integrated sensing device coupled to the vehicle, and the somatosensory device is used for The control signal is generated, and the feedback signal is used to generate a corresponding action. The somatosensory device integrates the feedback signal with a virtual image to generate and display multimedia information.

Accordingly, the mixed reality system of the present invention can drive the body of the somatosensory device to act in real time according to the feedback signal generated by the operating condition of the vehicle, and according to the control signal generated when the body is operated, the real-time Ground to drive the vehicle to run. In this way, the mixed reality system of the present invention can provide users with a safe and highly simulated operation experience, and has the effect of improving the simulation degree of simulated driving.

Wherein, the vehicle generates the feedback signal with a feedback module, and the vehicle controls a transmission unit with an operating unit to transmit the feedback signal to the somatosensory device and receive the control signal. The operating unit uses the The control signal is used to control the corresponding operation of the vehicle. In this way, the actual status of the vehicle can be transmitted to the somatosensory device in the form of the feedback signal in real time, so that the user can truly experience the vehicle’s real-time real-time movement status and visual field information. Has the effect of enhancing entertainment.

Wherein, the feedback module includes at least one of an image sensor and an attitude sensor. In this way, more information about the actual operating conditions of the vehicle can be obtained, which has the effect of further enhancing the simulation degree.

Wherein, the feedback signal generated by the image sensor is integrated with the virtual image to generate the multimedia information. In this way, it has the effect of providing the user with a real-time image combined with a virtual cockpit screen.

Among them, the somatosensory device has a body, and the system is a multi-axis motion platform. In this way, the user can experience the real operating conditions of the vehicle in different moving directions, which has the effect of further enhancing the simulation degree.

Wherein, the somatosensory device generates the control signal with an operating module, and the somatosensory device controls a communication unit with a processing unit to transmit the control signal to the vehicle and receive the feedback signal. The processing unit uses The feedback signal is used to control the main body to produce corresponding actions. In this way, the operation performed by the user on the main body can be transmitted to the vehicle in the form of the control signal in real time, so that the user can safely operate the vehicle in the sea, land, and airspace, which has improved The effect of safety during gaming and learning.

Wherein, the somatosensory device uses a display unit to display the multimedia information, and the display unit is a head-mounted display. In this way, it has the effect of enhancing the experience of virtual reality environment.

In order to make the above and other objectives, features and advantages of the present invention more comprehensible, the following describes the preferred embodiments of the present invention in conjunction with the accompanying drawings in detail as follows:

Please refer to FIG. 2, which is a preferred embodiment of the mixed reality system of the present invention, which includes a carrier 1 and a unity sensing device 2, and the somatosensory device 2 is coupled to the carrier 1.

The carrier 1 is used to generate a feedback signal, and generates a corresponding operation by a control signal. The aspect of the vehicle 1 is not limited in the present invention. For example, it may be a vehicle, a boat, or an aircraft that can be remotely controlled. Specifically, the carrier 1 may have a feedback module 11, a transmission unit 12, and an operating unit 13. The feedback module 11 can be used to sense and obtain the image in front of the lens of the carrier 1 when the carrier 1 is running, and the carrier 1 is the acceleration, angular velocity and other physical quantities to generate the feedback signal. In this embodiment, the feedback module 11 may include at least one of an image sensor (such as a camera) and a posture sensor (CGGs) One. The attitude sensor can integrate three-axis magnetic field, three-axis acceleration and three-axis gyroscope, but it is not limited to this.

The transmission unit 12 is used to receive the control signal and transmit the feedback signal to the somatosensory device 2. The transmission unit 12 can be a wireless transmission module such as WiFi, Zigbee, Sigfox, LoRa or NB-IOT, but not Is limited.

The operating unit 13 is electrically connected to the feedback module 11 and the transmission unit 12. The operating unit 13 can be any electronic device with functions of data processing, data access, timing, signal input and output, for example: programmable logic Controller (PLC), digital signal processor (DSP), microcontroller (MCU) or circuit board of embedded components with the above functions, etc. The operating unit 13 controls the transmission unit 12 to transmit the feedback signal to the motion sensing device 2 and receive the control signal. The operating unit 13 can use the control signal to control the carrier 1 to perform corresponding operations. For example, when the type of the vehicle 1 is a vehicle, the operating unit 13 may be a microcontroller of an electronic control unit (ECU) of the vehicle.

The somatosensory device 2 is coupled to the carrier 1, and the somatosensory device 2 is used to generate the control signal and use the feedback signal to generate corresponding actions. The somatosensory device 2 integrates the feedback signal with a virtual image to generate and display a multimedia information. The multimedia information may have text, sound, picture or/or in the real world according to the composition of the feedback module 11 And images and other information. Specifically, the somatosensory device 2 may have a body 21, a display unit 22, an operation module 23, a communication unit 24, and a processing unit 25. The body 21 can be used for a user to ride in. The aspect is not limited in the present invention, for example: a three-axis motion platform, a four-axis motion platform, a six-axis motion platform or a nine-axis motion platform and other multi-axis motion platforms. The display unit 22 is used to display the multimedia information. In this embodiment, the display unit 22 may be a head-mounted display (HMD). Preferably, the control signal generated by the somatosensory device 2 may be three-axis rotation data when the display unit 22 swings, so as to control the relative swing of the lens of the image sensor of the feedback module 11.

The operation module 23 is used for the user to operate to generate the control signal. For example, when the somatosensory device 2 is applied to a simulated racing game, the operation module 23 may include a light controller, a steering wheel, an accelerator pedal, a brake pedal, a gear lever, a clutch, etc. The operating unit can be understood by those with ordinary knowledge in the field.

The communication unit 24 is used for receiving the feedback signal and transmitting the control signal to the vehicle 1. The communication unit 24 is a device that can correspondingly receive the wireless signal of the transmission unit 12, such as a wireless transmission module such as WiFi, Zigbee, Sigfox, LoRa, or NB-IOT, but it is not limited thereto.

The processing unit 25 is electrically connected to the main body 21, the display unit 22, the operation module 23, and the communication unit 24. The processing unit 25 can be any function with data processing operations, data access, timing, signal input and output, etc. Electronic devices such as programmable logic controllers (PLC), digital signal processors (DSP), microcontrollers (MCU) or circuit boards with embedded components with the above functions. The processing unit 25 controls the communication unit 24 to transmit the control signal to the vehicle 1 and receive the feedback signal. The processing unit 25 can use the feedback signal to control the main body 21 to generate corresponding actions.

The processing unit 25 integrates the feedback signal with a virtual image to generate the multimedia information. Specifically, the multimedia information can be generated by integrating the feedback signal generated by the image sensor of the feedback module 11 with the virtual image. The processing unit 25 uses the display unit 22 to display the multimedia information. For example, when the somatosensory device 2 is applied to a simulated racing game, the virtual image system may include a virtual image of a dashboard screen of the vehicle, and a number of racing interiors with different themes, such as formula racing, motorcycles The interior of racing cars such as cars, touring cars or rally races. The virtual image can be stored in a storage unit 26 coupled to the processing unit 25. The storage unit 26 can be any electronic storage medium for storing electronic data, such as a mechanical hard disk (HDD), a Non-volatile memory such as a solid state drive (SSD) or a flash memory (Flash Memory), but not limited to this.

According to the aforementioned system architecture, when the mixed reality system of the present invention is applied to a simulated racing game, the user can ride on the body 21 of the somatosensory device 2 and wear the display unit 22. The mixed reality system is activated by the processing unit 25, the virtual image is loaded by the storage unit 26, and the virtual image is displayed in the display unit 22, and the user can select the virtual image through a human-machine interface Style to choose the interior of the car. When the game starts, the user controls the control signals generated by the main body 21 through the operation module 23, such as throttle control, steering control, direction light control, brake control, or gear control, etc., through the communication The unit 24 transmits to the vehicle 1 in real time, so that the operating unit 13 controls the vehicle 1 to generate corresponding operations in real time according to the control signal, such as acceleration, deceleration, turning left, turning right, or turning on the turn signal. . On the other hand, the vehicle 1 based on the feedback signal generated by the feedback module 11, such as dashboard image, engine speed, moving direction, etc., can be instantly transmitted to the somatosensory device 2 through the transmission unit 12, so that The processing unit 25 controls the somatosensory device 2 to generate corresponding actions in real time according to the feedback signal. For example, the main body 21 is tilted synchronously according to the posture of the vehicle 1, and the display unit 22 displays multimedia information integrating the dashboard image. .

In summary, the mixed reality system of the present invention can promptly drive the body of the somatosensory device to act according to the feedback signal generated by the operating condition of the vehicle, and according to the control signal generated when the body is operated. , Drive the vehicle to run immediately. In this way, the mixed reality system of the present invention can provide users with a safe and highly simulated operation experience, and has the effect of improving the simulation degree of simulated driving.

Although the present invention has been disclosed using the above-mentioned preferred embodiments, it is not intended to limit the present invention. Anyone who is familiar with the art without departing from the spirit and scope of the present invention may make various changes and modifications relative to the above-mentioned embodiments. The technical scope of the invention is protected. Therefore, the scope of protection of the invention shall be subject to the scope of the attached patent application.

﹝this invention﹞ 1 Vehicle 11 Feedback module 12 Transmission unit 13 Operation unit 2 Somatosensory device 21 Ontology 22 Display unit 23 Operation module 24 Communication unit 25 Processing unit 26 Storage unit ﹝Used ﹞ 9 Virtual reality game system 91 Hull 92 Attitude sensor 93 Wearable devices 94 Main controller

[Picture 1] A system block diagram of a conventional virtual reality game. [Figure 2] A system block diagram of a preferred embodiment of the present invention.

1 Vehicle 11 Feedback module 12 Transmission unit 13 Operation unit 2 Somatosensory device 21 Ontology 22 Display unit 23 Operation module 24 Communication unit 25 Processing unit 26 Storage unit

Claims (7)

A mixed reality system that includes: A vehicle for generating a feedback signal and generating corresponding operations by a control signal; and An integrated sensing device, coupled to the carrier, the somatosensory device for generating the control signal and generating corresponding actions with the feedback signal, the somatosensory device integrating the feedback signal with a virtual image to generate and Display a multimedia message. For example, the mixed reality system described in item 1 of the scope of patent application, wherein the vehicle generates the feedback signal with a feedback module, and the vehicle controls a transmission unit with an operating unit to transmit the feedback signal to the The somatosensory device receives the control signal, and the operating unit controls the vehicle to generate corresponding operations through the control signal. The mixed reality system described in item 2 of the scope of patent application, wherein the feedback module includes at least one of an image sensor and an attitude sensor. The mixed reality system described in item 3 of the scope of patent application, wherein the feedback signal generated by the image sensor is integrated with the virtual image to generate the multimedia information. The mixed reality system described in item 1 of the scope of patent application, wherein the somatosensory device has a body, and the system is a multi-axis motion platform. For example, the mixed reality system described in item 5 of the scope of patent application, wherein the somatosensory device uses an operating module to generate the control signal, and the somatosensory device uses a processing unit to control a communication unit to transmit the control signal To the vehicle and receive the feedback signal, the processing unit uses the feedback signal to control the main body to generate corresponding actions. For example, in the mixed reality system described in claim 1, wherein the somatosensory device uses a display unit to display the multimedia information, and the display unit is a head-mounted display.

Images