JP2024018887A - Intelligent terminal, handheld device, virtual system and intelligent terminal spatial positioning method - Google Patents

Abstract

The present invention provides an intelligent terminal, a handheld device (Chinese: palm desk), a virtual system, and a method for spatial positioning of an intelligent terminal. In an intelligent terminal that is detachably connected to a virtual head mounted display device, a main control chip that switches the function of the intelligent terminal from the virtual head mounted display function to the handheld device function is configured to enable the intelligent terminal to display the virtual head mounted display device. When attached to a device, it switches the functionality of the intelligent terminal from handheld device functionality to virtual head-mounted display functionality. The main control chip also determines the spatial positioning information of the intelligent terminal based on the environmental information of the intelligent terminal photographed by the first imaging device and the IMU data of the intelligent terminal detected by the first inertial sensor, and Achieve accurate positioning of the terminal. [Selection diagram] Figure 3

Description

The present invention relates to the field of mobile device technology, and specifically relates to an intelligent terminal, a handheld device (Chinese: palm desk), a virtual system and a method for spatial positioning of an intelligent terminal.

Currently, many new types of virtual reality (VR) head-mounted display devices consisting of VR boxes and mobile terminals have appeared on the market. The performance of such new VR head-mounted display equipment is mainly determined by the performance of the mobile terminal, which is much inferior to the traditional computer-VR head-mounted display equipment or VR all-in-one. Computer-VR head-mounted display devices, or VR all-in-one display devices and VR virtual head-mounted display devices, are generally inseparable and can only be used as a set, which limits their convenience.

On the other hand, there are currently some portable devices for game consoles, such as handheld devices, that are equipped with a host and a handle, and can realize command input through the handle for a gaming experience. There is no function to experience it.

Therefore, in order to meet the purpose of allowing users to experience two different product functions by purchasing one product, we have developed a new type of combined intelligent terminal that has the separable characteristics of a virtual box and can embody the functions of a handheld device. Equipment must be provided.

Also, in the prior art, it is difficult to achieve accurate repositioning of the mobile terminal relative to the control handle when changing the operating mode of the mobile terminal, for example, switching from handheld device function to virtual head-mounted display function. Can not.

In view of this, the present invention provides an intelligent terminal, a handheld device, a virtual system and a method for spatial positioning of an intelligent terminal, so that only one mobile terminal can experience the functions of two types of products, virtual and handheld devices. First, it solves the technical problem in the prior art that the mobile terminal cannot be accurately positioned in space when the operating mode of the mobile terminal is changed.

According to one aspect of the invention, the invention provides an intelligent terminal. The intelligent terminal includes an intelligent terminal body that can be detachably attached to a virtual head-mounted display device or a hand-held device, a display screen disposed on a first side of the intelligent terminal body, and a display screen disposed within the intelligent terminal body. a function switching module for controlling switching of the intelligent terminal between a virtual head-mounted display function and a handheld device function; a control chip; a first imaging device disposed on a second side surface of the intelligent terminal main body, the first imaging device and the second side surface being opposing surfaces; and the intelligent terminal main body. a first inertial sensor disposed in the intelligent terminal body for detecting IMU data of the intelligent terminal main body, and the first imaging device and the first inertial sensor are each communicatively connected to the main control chip.

In one embodiment of the present invention, the intelligent terminal further includes a brightness controller disposed within the intelligent terminal body, the brightness controller being communicatively connected to the main control chip and the display screen, respectively; is switched from the handheld device function to the virtual head mounted display function, the brightness controller reduces the display brightness of the display, and the intelligent terminal switches from the virtual head mounted display function to the handheld device function. When the brightness controller is turned on, the brightness controller increases the display brightness of the display.

In one embodiment of the invention, the intelligent terminal further includes an image processor disposed within the intelligent terminal body, the image processor is communicatively connected to the main control chip, and the image processor is connected to the display screen. It is used to perform asynchronous space warp processing, asynchronous time warp processing, and image rendering processing on the image information displayed above.

In one embodiment of the present invention, the intelligent terminal further includes a second imaging device disposed on the first side of the intelligent terminal body, the second imaging device being communicatively connected to the main control chip. Ru.

In one embodiment of the present invention, the number of the first imaging devices is four, and the four first imaging devices are divided into two imaging device sets, and the two first imaging devices in each imaging device set are divided into two imaging device groups. 1 imaging device is symmetrical about the center of the intelligent terminal body.

As a second aspect of the invention, the invention also provides a handheld device. This handheld device includes the above-mentioned intelligent terminal and a handheld device handle connected to the intelligent terminal, the handheld device handle is provided with a first infrared sensor and a second inertial sensor, and the handheld device handle is provided with a first infrared sensor and a second inertial sensor, and the second inertial sensor A sensor is used to detect IMU data of the handheld device handle, and the second inertial sensor is communicatively connected to the main control chip in the intelligent terminal.

In one embodiment of the present invention, the main control chip includes a first control unit and a first calculation unit, and the first control unit includes the function switching module, the first imaging device, and the first calculation unit. each of the first control unit is communicatively connected to the inertial sensors, and the first control unit controls the first imaging device to capture a first image of the surrounding environment in which the intelligent terminal is located; is used to detect IMU data of an intelligent terminal, and the first calculation unit is connected to the first control unit, the first imaging device, the first infrared sensor, the first inertial sensor and the second inertial sensor. The first calculation unit obtains the first image and the IMU data of the intelligent terminal body, calculates the IMU data of the intelligent terminal body and the first image, and calculates the space of the intelligent terminal. Used to generate positioning information.

In one embodiment of the present invention, the main control chip further includes a second control unit and a second calculation unit, and the second control unit includes the function switching module, the first imaging device, and the second calculation unit. the second control unit is communicatively connected to two inertial sensors, respectively, and the second control unit controls the first imaging device to control the first imaging device located on the handheld device handle when the function switching module switches the intelligent terminal to a handheld device function. 1 infrared sensor and controlling a second inertial sensor located on the handheld device handle to detect IMU data of the handheld device handle, the second calculation unit is used to capture the IMU data of the handheld device handle; The second control unit, the first imaging device and the second inertial sensor are each communicatively connected, and the second calculation unit calculates a first light spot image of the first infrared sensor transmitted by the first imaging device. , and obtaining the IMU data of the handheld device handle transmitted by the second inertial sensor, and calculating the first light point image, the IMU data of the handheld device handle and the spatial positioning information of the intelligent terminal, and Used to generate spatial positioning information for the handheld device handle.

In one embodiment of the invention, the handheld device further includes a connector, a multifunction pushbutton, and a multifunction processor, the connector being communicatively connected to the handheld device handle and the intelligent terminal, respectively; A multi-function push button is disposed on the handheld device handle, the multi-function processor is connected to the multi-function push button and the main control chip of the intelligent terminal, respectively, and the multi-function processor is configured to It is used to receive the operation command input through the button and send the operation command to the main control chip.

As a third aspect of the invention, the invention also provides a virtual system. This virtual system includes the above-mentioned intelligent terminal, a virtual head-mounted display device in which the intelligent terminal is removably attached to the main body of the virtual head-mounted display device, and a mounted display handle, the virtual head mounted display handle comprising: a control handle; a second infrared sensor disposed on the control handle; and a second infrared sensor disposed on the control handle to measure IMU data of the control handle. a third inertial sensor for the control handle, the control handle and the third inertial sensor each being communicatively connected to the main control chip.

In one embodiment of the invention, the virtual head-mounted display handle further includes a handle casing, the handle casing including an annular portion and a grasping portion, and a center of the grasping portion housing the control handle. A recess is provided for fixing.

In one embodiment of the present invention, the main control chip includes a third control unit and a third calculation unit, and the third control unit is configured to control the function switching module, the first imaging device, and the first inertial sensor. The third control unit controls the first imaging device to take a first image of the surrounding environment in which the intelligent terminal is located, and controls the first inertial sensor to take a first image of the surrounding environment in which the intelligent terminal is located. the third calculation unit is communicatively connected to the third control unit, the first imaging device, the first infrared sensor and the first inertial sensor, and the third calculation unit is used to detect IMU data; The unit is used to obtain the first image and the IMU data of the intelligent terminal body, and calculate the IMU data of the intelligent terminal body and the first image to generate spatial positioning information of the intelligent terminal. Ru.

In one embodiment of the present invention, the main control chip further includes a fourth control unit and a fourth calculation unit, and the fourth control unit includes the function switching module, the first imaging device, and the second infrared sensor. and the third inertial sensor, respectively, and the fourth control unit controls the first imaging device to display the virtual head-mounted display when the function switching module switches the intelligent terminal to a virtual head-mounted display function. The fourth calculation unit is used to photograph a second infrared sensor located on the operating handle and control a third inertial sensor located on the virtual operating handle to detect IMU data of the virtual operating handle; The third computing unit is communicatively connected to the first imaging device and the third inertial sensor, respectively, and the fourth computing unit is configured to calculate a second light spot image of the second infrared sensor transmitted by the first imaging device. , and obtain the IMU data of the virtual operating handle transmitted by a third inertial sensor, calculate the spatial positioning information of the intelligent terminal, the second light spot image and the IMU data of the virtual operating handle, and calculate the IMU data of the virtual operating handle transmitted by the third inertial sensor. Used to generate spatial positioning information for the operating handle.

As a fourth aspect of the invention, the invention also provides an intelligent terminal spatial positioning method for locating the above intelligent terminal. In the spatial positioning method of the intelligent terminal, the main control chip controls the first imaging device located in the intelligent terminal to take a first image of the surrounding environment in which the intelligent terminal is located; controlling a sensor to detect IMU data of the intelligent terminal; and causing the main control chip to acquire a first image of the surrounding environment in which the intelligent terminal is located, taken by the first imaging device; the main control chip acquiring IMU data of the intelligent terminal body detected by the first inertial sensor; and the main control chip calculating the IMU data of the intelligent terminal body and the first image. , generating spatial positioning information for the intelligent terminal.

In one embodiment of the invention, when the intelligent terminal is communicatively connected to a handheld device handle, the method for spatially locating the intelligent terminal includes the steps of: the function switching module switching the intelligent terminal to a handheld device function; A control chip controls the first imaging device to image a first infrared sensor located on the handheld device handle, and controls a second inertial sensor located on the handheld device handle to capture IMU data of the handheld device handle. detecting a first light spot image of the first infrared sensor transmitted by the first imaging device and IMU data of the handheld device handle transmitted by the second inertial sensor; and calculating spatial positioning information of the intelligent terminal, the first light spot image, and IMU data of the handheld device handle to generate spatial positioning information of the handheld device handle.

In one embodiment of the present invention, when the intelligent terminal is attached to a virtual head-mounted display device, and the intelligent terminal is communicatively connected to a virtual operating handle, the spatial positioning method of the intelligent terminal comprises:
the function switching module switching the intelligent terminal to a virtual head-mounted display function; the control chip controlling the first imaging device to image a second infrared sensor located on the virtual operating handle; controlling a third inertial sensor located on the virtual operating handle to detect IMU data of the virtual operating handle; obtaining two light spot images and the IMU data of the virtual operating handle transmitted by a second inertial sensor, and calculating spatial positioning information of the intelligent terminal, the second light spot image and the IMU data of the virtual operating handle; and generating spatial positioning information of the virtual operating handle.

The intelligent terminal provided by the present invention can be removably connected to a virtual head-mounted display device. That is, the virtual system can be made into a separate virtual system. That is, the intelligent terminal and the virtual head-mounted display device are installed so that they can be separated. When the intelligent terminal is separated from the virtual head-mounted display equipment, the intelligent terminal can establish a communication connection with the handheld device handle, and the main control chip in the intelligent terminal can transfer the functions of the intelligent terminal to the virtual head-mounted display. Switch from the function to the handheld device function, whereby the intelligent terminal and the handheld device handle constitute a handheld device. When the intelligent terminal is attached to the virtual head-mounted display device, the main control chip in the intelligent terminal switches the function of the intelligent terminal from the hand-held device function to the virtual head-mounted display function, so that the intelligent terminal, the virtual hand-held device and The virtual head mounted display device constitutes a virtual system. As a result, the intelligent terminal can have dual functions, and the user can experience the functions of two types of products, the virtual system and the handheld device, through one intelligent terminal. In addition, the intelligent terminal is provided with a first inertial sensor and a first imaging device, and the main control chip receives environmental information of the intelligent terminal photographed by the first imaging device and information of the intelligent terminal detected by the first inertial sensor. Based on the IMU data, the spatial positioning information of the intelligent terminal, ie 6DOF data, can be determined, thereby realizing accurate positioning of the intelligent terminal.

The above and other objects, features and advantages of the present invention will become more apparent by describing embodiments of the present invention in more detail with reference to the drawings. The drawings provide a further understanding of the embodiments of the invention, constitute a part of this specification, and together with the embodiments of the invention are intended to explain the invention, but are not intended to limit the invention. In the drawings, like reference numbers usually represent like parts or steps.
1 shows a front view of an intelligent terminal provided by an embodiment of the present invention; FIG. Figure 3 shows a rear view of an intelligent terminal provided by an embodiment of the present invention. FIG. 3 shows an operation principle diagram of an intelligent terminal provided by an embodiment of the present invention. FIG. 6 shows a working principle diagram of an intelligent terminal provided by another embodiment of the present invention. FIG. 6 shows a working principle diagram of an intelligent terminal provided by another embodiment of the present invention. FIG. 3 shows an operational principle diagram of a handheld device provided by an embodiment of the present invention. FIG. 6 shows a working principle diagram of a handheld device provided by another embodiment of the present invention. 8 shows a flowchart of a method for spatial positioning of an intelligent terminal of a handheld device shown in FIG. 7; FIG. 6 shows a working principle diagram of a handheld device provided by another embodiment of the present invention. 10 shows a flowchart of a method for spatial positioning of an intelligent terminal of a handheld device shown in FIG. 9; FIG. 2 shows an operational principle diagram of a virtual system provided by an embodiment of the present invention. FIG. 6 shows an operational principle diagram of a virtual system provided by another embodiment of the present invention. 13 shows a flowchart of a method for spatial positioning of an intelligent terminal of the virtual system shown in FIG. 12; FIG. 6 shows an operational principle diagram of a virtual system provided by another embodiment of the present invention. 15 shows a flowchart of a method for spatial positioning of an intelligent terminal of the virtual system shown in FIG. 14; FIG. 6 shows a structural schematic diagram of a virtual operating handle provided by another embodiment of the present invention. 1 is a diagram illustrating the operating principle of an electronic device provided by an embodiment of the present invention.

In the description of this invention, "plurality" means at least two, such as three, unless otherwise defined. All directional indications (up, down, left, right, front, back, top, bottom, etc.) in the embodiments of this application refer to the relative relationship between each member in a certain orientation (as shown in the figure). It is used only to explain physical relationships, movement conditions, etc. If the particular pose changes, the direction indication also changes accordingly. Also, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or apparatus comprising a series of steps or units is not limited to the steps or units listed, but optionally further comprises steps or units not listed, or processes such as , may further include other steps or units specific to the method, product, or apparatus.

Reference herein to an "embodiment" also means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of the term in various places in the specification are not necessarily referring to the same embodiment, nor are the terms mutually exclusive, independent or alternative embodiments. Those skilled in the art clearly and implicitly understand that the embodiments described herein can be combined with other embodiments.

The following describes the technical solution according to the embodiments of the present invention clearly and in detail with reference to the accompanying drawings according to the embodiments of the present invention. Of course, the embodiments described herein are only some, but not all, of the embodiments of the present invention. All other embodiments that those skilled in the art can obtain based on the embodiments of the present invention without creative work should fall within the protection scope of the present invention.

1 shows a front view of the intelligent terminal 1 provided by the present invention, FIG. 2 shows a rear view of the intelligent terminal 1 provided by the present invention, and FIG. 3 shows an intelligent terminal 1 provided by the present invention. 1 shows a diagram of the operating principle of No. 1. As shown in FIGS. 1, 2, and 3, this intelligent terminal 1 includes an intelligent terminal main body 100, a display screen 200, a function switching module 601, a main control chip 600, a first imaging device 400, 1 inertial sensor 700.
The intelligent terminal body 100 can be removably attached to a virtual head-mounted display device 31 or a handheld device such as a handheld device.
The display screen 200 is disposed on the first side 101 of the intelligent terminal body 100, has a normal display function, and can display video information, image information, etc.
The function switching module 601 is located in the intelligent terminal body 100, that is, the function switching module 601 can control the switching of the intelligent terminal 1 between the virtual head-mounted display function and the handheld device function. For example, when the intelligent terminal 1 is attached to a virtual head-mounted display, the intelligent terminal 1 and the virtual head-mounted display are connected to a virtual system, such as a virtual reality system (hereinafter referred to as a VR system) or an augmented reality system (hereinafter referred to as an AR system). system). In this case, the function switching module 601 receives the information that the intelligent terminal 1 is attached to the virtual head-mounted display, turns on the virtual head-mounted display function according to the received information, and switches the intelligent terminal 1 to the virtual head-mounted display. It can be adapted to perform head-mounted display functions.
The main control chip 600 is disposed within the intelligent terminal body 100 and is communicatively connected to the function switching module 601 .
The first imaging device 400 is disposed on the second side surface 102 of the intelligent terminal body 100, and the first side surface 101 and the second side surface 102 are opposite surfaces.
The first imaging device 400 can be used to photograph an infrared lamp. When the intelligent terminal 1 is attached to the virtual head-mounted display device 31, the intelligent terminal 1, the virtual operating handle 32 and the virtual head-mounted display device 31 constitute a virtual system. In this case, the intelligent terminal 1 needs to establish a communication connection with the virtual operating handle 32. An infrared lamp is attached to the virtual operation handle 32. At this time, the main control chip 600 of the intelligent terminal 1 can control the first imaging device 400 to turn on. Thereby, the first imaging device 400 photographs the infrared lamp located on the virtual operation handle 32, photographs a light spot image of the infrared lamp located on the virtual operation handle 32, and then transmits the light spot image to the main control chip 600. Transmit.
The first imaging device 400 can also be used to capture the surrounding environment in which the intelligent terminal 1 is located. Whether the intelligent terminal 1 performs a virtual head-mounted display function or a handheld device function, it is necessary to position the intelligent terminal 1. For example, when the intelligent terminal 1 and the handheld device handle 2 are combined to form a handheld device, the intelligent terminal 1 and the handheld device handle 2 need to be positioned. When the intelligent terminal 1 is attached to the virtual head-mounted display device 31 and forms a virtual system with the virtual operating handle 32, the intelligent terminal 1 and the virtual operating handle 32 also need to be positioned. The first imaging device 400 can form image information or video information that can reflect the surrounding environment by photographing the surrounding environment where the intelligent terminal 1 is located, and send the image information and video information to the main control chip 600. can. The main control chip 600 can obtain the location information of the intelligent terminal 1 based on image information or video information.
The first inertial sensor 700 is disposed on the intelligent terminal body 100 and is used to detect IMU data of the intelligent terminal body 100. The first inertial sensor 700 detects IMU data of the intelligent terminal 1 and then transmits the IMU data to the main control chip 600. The main control chip 600 generates spatial positioning information of the intelligent terminal 1 based on the IMU data and the image information or video information of the environment in which the intelligent terminal 1 is located photographed by the first imaging device 400. 6 DOF data can be determined, ie, 6 angular degrees of freedom can be obtained based on the translational and rotational degrees of freedom.

Specifically, an inertial sensor (abbreviated as Inertial Measurement Unit, IMU) is a device that measures the three-axis attitude angle (or angular velocity) and acceleration of an object. Typically, an IMU includes three single-axis accelerometers and three single-axis gyroscopes. An accelerometer detects the acceleration signal of an object in three independent axes in the carrier coordinate system, a gyroscope detects the angular velocity signal of the carrier relative to the navigation coordinate system, and an inertial sensor detects the angular velocity of an object in three-dimensional space. Based on this, the orientation of the object, including its rotational degrees of freedom, can be determined. The rotational degree of freedom means the degree of freedom regarding three positions: up and down, front and back, and left and right. The IMU data is the result data of the object detected by the inertial sensor, that is, the angular velocity and acceleration of the object in three-dimensional space detected by the inertial sensor. Therefore, the first inertial sensor 700 can detect IMU data for detecting the intelligent terminal body 100, and can determine the posture of the intelligent terminal body 100, for example, based on the IMU data for detecting the intelligent terminal body 100. 100 rotational degrees of freedom can be solved. The rotational degree of freedom means the degree of freedom regarding three positions: up and down, front and back, and left and right.

The intelligent terminal 1 provided by the present invention can be detachably connected to a virtual head-mounted display device 31. That is, the virtual system can be made into a separate virtual system. That is, the intelligent terminal 1 and the virtual head-mounted display device 31 are installed so that they can be separated. When the intelligent terminal 1 is separated from the virtual head-mounted display device 31, the intelligent terminal 1 can establish a communication connection with the handheld device handle 2, and the main control chip 600 in the intelligent terminal 1 The function is switched from a virtual head-mounted display function to a handheld device function, whereby the intelligent terminal 1 and the handheld device handle 2 constitute a handheld device. When the intelligent terminal 1 is attached to the virtual head-mounted display device 31, the main control chip 600 in the intelligent terminal 1 switches the function of the intelligent terminal 1 from the handheld device function to the virtual head-mounted display function, so that the intelligent terminal 1. The virtual handheld device and the virtual head-mounted display device 31 constitute a virtual system. As a result, the intelligent terminal 1 can have two functions, and the user can experience the functions of two types of products, the virtual system and the handheld device, through one intelligent terminal 1. Furthermore, the intelligent terminal 1 is provided with a first inertial sensor 700 and a first imaging device 400. The main control chip 600 provides spatial positioning information of the intelligent terminal 1 based on the environmental information of the intelligent terminal 1 photographed by the first imaging device 400 and the IMU data of the intelligent terminal 1 detected by the first inertial sensor 700. That is, 6DOF data can be determined, thereby realizing accurate positioning of the intelligent terminal 1.

Optionally, as shown in FIG. 1, the intelligent terminal 1 further includes a third imaging device 300 disposed on the second side 102 of the intelligent terminal body 100. The third imaging device 300 is a depth imaging device and is capable of photographing the surrounding environment where the intelligent terminal 1 is located. In addition to a planar image of the surrounding environment where the intelligent terminal 1 is located, the depth imaging device captures depth information of the surrounding environment where the intelligent terminal 1 is located, that is, the three-dimensional position and size of the surrounding environment where the intelligent terminal 1 is located. Since the main control chip 600 can acquire information and obtain richer positional relationships between objects through distance information, the main control chip 600 uses the first imaging device when performing spatial positioning for the intelligent terminal 1. Based on the environmental information photographed by the third imaging device 400, the depth information of the environment photographed by the third imaging device 300, and the IMU data of the intelligent terminal 1, the spatial positioning information of the intelligent terminal 1 can be determined. The accuracy of spatial positioning of the intelligent terminal 1 is improved.

Optionally, the number of first imaging devices 400 is four, and the four first imaging devices 400 are divided into two imaging device sets, and the two first imaging devices 400 in each imaging device set are: It is symmetrical about the center of the intelligent terminal body 100. For example, when the intelligent terminal main body 100 is a quadrilateral, that is, when the second side surface 102 on the intelligent terminal main body 100 is a quadrilateral, the four first imaging devices 400 are placed at the four corners of the quadrilateral. can be distributed. In this way, the position of the first imaging device 400 can be maximized to capture the light spot image of the infrared sensor and the surrounding environment where the intelligent terminal 1 is located.

In one embodiment of the invention, FIG. 4 shows an operating principle diagram of the intelligent terminal 1 provided by another embodiment of the invention. As shown in FIG. 4, the intelligent terminal further includes a brightness controller 201 disposed within the intelligent terminal body. Brightness controller 201 is communicatively connected to main control chip 600 and display screen 200, respectively. When the intelligent terminal 1 is attached to the virtual head-mounted display, the function switching module 601 switches the function of the intelligent terminal 1 to the virtual head-mounted display function, and the brightness controller 201 reduces the brightness of the display screen 200. Since the brightness of the display screen 200 decreases, the lighting time of the display pixels when displaying one frame of image on the display screen 200 also decreases. That is, when one frame of a display image is shortened, the length of time for lighting up a pixel is shortened, and the afterglow time is also shortened accordingly. Therefore, deterioration in the clarity of the display screen 200 due to the smear phenomenon is reduced, and the user's experience is improved. When the intelligent terminal 1 is attached to the handheld device equipment, the function switching module 601 switches the function of the intelligent terminal 1 to the handheld device function, and the brightness controller 201 improves the brightness of the display screen 200.

In one embodiment of the invention, FIG. 5 shows an operating principle diagram of the intelligent terminal 1 provided by another embodiment of the invention. As shown in FIG. 5, this intelligent terminal 1 further includes an image processor 800 disposed within the intelligent terminal body 100. Image processor 800 is communicatively coupled to main control chip 600 . When the intelligent terminal 1 is attached to the virtual head-mounted display device 31, the function switching module 601 in the main control chip 600 switches the function of the intelligent terminal 1 to the virtual head-mounted display function, and the main control chip 600 Information that the intelligent terminal 1 has been switched to the virtual head-mounted display function is transmitted to the image processor 800. Since the image processor 800 starts operating based on this information, it performs asynchronous space warp processing, asynchronous time warp processing, and image rendering processing on the image information displayed on the display screen 200 to improve the clarity of the screen. can be done.

In one embodiment of the present invention, as shown in FIG. 1, the intelligent terminal 1 further includes a second imaging device 500 disposed on the first side surface 101 of the intelligent terminal body 100. The second imaging device 500 is used to photograph the user of the intelligent terminal 1 . The second imaging device 500 is communicatively connected to the main control chip 600. The second imaging device 500 can take a picture of the user, that is, can take a selfie of the user. The main control chip 600 can obtain the user's image information, so it can authenticate the user or generate a virtual profile image of the user based on the user's image information.

According to a second aspect of the invention, the invention further provides a handheld device. FIG. 6 shows an operational principle diagram of a handheld device provided by an embodiment of the present invention. As shown in FIG. 6, this handheld device includes the above-mentioned intelligent terminal 1 and a handheld device handle 2 connected to the intelligent terminal 1. The handheld device handle 2 is provided with a first infrared sensor 22 and a second inertial sensor 23 . The second inertial sensor 23 is used to detect IMU data of the handheld device handle 2. The second inertial sensor 23 is communicatively connected to the main control chip 600 in the intelligent terminal 1 . The first imaging device 400 on the intelligent terminal 1 is used to photograph the first infrared sensor 22 and the surrounding environment in which the intelligent terminal 1 is located. When the handheld device handle 2 is communicatively connected to the intelligent terminal 1, the function switching module 601 switches the function of the intelligent terminal 1 to the handheld device function. At this time, the intelligent terminal 1 and the handheld device handle 2 constitute a handheld device.

Optionally, as shown in FIG. 6, the handheld device further includes a connector 24, a multifunction pushbutton 25, and a function processor 26. Connector 24 is communicatively connected to handheld device handle 2 and intelligent terminal 1, respectively. The connector 24 can realize a communication connection between the handheld device handle 2 and the intelligent terminal 1. A multi-function pushbutton 25 is located on the handheld device handle 2. The function processor 26 is connected to the multifunction push button 25 and the main control chip 600 of the intelligent terminal 1, respectively. The function processor 26 is used to receive operation commands input by the user via the multi-function push button 25 and send the operation commands to the main control chip 600.

In one embodiment of the present invention, after the intelligent terminal 1 and the handheld device handle 2 constitute a handheld device, the spatial positioning for both the intelligent terminal 1 and the handheld device handle 2 is performed during the process of the user operating the handheld device. need to be done. FIG. 7 shows an operational principle diagram of a handheld device provided by another embodiment of the present invention. As shown in FIG. 7, the main control chip 600 includes a first control unit 602 and a first calculation unit 603.
The first control unit 602 is communicatively connected to the function switching module 601, the first imaging device 400, and the first inertial sensor 700, respectively. The first control unit 602 controls the first imaging device 400 to capture a first image of the surrounding environment where the intelligent terminal 1 is located, and controls the first inertial sensor 700 to detect IMU data of the intelligent terminal 1. used for.
The first calculation unit 603 is communicatively connected to the first control unit 602, the first imaging device 400, the first infrared sensor 22, the first inertial sensor 700, and the second inertial sensor 23, respectively. The first calculation unit 603 is used to obtain the first image and the IMU data of the intelligent terminal body, calculate the IMU data of the intelligent terminal body and the first image, and generate spatial positioning information of the intelligent terminal. .

Specifically, FIG. 8 shows a flowchart of a method for spatial positioning of the intelligent terminal 1 of the handheld device shown in FIG. That is, the positioning method during spatial positioning of the intelligent terminal of the handheld device shown in FIG. 7 is shown in FIG. The spatial positioning method of the intelligent terminal 1 includes the following steps S101 to S103.
In step S101, the function switching module 601 receives connection information that the intelligent terminal 1 is communicatively connected to the handheld device handle 2, and switches the function of the intelligent terminal 1 to the handheld device function based on the connection information.
In step S102, when the function switching module 601 successfully switches the intelligent terminal 1 to the handheld device function, the first control unit 602 controls the first imaging device 400 located in the intelligent terminal 1 to position the intelligent terminal 1. A first image of the surrounding environment is taken, and the first inertial sensor 700 is controlled to detect IMU data of the intelligent terminal 1.
Under the control of the first control unit 602 , the first imaging device 400 photographs the surrounding environment in which the intelligent terminal 1 is located to form a first image, and transmits the first image to the first calculation unit 603 . The first inertial sensor 700 detects the IMU data of the intelligent terminal 1 under the control of the first control unit 602 and transmits the IMU data of the intelligent terminal 1 to the first calculation unit 603 .
In step S103, after the first calculation unit 603 receives the first image transmitted by the first imaging device 400 and the IMU data of the intelligent terminal 1 transmitted by the first inertial sensor 700, the first calculation unit 603 , calculate the IMU data and the first image of the intelligent terminal main body 100 to generate spatial positioning information of the intelligent terminal 1.

The first inertial sensor 700 is used to detect IMU data of the intelligent terminal body 100. IMU data means three rotation angle degrees of freedom. After the first inertial sensor 700 detects the IMU data of the intelligent terminal 1 , it transmits the IMU data to the first calculation unit 603 . The first calculation unit 603 calculates the spatial positioning information of the intelligent terminal 1 based on the IMU data and the image information or video information of the environment in which the intelligent terminal 1 is photographed by the first imaging device 400, that is, the spatial positioning information of the intelligent terminal 1. 6DOF data (hereinafter abbreviated as 6DOF data) can be determined, that is, six angular degrees of freedom can be obtained based on the translational and rotational degrees of freedom.

Through steps S101 to S103, spatial positioning of the intelligent terminal 1 can be realized. That is, spatial positioning of the intelligent terminal 1 can be realized by the first inertial sensor 700 and the first imaging device 400 disposed in the intelligent terminal 1.

Furthermore, as shown in FIG. 9, the main control chip 600 further includes a second control unit 604 and a second calculation unit 605.
The second control unit 604 is communicatively connected to the function switching module 601, the first imaging device 400, and the second inertial sensor 23, respectively. When the function switching module 601 switches the intelligent terminal to the handheld device function, the second control unit 604 controls the first imaging device 400 to photograph the first infrared sensor 22 located on the handheld device handle, and The second inertial sensor 23 located at the second inertial sensor 23 is used to detect IMU data of the handheld device handle.
The second calculation unit 605 is communicatively connected to the first calculation unit 603, the second control unit 604, the first imaging device 400, and the second inertial sensor 23, respectively. The second calculation unit 604 calculates the spatial positioning information of the intelligent terminal 1 transmitted by the first calculation unit 603, the first light spot image of the first infrared sensor 22 transmitted by the first imaging device 400, and the second inertial sensor. 23, and calculate the spatial positioning information of the intelligent terminal 1, the first light spot image, and the IMU data of the handheld device handle to generate the spatial positioning information of the handheld device handle. used for.

Specifically, FIG. 10 shows a flowchart of the method for spatial positioning of the intelligent terminal 1. That is, the positioning method for spatial positioning of the intelligent terminal in the handheld device shown in FIG. 9 is shown in FIG. The spatial positioning method of the intelligent terminal 1 further includes the following steps S104 to S105.
In step S104, the second control unit 604 controls the first imaging device 400 to take an image of the first infrared sensor 22 located on the handheld device handle 2, and controls the second inertial sensor 23 located on the handheld device handle 2. to detect the IMU data of the handheld device handle 2.
The first imaging device 400 photographs the first infrared sensor 22 of the handheld device handle 2 under the control of the second control unit 604 to form a first light spot image of the first infrared sensor 22 . The second inertial sensor 23 detects the IMU data of the handheld device handle 2 and transmits the IMU data of the handheld device handle 2 to the first spatial positioning unit 606 under the control of the second control unit 604 .

In step S105, the second calculation unit 605 calculates the spatial positioning information of the intelligent terminal 1 transmitted by the first calculation unit 603, the first light spot image of the first infrared sensor 22 transmitted by the first imaging device 400, and Obtain the IMU data of the handheld device handle 2 transmitted by the second inertial sensor 23, calculate the spatial positioning information of the intelligent terminal 1, the first light spot image and the IMU data of the handheld device handle 2, and generate spatial positioning information. In other words, six angular degrees of freedom can be obtained based on the 6DOF data (hereinafter abbreviated as 6DOF data) of the handheld device handle 2, that is, the parallel movement degree of freedom and the rotational degree of freedom.

Through steps S104 and S105, spatial positioning of the handheld device handle 2 can be realized. That is, the first light spot image of the first infrared sensor 22 on the handheld device handle 2 is taken by the first imaging device 400 on the intelligent terminal 1, and the first light spot image is detected by the second inertial sensor 23 located on the handheld device handle 2. Based on the IMU data of the handheld device handle 2, the spatial positioning information of the handheld device handle 2 can be determined.

Exemplary Virtual System As a third aspect of the invention, the invention also provides a virtual system. FIG. 11 shows an operational principle diagram of a handheld device provided by an embodiment of the present invention. As shown in FIG. 11, this virtual system includes the above-mentioned intelligent terminal 1 and a virtual head-mounted display device 31, in which the intelligent terminal 1 is attached to a virtual head that is removably attached to the main body of the virtual head-mounted display device 31. It includes an attached display device 31 and a virtual operation handle 32. The virtual operation handle 32 includes a control handle 33 , a second infrared sensor 34 disposed on the control handle 33 , and a third inertial sensor 35 disposed on the control handle 33 for measuring IMU data of the control handle 33 . including. Control handle 33 and third inertial sensor 35 are each communicatively connected to main control chip 600 .

When the intelligent terminal 1 is attached to the virtual head mounted display device 31, the function switching module 601 switches the function of the intelligent terminal 1 to the virtual head mounted display function. At this time, the intelligent terminal 1, the virtual head-mounted display device 31, and the virtual operating handle 32 constitute a virtual system. For example, the intelligent terminal 1, the VR headset, and the VR handle constitute a VR system.

After the intelligent terminal 1 is attached to the virtual head-mounted display device 31, in the process of the user operating the virtual system, it is necessary to perform spatial positioning for both the intelligent terminal 1 and the virtual operating handle 32. FIG. 12 shows an operational principle diagram of a virtual system provided by another embodiment of the present invention. As shown in FIG. 12, the main control chip 600 includes a third control unit 607 and a third calculation unit 408.
The third control unit 607 is communicatively connected to the function switching module 601, the first imaging device 400, and the first inertial sensor 700, respectively. The third control unit 607 controls the first imaging device 400 to capture a first image of the surrounding environment in which the intelligent terminal is located, and controls the first inertial sensor 700 to detect IMU data of the intelligent terminal. used.
The third calculation unit 408 is communicatively connected to the third control unit 407, the first imaging device 400, the first infrared sensor 22, and the first inertial sensor 700, respectively. A third calculation unit 408 is used to obtain the first image and the IMU data of the intelligent terminal body, calculate the IMU data of the intelligent terminal body and the first image, and generate spatial positioning information of the intelligent terminal. .

Specifically, FIG. 13 shows a flowchart of a method for spatial positioning of an intelligent terminal in a virtual system. That is, FIG. 13 shows a positioning method when spatially positioning an intelligent terminal in a virtual system. As shown in Figure 13,
The spatial positioning method of the intelligent terminal 1 includes the following steps S201 to S203.
In step S201, after receiving the installation information that the intelligent terminal 1 is attached to the virtual head mounted display device 31, the function switching module 601 switches the function of the intelligent terminal 1 to the virtual head mounted display device 31 based on this installation information. Switch to display function.
In step S202, when the function switching module 601 successfully switches the intelligent terminal 1 to the virtual head mounted display function, the third control unit 607 controls the first imaging device 400 located in the intelligent terminal 1 to A first image of the surrounding environment where the intelligent terminal 1 is located is taken, and the first inertial sensor 700 is controlled to detect IMU data of the intelligent terminal 1 .
Under the control of the third control unit 607 , the first imaging device photographs the surrounding environment in which the intelligent terminal 1 is located to form a first image, and transmits the first image to the third calculation unit 608 . The first inertial sensor 700 detects the IMU data of the intelligent terminal 1 under the control of the third control unit 607 and transmits the IMU data of the intelligent terminal 1 to the third calculation unit 608 .
In step S203, after the third calculation unit 608 receives the first image transmitted by the first imaging device 400 and the IMU data of the intelligent terminal 1 transmitted by the first inertial sensor 700, the third calculation unit 608 , calculate the IMU data and the first image of the intelligent terminal main body 100 to generate spatial positioning information of the intelligent terminal 1.

The first inertial sensor 700 is used to detect IMU data of the intelligent terminal body 100. IMU data means three rotation angle degrees of freedom. After the first inertial sensor 700 detects the IMU data of the intelligent terminal body 100, the first inertial sensor 700 transmits the IMU data to the third calculation unit 608. The third calculation unit 608 calculates the spatial positioning information of the intelligent terminal 1 based on the IMU data and the image information or video information of the environment in which the intelligent terminal 1 is located photographed by the first imaging device 400. 6DOF data (hereinafter abbreviated as 6DOF data) can be determined, that is, six angular degrees of freedom can be obtained based on the translational and rotational degrees of freedom.

Through steps S201 to S203, spatial positioning of the intelligent terminal 1 can be realized. That is, spatial positioning of the intelligent terminal 1 can be realized by the first inertial sensor 700 and the first imaging device 400 disposed in the intelligent terminal 1.

Furthermore, FIG. 14 shows an operational principle diagram of a virtual system provided by another embodiment of the present invention. As shown in FIG. 14, the main control chip 600 further includes a fourth control unit 609 and a fourth calculation unit 6091.
The fourth control unit 609 is communicatively connected to the function switching module 601, the first imaging device 400, the second infrared sensor 34, and the third inertial sensor 35, respectively. The fourth control unit 609 controls the first imaging device 400 to photograph the second infrared sensor 34 located on the virtual operation handle when the function switching module 601 switches the intelligent terminal to the virtual head-mounted display function; It is used to control the third inertial sensor 35 located on the virtual operating handle to detect IMU data of the virtual operating handle.
The fourth calculation unit 6091 is communicatively connected to the third calculation unit 608, the first imaging device 400, and the third inertial sensor 35, respectively. The fourth calculation unit 4091 calculates the spatial positioning information of the intelligent terminal 1 calculated by the third calculation unit 608, the second light spot image of the second infrared sensor 34 transmitted by the first imaging device 400, and the third inertial sensor. 35, and calculate the spatial positioning information of the intelligent terminal 1, the second light spot image, and the IMU data of the virtual operating handle to generate the spatial positioning information of the virtual operating handle. used for.

Specifically, FIG. 15 shows a flowchart of a method for spatial positioning of an intelligent terminal of the virtual system shown in FIG. That is, FIG. 15 shows a positioning method when spatially positioning an intelligent terminal of a virtual system. As shown in FIG. 15, the spatial positioning method of the intelligent terminal 1 includes the following steps S204 to S205.
In step S204, the fourth control unit 609 controls the first imaging device 400 to photograph the second infrared sensor 34 located on the virtual operation handle 32, and controls the third inertial sensor 35 located on the virtual operation handle 32. to detect the IMU data of the virtual operation handle 32.
In step S205, the fourth calculation unit 6091 calculates the spatial positioning information of the intelligent terminal 1 calculated by the third calculation unit 608, the second light spot image of the second infrared sensor 34 transmitted by the first imaging device 400, and The IMU data of the virtual operating handle 32 transmitted by the second inertial sensor 23 is acquired, and the spatial positioning information of the intelligent terminal 1, the second light spot image, and the IMU data of the virtual operating handle 32 are calculated. generate spatial positioning information. In other words, six angular degrees of freedom can be obtained based on the 6DOF data (hereinafter abbreviated as 6DOF data) of the virtual operation handle 32, that is, the parallel movement degree of freedom and the rotation degree of freedom.

Through steps S204 and S205, spatial positioning of the virtual operating handle 32 can be realized. That is, the second light spot image of the second infrared sensor 34 on the virtual operating handle 32 is captured by the first imaging device 400 on the intelligent terminal 1, and the second light spot image is detected by the third inertial sensor 35 located on the virtual operating handle 32. Based on the IMU data of the handheld device handle 2, the spatial positioning information of the virtual operating handle 32 can be determined.

Optionally, FIG. 16 shows a structural schematic diagram of a virtual operating handle 32 provided by another embodiment of the present invention. As shown in FIG. 16, the virtual operating handle 32 further includes a handle casing 36. Handle casing 36 includes an annular portion 361 and a grasping portion 362. A recess is provided in the center of the grasping part 362 to accommodate and secure the control handle 33.

Hereinafter, an electronic device according to an embodiment of the present invention will be described with reference to FIG. FIG. 17 shows a schematic structural diagram of an electronic device according to an embodiment of the present invention.

As shown in FIG. 17, electronic device 900 includes one or more processors 901 and memory 902.

Processor 901 may be a central processing unit (CPU) or other type of processing device with data processing and/or information execution capabilities to control other components in electronic device 900 and perform desired functions. can be executed.

Memory 901 may include one or more computer program products. The computer program product may include various forms of computer readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache). The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, and the like. One or more computer program information may be stored on the computer readable storage medium. The processor 901 can execute the program information to implement the intelligent terminal spatial positioning method or other desired functions of the embodiments of the present invention described above.

In one example, electronic device 900 may also include an input device 903 and an output device 904. These components are connected to each other via a bus system and/or other type of connection mechanism (not shown).

This input device 903 may include, for example, a keyboard, a mouse, and the like.

This output device 904 can output various information to the outside. The output device 904 may include, for example, a display, a communication network, and a remote output device to which it is connected.

Of course, for simplicity, FIG. 17 shows only some of the components relevant to the present invention in electronic device 900 and omits components such as buses, input/output interfaces, etc. In addition, depending on the particular application, electronic device 900 may include any other suitable components.

In addition to the methods and apparatus described above, embodiments of the invention may be computer program products that include computer program information. The computer program information, when executed by a processor, causes the processor to perform the steps of a method for spatial positioning of an intelligent terminal according to various embodiments of the invention described herein.

The computer program product may compile program code for carrying out operations of embodiments of the invention in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, and procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on user computing equipment, partially on user equipment, or as an independent software package, partially on user computing equipment and partially remotely. It can be executed on a computing device, or it can be executed entirely on a remote computing device or server.

Additionally, embodiments of the present invention may be a computer readable storage medium having computer program information stored thereon. The computer program information, when executed by a processor, causes the processor to perform the steps of a method for spatial positioning of an intelligent terminal according to various embodiments of the invention herein.

The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example and without limitation, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include communication connections with one or more lead wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM). ), erasable programmable read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any combination of the above. .

The basic principles of the present invention have been explained above along with specific examples, but the advantages, merits, effects, etc. described in the present invention are merely illustrative and are not limiting. It should be noted that no effect or the like should be considered necessary for each embodiment of the invention. Additionally, the specific details disclosed above are for illustrative purposes and to facilitate understanding, and are not limiting; However, the present invention is not limited to the following.

Block diagrams of devices, apparatus, equipment, or systems in accordance with the present invention are illustrative examples only and are not intended to require or imply that the devices, apparatus, equipment, or systems involved may be connected, arranged, or arranged as shown in the block diagrams. It's not a thing. As those skilled in the art will appreciate, these devices, apparatus, equipment, and systems may be connected, arranged, and arranged in any manner. Terms such as "comprising," "comprising," "having," and the like are open terms meaning "including, but not limited to," and can be used interchangeably. As used herein, the terms "or" and "and" mean and can be used interchangeably with the term "and/or," unless the context clearly indicates otherwise. As used herein, the term "such as" means the phrase "such as, but not limited to" and can be used interchangeably.

In addition, in the apparatus, apparatus, and method of the present invention, each member or each step can be disassembled and/or recombined. These decompositions and/or recombinations should be considered equivalents of the present invention.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the invention. Accordingly, this invention is not intended to be limited to the embodiments herein shown, but is to be accorded the widest scope consistent with the principles and novel features of the invention.

The above description is only a preferred embodiment of the invention and does not limit the invention, and all modifications, equivalent substitutions, etc. that are made are within the spirit and principles of the invention. shall be included within the protection scope of the invention-creation.

1-intelligent terminal; 100-intelligent terminal body; 101-first side; 102-second side; 200-display screen; 300-third imaging device; 400-first imaging device; 500-second imaging device; 600 - Main control chip; 601 - Function switching module; 700 - First inertial sensor; 800 - Image processor; 201 - Brightness controller; 602 - First control unit; 603 - First calculation unit; Second control unit 604; Second Calculation unit 605; third control unit 607; third calculation unit 608; fourth control unit 609; fourth calculation unit 6091;
2-handheld device handle; 22-first infrared sensor; 23-second inertial sensor; 24-connector; 25-multifunction pushbutton; 26-function processor; 31-virtual head-mounted display device; 32-virtual operating handle 33-control handle; 34-second infrared sensor; 35-third inertial sensor; 36-handle casing; 361-annulus; 362-grip; 37-third infrared sensor; 38-fourth inertial sensor; 900 - Electronic equipment; 901 - Processor 902 - Memory 903 - Input device; 904 - Output device

Claims (16)

An intelligent terminal,
an intelligent terminal body that can be detachably attached to a virtual head-mounted display device or a handheld device;
a display screen disposed on a first side of the intelligent terminal body;
a function switching module disposed within the intelligent terminal body for controlling switching of the intelligent terminal between a virtual head mounted display function and a handheld device function;
a main control chip disposed within the intelligent terminal body and communicatively connected to the function switching module;
a first imaging device disposed on a second side surface of the intelligent terminal main body, the first imaging device and the second side surface being opposing surfaces;
a first inertial sensor disposed on the intelligent terminal body for detecting IMU data of the intelligent terminal body;
The first imaging device and the first inertial sensor are each communicatively connected to the main control chip. further comprising a brightness controller disposed within the intelligent terminal body;
the brightness controller is communicatively connected to the main control chip and the display screen, respectively;
when the intelligent terminal is switched from the handheld device function to the virtual head mounted display function, the brightness controller reduces the display brightness of the display;
When the intelligent terminal is switched from the virtual head-mounted display function to the handheld device function, the brightness controller increases the display brightness of the display. further comprising an image processor disposed within the intelligent terminal body;
the image processor is communicatively connected to the main control chip;
The intelligent image processor of claim 1, wherein the image processor is used to perform asynchronous space warping, asynchronous time warping, and image rendering processing on image information displayed on the display screen. terminal. further comprising a second imaging device disposed on the first side of the intelligent terminal body;
The intelligent terminal of claim 1, wherein the second imaging device is communicatively connected to the main control chip. The number of the first imaging devices is four, and the four first imaging devices are divided into two imaging device groups, and the two first imaging devices in each imaging device group are connected to the intelligent terminal. Intelligent terminal according to claim 1, characterized in that it is symmetrical about the center of the body. A handheld device,
The intelligent terminal according to claim 1;
a handheld device handle connected to the intelligent terminal;
the handheld device handle is provided with a first infrared sensor and a second inertial sensor, the second inertial sensor is used to detect IMU data of the handheld device handle;
The handheld device according to claim 1, wherein the second inertial sensor is communicatively connected to the main control chip in the intelligent terminal. The main control chip includes a first control unit and a first calculation unit,
The first control unit is communicatively connected to the function switching module, the first imaging device, and the first inertial sensor, and the first control unit controls the first imaging device to determine the position of the intelligent terminal. is used to take a first image of the surrounding environment and control the first inertial sensor to detect IMU data of the intelligent terminal;
The first computing unit is communicatively connected to the first control unit, the first imaging device, the first infrared sensor, the first inertial sensor, and the second inertial sensor, respectively, and the first computing unit is connected to the first inertial sensor. obtaining a first image and IMU data of the intelligent terminal body, and calculating the IMU data of the intelligent terminal body and the first image to be used to generate spatial positioning information of the intelligent terminal; 7. The handheld device of claim 6. The main control chip further includes a second control unit and a second calculation unit,
The second control unit is communicatively connected to the function switching module, the first imaging device, and the second inertial sensor, respectively, and the second control unit is configured to switch the intelligent terminal to a handheld device function. when the first imaging device is controlled to capture a first infrared sensor located on the handheld device handle, and a second inertial sensor located on the handheld device handle is controlled to capture IMU data of the handheld device handle. used to detect,
The second computing unit is communicatively connected to the first computing unit, the second control unit, the first imaging device, and the second inertial sensor, respectively, and the second computing unit is communicatively connected to the first computing unit, the second control unit, the first imaging device, and the second inertial sensor. a first light spot image of the first infrared sensor and IMU data of the handheld device handle transmitted by the second inertial sensor; 8. The handheld device of claim 7, wherein spatial positioning information of the intelligent terminal is used to calculate spatial positioning information of the handheld device handle. further including a connector, a multifunction pushbutton, and a multifunction processor;
the connector is communicatively connected to the handheld device handle and the intelligent terminal, respectively;
the multi-function pushbutton is located on the handheld device handle;
The multi-function processor is connected to the multi-function push button and the main control chip of the intelligent terminal, respectively, and the multi-function processor receives the operation command input by the user through the multi-function push button, and executes the operation. The handheld device according to claim 6, characterized in that the handheld device is used to send commands to the operating command to the main control chip. A virtual system,
The intelligent terminal according to claim 1;
a virtual head-mounted display device, wherein the intelligent terminal is removably attached to the virtual head-mounted display device body;
a virtual head-mounted display handle;
The virtual head-mounted display handle includes:
a control handle;
a second infrared sensor located on the control handle;
a third inertial sensor disposed on the control handle for measuring IMU data of the control handle;
The virtual system wherein the control handle and the third inertial sensor are each communicatively coupled to the main control chip. The virtual head mounted display handle further includes a handle casing;
11. The virtual system of claim 10, wherein the handle casing includes an annular portion and a gripping portion, and a recess is provided in the center of the gripping portion to accommodate and secure the control handle. The main control chip includes a third control unit and a third calculation unit,
The third control unit is communicatively connected to the function switching module, the first imaging device, and the first inertial sensor, and the third control unit controls the first imaging device to determine the location of the intelligent terminal. is used to take a first image of the surrounding environment and control the first inertial sensor to detect IMU data of the intelligent terminal;
The third computing unit is communicatively connected to the third control unit, the first imaging device, the first infrared sensor, and the first inertial sensor, respectively, and the third computing unit is configured to communicate with the first image and the intelligent 10. The method is used to obtain IMU data of a terminal body, calculate the IMU data of the intelligent terminal body and the first image, and generate spatial positioning information of the intelligent terminal. The virtual system described in The main control chip further includes a fourth control unit and a fourth calculation unit,
The fourth control unit is communicatively connected to the function switching module, the first imaging device, the second infrared sensor, and the third inertial sensor, and the fourth control unit is configured to connect the function switching module to the intelligent terminal. When switching to a virtual head-mounted display function, the first imaging device is controlled to take an image of a second infrared sensor located on the virtual operating handle, and the third inertial sensor located on the virtual operating handle is controlled. is used to detect IMU data of the virtual operation handle;
The fourth calculation unit is communicatively connected to the third calculation unit, the first imaging device, and the third inertial sensor, and the fourth calculation unit is configured to receive the second infrared ray transmitted by the first imaging device. Obtaining a second light point image of a sensor and IMU data of the virtual operating handle transmitted by a third inertial sensor; spatial positioning information of the intelligent terminal, the second light spot image and IMU data of the virtual operating handle; 13. The virtual system of claim 12, wherein the virtual operating handle is used to calculate spatial positioning information for the virtual operating handle. An intelligent terminal spatial positioning method for positioning an intelligent terminal according to claim 1, comprising:
The main control chip controls the first imaging device located in the intelligent terminal to take a first image of the surrounding environment in which the intelligent terminal is located, and controls the first inertial sensor to capture a first image of the surrounding environment in which the intelligent terminal is located. detecting IMU data;
the main control chip acquiring a first image of the surrounding environment in which the intelligent terminal is located, taken by the first imaging device;
the main control chip acquiring IMU data of the intelligent terminal body detected by the first inertial sensor;
The method for spatial positioning of an intelligent terminal, characterized in that the main control chip calculates the IMU data of the intelligent terminal body and the first image to generate spatial positioning information of the intelligent terminal. When the intelligent terminal is communicatively connected to a handheld device handle, the spatial positioning method of the intelligent terminal comprises:
the functionality switching module switching the intelligent terminal to handheld device functionality;
The main control chip controls the first imaging device to take an image of a first infrared sensor located on the handheld device handle, and controls a second inertial sensor located on the handheld device handle to take an image of the handheld device handle. detecting IMU data;
The main control chip acquires a first light spot image of the first infrared sensor transmitted by the first imaging device and IMU data of the handheld device handle transmitted by the second inertial sensor, and 15. The method of claim 14, further comprising: calculating spatial positioning information of a terminal, the first light spot image, and IMU data of the handheld device handle to generate spatial positioning information of the handheld device handle. Intelligent terminal spatial positioning method described. When the intelligent terminal is attached to a virtual head-mounted display device, and the intelligent terminal is communicatively connected to a virtual operating handle, the spatial positioning method of the intelligent terminal comprises:
the function switching module switching the intelligent terminal to a virtual head mounted display function;
The control chip controls the first imaging device to take an image of a second infrared sensor located on the virtual operating handle, and controls a third inertial sensor located on the virtual operating handle to capture an IMU of the virtual operating handle. detecting data;
The main control chip acquires a second light spot image of the second infrared sensor transmitted by the first imaging device and IMU data of the virtual operating handle transmitted by a second inertial sensor, and controls the intelligent terminal. 15. The method further comprises: calculating spatial positioning information of the virtual operating handle, the second light spot image, and IMU data of the virtual operating handle to generate spatial positioning information of the virtual operating handle. The spatial positioning method of intelligent terminals described.