KR20200054610A - apparatus, manufacturing and placement method for fixing marker for recognition stability and multiple object tracking with infrared camera used for VR content user location and motion recognition - Google Patents

Abstract

According to one embodiment of the present specification, provided are a head mounted display (HMD) cover and a virtual reality experience system using the same. The HMD cover, which is the cover of the HMD worn on a head of a user and displaying an image in front of an eye of the user, comprises: a body portion detachably coupled to the HMD and surrounding the outside of the HMD; a plurality of insertion portions provided on the body portion; and a marker detachably coupled to the insertion portion.

Description

{Apparatus, manufacturing and placement method for fixing marker for recognition stability and multiple object tracking with infrared camera for tracking multiple objects and securing recognition stability of infrared cameras used to check user location and motion of VR content used for VR content user location and motion recognition}

The present invention relates to an HMD cover and a virtual reality experience system using the same, and in particular, an HMD cover having an insertion hole in which a marker for inserting a marker for securing multiple objects tracking and recognition stability of an infrared camera used for confirming a VR content experiencer's location and motion and It is for a virtual reality experience system using this.

With the recent trend toward miniaturization of display elements, the demand for personal display devices is increasing. The image output from the small image display element is enlarged using an optical system, and the virtual screen is enlarged at a certain distance while the virtual image is seen. 2. Description of the Related Art Display devices that have the same effect as viewing a picture have been developed.

Most of these devices are called head mounted displays (hereinafter referred to as HMDs) because they have a form worn on the head in the same way as glasses, and in general, HMDs use the precision optics included in the eyes of the user. Refers to an image display system that allows a predetermined image to be formed in a single eye or both eyes at an adjacent location.

The HMD is designed to form a very short focal length to exert these device characteristics, and has been mainly used for realizing images such as military, medical and educational imaging systems, and entertainment virtual reality systems. As various hardware such as portable audio / video (AV) systems (e.g., DVD systems, home theaters, etc.) and wired / wireless computers have been developed and distributed in various forms in accordance with the trend of high-grade, HMD's utilization and value of use Is increasing.

In particular, the Oculus Rift incorporates a gyroscope sensor, acceleration sensor, magnetometer, etc., so that the user's head movements are accurately measured in real time, and 3D images matched to the head movements are output. In addition, by forming a marker such as an infrared LED on the surface of the Oculus Rift, it is possible to distinguish not only the head movement but also the position by implementing the positive tracking while detecting the marker with a proximity infrared camera.

Meanwhile, the existing virtual reality experience system was a method in which only one user enjoyed the content. For example, the existing virtual reality experience system was a method in which a single user played a game alone in the content using a head mounted display and a controller.

An object of the present invention is to provide a virtual reality experience system capable of interacting with multiple users and making each user recognizable.

In addition, an object of the present invention is to provide an HMD cover that enables a marker to be inserted or removed by a user and detachable from the HMD.

Another object is to provide an HMD cover that prevents the marker from coming off the HMD cover by an external impact.

The cover of the HMD (Head Mount Display), which is worn on the user's head and displays an image in front of the user's eyes according to the present invention for achieving the above object, is detachably coupled with the HMD and is outside the HMD. Body portion surrounding the; A plurality of insertion portions provided on the body portion; And a marker detachably coupled to the insertion portion.

The marker includes a light emitting portion and a support portion supporting the light emitting portion, and the light emitting portion includes at least one of a reflector capable of reflecting infrared rays, a light emitting body generating infrared rays, and a light emitting diode.

The insertion portion has a hole in the center, and the support portion is screwed to the hole.

When the insert is screwed to the support, the hole surrounds the entire support.

The hole has a diameter smaller than the diameter of the marker.

The body portion is made of any one or more of plastic, photo-curable resin, metal powder.

The body portion includes a front body, a top body, a left body and a right body.

The body portion is divided into first and second segments, the first segment includes a first coupling portion, and the second segment includes a second coupling portion positioned at a point corresponding to the first coupling portion, and the first The coupling portion and the second coupling portion are connected to each other by screwing.

The first segment and the second segment are symmetrical to each other.

The virtual reality experience system using the HMD cover according to the present invention for achieving the above object is worn on the user's head and displays an image in front of the user's eyes (HMD (head mount display), a plurality of markers can be inserted Includes an HMD cover including an insert, a detector for detecting a marker inserted in the HMD cover, and a control device for recognizing a user by recognizing detection information from the sensor and providing image information based on the user's location to the HMD do.

The control device identifies the user based on the position and number of markers inserted in the HMD cover.

The insertion portion is screwed with the marker.

The multi-user virtual reality experience system according to the present invention provides the following effects.

According to the present invention, it is possible to provide an HMD cover for inserting a marker so as not to be separated by an external impact.

In addition, an object of the present invention is to provide an HMD cover that enables a marker to be inserted or removed by a user and detachable from the HMD.

Another object is to provide an HMD cover that prevents the marker from coming off the HMD cover by an external impact.

In addition, it is possible to secure the tracking and recognition stability of multiple objects of the infrared camera used to confirm the location and motion of the VR content experienced person.

The experience system can identify a plurality of users in virtual reality, so that a plurality of users can share one virtual reality.

1 is an exploded perspective view showing a head mounted display (HMD) assembly according to an embodiment of the present invention.
2 is a perspective view of the HMD cover 10 according to an embodiment of the present invention.
3 is a view showing a marker of an embodiment of the present invention.
4 is a block diagram schematically showing the configuration of a multi-user virtual reality experience system according to an embodiment of the present invention.
5A and 5B are diagrams illustrating a plurality of markers having different patterns according to an embodiment of the present invention.
6 is a perspective view of the left HMD cover 10 according to an embodiment of the present invention.
7 is a perspective view of the left and right HMD cover 10 according to an embodiment of the present invention.

In the drawings, the thickness is enlarged to clearly express the various layers and regions. The same reference numerals are used for similar parts throughout the specification. When a portion of a layer, film, region, plate, etc. is said to be “above” another portion, this includes not only the case “directly above” the other portion but also another portion in the middle. Conversely, when one part is "just above" another part, it means that there is no other part in the middle. Further, when a part such as a layer, film, region, plate, etc. is said to be "below" another part, this includes the case where another part is "just below" as well as another part in the middle. Conversely, when one part is "just below" another part, it means that there is no other part in the middle.

The spatially relative terms “below”, “beneath”, “lower”, “above”, “upper”, etc., are as shown in the figure. It can be used to easily describe the correlation of a device or components with other devices or components. The spatially relative terms should be understood as terms including different directions of the device in use or operation in addition to the directions shown in the drawings. For example, if the device shown in the figure is turned over, the device described as "below" or "beneath" the other device may be placed "above" the other device. Accordingly, the exemplary term “below” can include both the directions below and above. The device can also be oriented in other directions, so that spatially relative terms can be interpreted according to the orientation.

In the present specification, when a part is connected to another part, this includes not only the case of being directly connected, but also the case of being electrically connected with another element in between. In addition, when a part includes a certain component, this means that other components may be further included instead of excluding other components, unless otherwise specified.

In the present specification, terms such as first, second, and third may be used to describe various components, but these components are not limited by the terms. The terms are used to distinguish one component from other components. For example, without departing from the scope of the present invention, the first component may be referred to as a second or third component, etc., and similarly, the second or third component may also be alternately named.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings commonly understood by those skilled in the art to which the present invention pertains. In addition, terms defined in the commonly used dictionary are not ideally or excessively interpreted unless explicitly defined.

Hereinafter, an embodiment of the present invention will be described with reference to FIGS. 1 to 4.

1 is an exploded perspective view showing a HMD (Head Mounted Display) assembly according to an embodiment of the present invention, Figure 2 is a perspective view of the HMD cover 10 according to an embodiment of the present invention, Figure 3 is the present invention It is a diagram showing a marker of one embodiment.

The HMD assembly 1 according to one embodiment of the present invention includes an HMD body 20 and an HMD cover 10 into which a marker 130 is inserted.

The HMD body 20 is mounted on the user's head and displays a virtual reality image in front of the user's eyes.

The HMD body 20 may be a general device such as Oculus.

The HMD cover 10 is disposed on the outer surface of the HMD body 20.

The HMD cover 10 according to an embodiment of the present invention may be mounted on, for example, the HMD body 20. The HMD cover 10 may be detachably coupled to the HMD body 20.

2, the HMD cover 10 includes a body portion 110, a plurality of inserting portions 120, and a plurality of markers 130.

The body portion 110 includes a front body 111, a top body 112, a left side body 113, and a right side body (not shown).

The body portion 110 may be manufactured by a 3D printer. For reference, the 3D printer creates a three-dimensional article based on the 3D drawing input by adding the front and back (X-axis), left and right (Y-axis), and up and down (z-axis) movements based on the 3D drawing.

When the body part 110 is manufactured by a 3D printer, the filament may be plastic (PLA, ABS, etc.), photocurable resin, metal powder (titanium, aluminum), or the like. That is, the HMD cover 10 may be made of plastic (PLA, ABS, etc.), photocurable resin, metal powder (titanium, aluminum).

In addition, the body portion 110 may be formed in a mesh structure as shown in FIG. 2. When the body portion 110 is formed of a mesh structure, it is economical because the user can feel lightly when mounted and can reduce the use of the filament.

The front body 111, the top body 112, the left side body 113 and the right side body 114 each include a plurality of inserts 120. In one embodiment of the present invention, a total of 32 insertion parts 120 are not limited thereto.

The plurality of insertion portions 120 each have a hole 120a.

The marker 130 is inserted into the hole 120a of the insertion portion 120.

The marker 130 includes a support portion 132 and a light emitting portion 131 on the support portion 132. The support portion 132 of the marker 130 is inserted into the hole 120a of the insertion portion 120. When the marker 130 is inserted into the hole 120a of the insertion portion 120, the support portion 132 is surrounded by the hole 120a. The support portion 132 is formed with a thread 132a and a threaded bone 132b so as to be screwed to the insertion portion 120. Threads and threaded bones are also formed in the holes 120a of the insertion portion 120, and the threads and screwed bones of the support portion 132 and the hole 120a are fastened to each other.

When the marker 130 is inserted into the hole 120a of the insertion portion 120 and screwed, the support portion 132 is not exposed to the outside. That is, when the support portion 132 is screwed to the insertion portion 120, the hole 120a of the insertion portion 120 surrounds the entire support portion 132. As a result, the support portion 132 can prevent the marker from leaving the object due to an external impact. In addition, the diameter of the hole 120a of the insertion portion 120 is smaller than the marker. When the marker 130 is inserted into the hole 120a of the insertion unit 120, the entire support unit 132 and a part of the light emitting unit 131 may be inserted.

A light emitting diode that emits infrared light may be included in the light emitting unit 131 of the marker 130.

The HMD assembly 1 may further include a gyroscope sensor, an acceleration sensor and a magnetometer. The gyroscope sensor, acceleration sensor, and magnetometer may be included in the HMD body 20, or may be configured separately.

In another modification, the hole is provided with a locking groove, a locking protrusion is provided on the support portion 132 of the marker, and the locking groove and the locking protrusion can be combined by a locking method.

4 is a block diagram schematically showing the configuration of a multi-user virtual reality experience system according to an embodiment of the present invention, and (a) and (b) of FIG. 5 are shown in an embodiment of the present invention. It is a diagram illustrating a plurality of markers having different patterns according to each other.

As shown in Fig. 4, the experience system includes one or more HMD assemblies 1, a plurality of sensors S1, S2, ... Sn) 2, and a control device 3. The HMD assembly 1 and the control device 3 can transmit and receive information to each other. A plurality of detectors (S1, S2, ... Sn) (2) transmits the detection information to the control device 3, the control device 3 is transmitted from the plurality of detectors (S1, S2, ... Sn) (2) Detection information can be received and recognized.

A plurality of detectors (S1, S2, ... Sn) (2) may include an infrared camera. A plurality of detectors (S1, S2, ... Sn) (2) photographs the game area, and transmits the captured information to the control device (3).

A plurality of users with the HMD assembly 1 may be located in the sensing area of the plurality of detectors S1, S2, ... Sn 2.

For example, a first user has a first HMD assembly comprising a first HMD body and a first HMD cover, and a second user has a second HMD assembly comprising a second HMD body and a second HMD cover. The m-th user has a second HMD assembly including the m-th HMD body and the m-th HMD cover.

Each of the HMD covers includes at least one marker, and each of the HMD covers has a different number of markers and one or more of the insertion positions.

The control device 30 may recognize the user according to the number of markers and the insertion position.

Although not shown in the present invention, each user may further have a portable manipulation mechanism having the same shape as a firearm. The shape of the portable operation mechanism is not limited, and may have other shapes. For example, the portable manipulation mechanism may have a sword-like shape. A portable manipulation mechanism cover may also be provided on the portable manipulation mechanism, and a removable marker may be provided on the portable manipulation mechanism cover.

A plurality of detectors (S1, S2, ... Sn) (2) senses the infrared light emitted from the marker (130 in Fig. 1) of the user's HMD cover (10 in Fig. 1). The signals sensed by the plurality of detectors S1, S2, ... Sn 2 are transmitted to the control device 3.

The control device 30 identifies m users from input information transmitted from the plurality of detectors 2 and calculates m positions of m users using motion capture software. In addition, when each of the m users has a portable operation mechanism, the positions of the m portable operation mechanisms are calculated. The input information may be relative coordinates of the marker. The control device 30 may identify the user based on the position of the marker and the number of markers.

For example, the controller 30 assumes a plurality of markers sensed on the HMD cover as arbitrary vertices, and virtually connects neighboring vertices to identify the user according to the shape of the generated polygon. For example, as shown in FIGS. 5A and 5B, polygons A, B, C, and D, which are different from each other, are recognized as different users.

The control device 30 calculates the user's location so that the user's location in the virtual reality image is adjusted in real time.

At this time, the control device 30 updates and outputs m user location data based on the most recent information.

The control device 30 reflects the calculated location data of users in the content logic of the content system. This content logic is logic for controlling game content shared by m users in real time, for example, game themes, game backgrounds, items, types of non-playable characters (NPCs), and placement of NPCs in virtual reality. It can include logic to control.

The control device 30 may transmit the information reflecting the calculated user's location data to each HMD body 20 of the user.

The control device 30 may match and store the ID and user information of the HMD body 20.

6 is a perspective view of the left HMD cover 10 according to an embodiment of the present invention, and FIG. 7 is a perspective view of the left and right HMD cover 10 according to an embodiment of the present invention.

When the HMD cover 10 is manufactured by a 3D printer, the left and right sides may be respectively manufactured. Therefore, the HMD cover 10 may be manufactured by separating the first segment and the second segment. Also, the first segment and the second segment may be symmetrical to each other. The first segment and the second segment may be screwed. For screw coupling, the HMD cover 10 may include one or more engaging portions 140 and 150 in each of the first segment and the second segment. 2, 7 and 7, the coupling part 140 may be provided on the top of the HMD cover 10. Another coupling part 150 may be provided under the HMD cover 10.

The coupling portion 140 has a coupling hole 140a. When the end of the front body of the first segment and the end of the front body of the second segment contact each other, the first coupling portion 140-1 of the first segment and the first coupling portion 140-2 of the second segment Is located at a point corresponding to each other, and the second coupling part 150-1 of the first segment and the second coupling part 150-2 of the second segment are located at a point corresponding to each other.

The first coupling portion 140-1 of the first segment and the first coupling portion 140-2 of the second segment may be screwed together. In addition, the coupling portion 140-1 of the first segment and the coupling portion 140-2 of the second segment may be screwed to each other by separate screws.

The coupling portions of the first segment and the second segment are disposed to face each other, and screws are fastened to the coupling holes of the first segment and the coupling holes of the second segment.

In addition, the contact portion of the front body of the first segment and the second segment may be adhered by an adhesive or the like. The contact portions of the first segment and the second segment may be joined by magnet coupling. For magnet coupling, at least one of the first segment and the second segment may have a magnet disposed on the contact portion, and a magnet or a metal capable of magnet coupling may be disposed on the corresponding other contact portion.

According to an embodiment of the present invention, the marker may be detachably inserted into the insertion portion of the HMD cover 10.

In one embodiment of the present invention, assuming that 5 markers are inserted into the HMD cover 10 with a total of 32 inserts, 2,400 patterns can be arithmetically constructed.

However, in order to reduce errors in head trekking of moving users, even if similar marker patterns are removed, at least 30 patterns can be constructed.

Therefore, when matching different people for each pattern, more than 30 users can be identified.

The embodiments disclosed herein have been described with reference to the accompanying drawings. As described above, the embodiments illustrated in each drawing should not be interpreted as being limited, and may be combined with each other by those skilled in the art who understand the contents of the present specification, and when combined, some components may be omitted.

Here, the terms or words used in the specification and claims should not be interpreted as being limited to ordinary or dictionary meanings, but should be interpreted as meanings and concepts corresponding to the technical ideas disclosed in this specification.

Therefore, the embodiments illustrated in the present specification and the configuration illustrated in the drawings are only exemplary embodiments disclosed in the present specification, and do not represent all of the technical ideas disclosed in the present specification, and thus can replace them at the time of application. It should be understood that there may be equivalents and variations.

10: HMD cover
20: HMD body
110: body part
120: a plurality of inserts
130: marker

Claims (12)

As a cover of the head mounted display (HMD) that is worn on the user's head and displays an image in front of the user's eyes,
A body part detachably coupled to the HMD and surrounding the outside of the HMD;
A plurality of insertion parts provided in the body part; And
Marker detachably coupled to the insert
HMD cover comprising a. According to claim 1,
The marker includes a light emitting portion and a support portion supporting the light emitting portion,
The light emitting unit HMD cover including at least one of a reflector capable of reflecting infrared light, a light emitting element for generating infrared light, and a light emitting diode. According to claim 2,
The insertion portion has a hole in the center, the HMD cover is screwed to the support portion to the hole. The HMD cover of claim 3, wherein when the support portion is screwed, the hole surrounds the entire support portion. The HMD cover according to claim 4, wherein the hole has a diameter smaller than the diameter of the marker. The HMD cover of claim 1, wherein the body portion is made of at least one of plastic, photo-curable resin, and metal powder. The HMD cover of claim 1, wherein the body portion includes a front body, a top body, a left body, and a right body. The body portion of claim 1, wherein the body portion is divided into first and second segments, the first segment includes a first coupling portion, and the second segment is a second coupling portion positioned at a point corresponding to the first coupling portion. Included, the first coupling portion and the second coupling portion HMD cover is connected to each other by screwing. The method of claim 8,
The first segment and the second segment are HMD covers that are symmetrical to each other. A head mounted display (HMD) worn on the user's head to display an image in front of the user's eyes;
HMD cover including a plurality of insertable insertable marker;
A detector detecting a marker inserted in the HMD cover;
A control device that recognizes the user by recognizing the detection information from the sensor and provides image information based on the user's location to the HMD
Virtual reality experience system comprising a. The virtual reality experience system of claim 10, wherein the control unit identifies a user based on the location and number of markers inserted in the HMD cover. The method of claim 10,
The insertion unit is a virtual reality experience system that is screwed with the marker.

Images