KR20170107137A - Head-mounted display apparatus using a plurality of data and system for transmitting and receiving the plurality of data - Google Patents

Head-mounted display apparatus using a plurality of data and system for transmitting and receiving the plurality of data Download PDF

Info

Publication number
KR20170107137A
KR20170107137A KR1020160030427A KR20160030427A KR20170107137A KR 20170107137 A KR20170107137 A KR 20170107137A KR 1020160030427 A KR1020160030427 A KR 1020160030427A KR 20160030427 A KR20160030427 A KR 20160030427A KR 20170107137 A KR20170107137 A KR 20170107137A
Authority
KR
South Korea
Prior art keywords
image data
unit
server
head
camera
Prior art date
Application number
KR1020160030427A
Other languages
Korean (ko)
Other versions
KR101784095B1 (en
Inventor
윤승훈
홍지완
Original Assignee
윤승훈
홍지완
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 윤승훈, 홍지완 filed Critical 윤승훈
Priority to KR1020160030427A priority Critical patent/KR101784095B1/en
Publication of KR20170107137A publication Critical patent/KR20170107137A/en
Application granted granted Critical
Publication of KR101784095B1 publication Critical patent/KR101784095B1/en

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • H04N13/044—
    • H04N5/2257—
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0149—Head-up displays characterised by mechanical features
    • G02B2027/0154—Head-up displays characterised by mechanical features with movable elements
    • G02B2027/0159—Head-up displays characterised by mechanical features with movable elements with mechanical means other than scaning means for positioning the whole image
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0149—Head-up displays characterised by mechanical features
    • G02B2027/0161—Head-up displays characterised by mechanical features characterised by the relative positioning of the constitutive elements

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Controls And Circuits For Display Device (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The head-mounted display device according to the present invention is a head-mounted display device capable of connecting to a server in which video data is stored and capable of receiving the video data. The head-mounted display device includes a communication unit for receiving the video data from the server, A controller for controlling operations of the communication unit and the display unit; and a sensor unit for detecting whether the head movement and the viewing position of the user wearing the head mount display apparatus are changed or not, And the first image data and the second image data are stored in the first image data and the second image data, respectively, wherein the first image data and the second image data include a position of a camera that captures the first image data, And the communication unit is connected to the communication unit First image data from the server and the recipient 2 is characterized in that transmitting the video data at the same time.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a head-mounted display device using a plurality of image data, and a system for transmitting and receiving a plurality of image data,

The present invention relates to a head-mounted display device using a plurality of video data, and more particularly, to a device capable of simultaneously transmitting a plurality of video data photographed at different positions, thereby allowing a user to view photographed videos at different positions without delay .

[0003] As the weight and size of digital devices are increasing, various wearable devices are being developed. A head-mounted display (HMD), which is a type of wearable device, refers to various digital devices that can be worn on a user's head to receive multimedia contents and the like. The head mount display can be implemented in various forms for wearing on the head, such as glasses or helmets.

Behavior-based contents that can maximize the living and real feeling of living by converging stereoscopic images and interactive technologies that can view 3D contents in the same way as actual ones are also being developed. That is, development of experiential contents in the fields of sports, games, entertainment, education and the like is progressing by using a 3D or 360-degree viewable interface based on motion tracking that allows a user to directly interact with a virtual object.

However, the technology development is focused only on the user's movement tracking to be reflected in the contents, and development for providing a variety of services is lacking.

The present invention proposes a device capable of viewing an image while changing positions of images photographed at various positions in a head mount display device without delay.

A positional relationship information is input between image data photographed at different positions so that an image can be viewed while changing a photographing position according to a user's gesture or motion.

The head-mounted display device according to the present invention is a head-mounted display device capable of connecting to a server in which video data is stored and capable of receiving the video data. The head-mounted display device includes a communication unit for receiving the video data from the server, A controller for controlling operations of the communication unit and the display unit; and a sensor unit for detecting whether the head movement and the viewing position of the user wearing the head mount display apparatus are changed or not, And the first image data and the second image data are stored in the first image data and the second image data, respectively, wherein the first image data and the second image data include a position of a camera that captures the first image data, And the communication unit is connected to the communication unit First image data from the server and the recipient 2 is characterized in that transmitting the video data at the same time.

According to an embodiment of the present invention, a plurality of related image data photographed at different positions from a server are all received, and images photographed by cameras at different positions are displayed successively according to the gesture or motion of the user, There is an advantage that an image can be viewed at various positions.

1 is a diagram showing a system configuration to which the concept of the present invention can be applied.
2 is a view showing a configuration of a head-mounted display device according to an embodiment of the present invention.
3 is a view showing cameras for photographing images at different positions.
4 is a diagram for explaining a method of inserting position information for each image data according to an embodiment of the present invention.
5 and 6 are images taken by the first camera and the second camera according to the present embodiment.
FIGS. 7 and 8 are diagrams illustrating a case where object analysis is performed by the image analysis unit of the present embodiment.
9 is a view showing that image data photographed at different positions are simultaneously transmitted to a head-mounted display device according to an embodiment of the present invention.

Hereinafter, the present embodiment will be described in detail with reference to the accompanying drawings. It should be understood, however, that the scope of the inventive concept of the present embodiment can be determined from the matters disclosed in the present embodiment, and the spirit of the present invention possessed by the present embodiment is not limited to the embodiments in which addition, Variations.

Suffix "module" and " part "used in the description relating to the present invention are to be given or mixed in consideration only of ease of specification, and they do not have a meaning or role that distinguish themselves.

1 is a diagram showing a system configuration to which the concept of the present invention can be applied.

 Referring to FIG. 1, the head-mounted display device 500 of the present invention may be configured in the form of glasses that are fixed to the face of the user through the user's ear and nose, but not limited thereto, ). ≪ / RTI >

An image displayed on the head mount display device 500 is an image captured by a camera capable of capturing 360 degrees. When a user turns his / her head while wearing the head mount display device 500, do.

The image source displayed on the head mount display apparatus 500 is similar to the case in which the head mount display apparatus 500 connects to the server 100 through the network 200, The computer 300 can access the server 100 while the computer 500 is connected to the computer 300.

In another embodiment, the smartphone storing images is coupled to the head-mounted display device 500, and the head-mounted display device 500 can display images stored in the smart phone or images received through the smart phone.

The server 100 provides an image displayed on the head mounted display device 500. The server 100 provides images photographed at different positions to the head mounted display device 500 can do.

In detail, the server 100 simultaneously provides the images photographed at a plurality of positions to the head-mounted display apparatus 500 so that the server 100 can enjoy the contents while changing the position on the head-mounted display apparatus 500.

The reason why the server 100 transmits images photographed at different positions to the head-mounted display device 500 at the same time is to reduce a delay caused when a user requests a position change.

That is, when the server 100 provides only the images photographed at a specific position to the head-mounted display apparatus 500 and requests the head-mounted display apparatus 500 to transmit images at other positions, In the case of providing an image, a delay occurs between the images viewed by the user.

2 is a view showing a configuration of a head-mounted display device according to an embodiment of the present invention.

The head mount display device 500 of one embodiment includes an input / output unit 510, a memory 520, a sensor unit 525, a battery 530, a power management unit 535, a communication unit 540, a touch sensor 550 A camera 560, a display unit 570, and a control unit 590.

The input / output unit 510 includes means for receiving a user's input, informing a user of the information, receiving data from the outside, or outputting data to the outside. The input / output unit 510 may include at least one speaker 511, at least one microphone 512, at least one button, a connector, a keypad, or a combination thereof. However, the present invention is not limited thereto.

The speaker 511 can output sounds corresponding to various data to the outside of the head mounted display apparatus 500 under the control of the control unit 590. In addition, the speaker 511 can output sound corresponding to the function performed by the head-mounted display apparatus 500. The speakers 511 may be disposed at one or a plurality of positions at appropriate positions or positions of the head-mounted display apparatus 500.

The microphone 512 receives voice or sound from the outside of the head mounted display apparatus 500, generates an electrical signal based on the received voice or sound, and outputs the generated electrical signal to the control unit 590 . The microphones 512 may be disposed at one or a plurality of positions at appropriate positions or positions of the head mounted display apparatus 500.

The sensor unit 525 may include at least one sensor for detecting the state of the head-mounted display apparatus 500 or the state of the surrounding environment. For example, the sensor unit 525 may include a proximity sensor that detects whether or not the user accesses the head-mounted display apparatus 500, a motion of the head-mounted display apparatus 500 (e.g., rotation, acceleration, deceleration, A light / darkness sensor for detecting light / darkness, an illuminance sensor for detecting ambient illuminance, a photosensor for sensing the color or spectrum of light, or a combination thereof.

Further, the motion / orientation sensor includes at least one of an acceleration sensor, a gravity sensor, a geomagnetic sensor, a gyro sensor, a shock sensor, a GPS, a compass sensor, and an acceleration sensor.

The power management unit 535 supplies power to the head mounted display apparatus 500 under the control of the control unit 590 and is connected to one or more batteries 530 and supplies power to the head mounted display apparatus 500 through a wired cable To the device (500).

The communication unit 540 may be a wired or wireless communication unit or a wired or wireless communication unit. The communication unit 540 may transmit the data from the control unit 590 to the external device through an external communication network or the air in a wired or wireless manner, As shown in FIG. According to an embodiment, the communication unit 540 may include at least one of a mobile communication module, a wireless LAN module, and a local area communication module.

The mobile communication module performs communication with an external device through a mobile communication network using an antenna under the control of the controller 590. [ The mobile communication module may transmit or receive a wireless signal for a voice call, a video call, a short message, or a multimedia message with a cellular phone, smart phone, tablet PC, or other communication device having a network address or telephone number such as IP.

The wireless LAN module may be connected to the Internet through a wireless AP in the vicinity of the wireless AP under the control of the controller 590. [ The wireless LAN module can support the wireless rack standard (IEEE802.11x) of the American Institute of Electrical and Electronics Engineers.

The short-range communication module may perform short-range communication with an external communication device under the control of the controller 590, for example, Bluetooth, infrared communication, Wi-Fi direct communication, NFC or a combination thereof.

The display unit 570 may be implemented in various forms according to the embodiment. For example, the display unit 570 may display an image by forming an image on the user's retina using an optical system.

The control unit 590 controls the display unit 570 to display the image data received through the communication unit 540 or the image data stored in the memory 520.

Particularly, according to the present invention, the control unit 590 or the server 100 of the head-mounted display apparatus 500 can analyze the image data photographed at different positions and set the positional relationship for each image data.

That is, when the user of the head-mounted display apparatus 500 takes a motion / gesture to move left in the image currently being viewed, in order to move the gaze to the camera located on the left side of the image, Establish relationships.

According to an embodiment of the present invention, a positional relationship may be set for each camera in the server 100 providing image data to the head-mounted display device 500, and image data simultaneously received by the head- The positional relationship for each camera may be set.

Hereinafter, for convenience of description, it will be described that the server 100 performs an image data analysis operation, that is, an operation of setting a camera position relation for each image data.

FIG. 3 is a view showing cameras for capturing an image at different positions, and FIG. 4 is a view for explaining a method of inserting position information for each image data according to an embodiment of the present invention.

First, as shown in FIG. 3, cameras 11, 12, 13, and 14 capable of photographing the surroundings at different positions are installed when an image is produced. At this time, the cameras are devices capable of photographing the surround (for example, 360 degrees) such that the head-mounted display device tracks the movement of the user's head while displaying images corresponding thereto.

It is assumed that the first camera 11 to the fourth camera 14 are installed at the first position to the fourth position which are different positions.

And is provided to an image analysis unit 20 for acquiring position information of each camera with respect to images photographed by the first to fourth cameras 11, 12, 13, The image analysis unit 20 may be configured in a server 100 capable of transmitting image data to a head mounted display device through a network or the image analysis unit 20 may be configured in the head mounted display device.

It is assumed that the image analysis unit 20 is configured in the server 100.

The image analysis unit 20 of the server 100 includes an object analysis unit 21 that receives images from the first to fourth cameras 11, 12, 13, and 14 and performs object analysis, A camera position determination unit 22 for determining a camera position based on the positions of the analyzed objects, and a position information insertion unit 23 for inserting and storing camera position information in the captured image data .

FIGS. 5 and 6 are images taken by the first camera and the second camera according to the present embodiment, and FIGS. 7 and 8 are views showing the case of performing object analysis by the image analysis unit of the present embodiment.

The object analyzing unit 21 of the image analyzing unit 20 receives the captured images of the cameras. For example, the object analyzing unit 21 may include a first image 11a captured by the first camera, And receives the second image 12a.

The object analyzer 21 performs an object analysis included in the first image 11a and the second image 12a. According to the present embodiment, the object analyzer 21 analyzes the position And a variable object whose position or shape is changed are discriminated.

7, the object analyzer 21 marks the sub objects 711, 712, 721, and 722, which are determined to indicate the characteristics of the object, in order to extract the object in the image.

In particular, the object analyzer 21 extracts second sub-objects 721 and 722 that hold a fixed position in the image to identify an object whose position is fixed in the image, Or the first sub-objects 711 and 712 whose shape is changed are extracted.

The object analyzer 21 includes a first object 710 formed by first sub-objects 711 and 712 whose positions or shapes are variable and a second object 710 formed by a second sub- (720) formed by the first and second objects (721, 722).

The object identifying unit 21 performs an object identifying operation on the second image 12a. As shown in FIG. 8, a first object 810 composed of first sub-objects 811 and 812, And a second object 820 composed of second sub-objects 821 and 822. [

Next, the camera position determination unit 22 determines a relative position of each camera using the first object (variable object) extracted by the object analysis unit 21 and the second object (fixed object). Although it is of course possible to input positional information for each camera directly or to use GPS information or the like before an image is taken through each camera for video production from the beginning, when an image is shared among different users It is necessary to set the positional relationship for each camera.

The camera position determination unit 22 determines the positions, angles, and distances in the image for the second objects 720 and 820 as fixed objects and the position, angle, and distance in the image for the first objects 710 and 810, The relative position of each camera is determined by using an angle or a distance change.

For example, the first camera 11 that has photographed the image shown in Fig. 7 determines that the camera is located on the left side of the second camera 12 that photographed the image shown in Fig. The distance and angle between the first object 710 and the second object 720 in the first image and the distance and angle between the first object 810 and the second object 820 in the second image, Can be used. The size and angle of the second objects 720 and 820, which are fixed objects, may use the change values in the first and second images.

Through the object analysis, the camera position determination unit 22 determines a relative position for each camera.

Then, the position information inserting unit 23 inserts relative position information in each image data. The reference position information may be inserted in the first image data and the first position information on the right side may be inserted into the second image data from the reference position. For example, the (0,0) position is inserted in the first video data, the (1,0) position is inserted in the second video data, and the (2,0) .

After the relative position information is inserted into the image data, the server 100 transmits the four image data to the head-mounted display device 500 or the computer 300 at the same time.

9 is a view showing that image data photographed at different positions are simultaneously transmitted to a head-mounted display device according to an embodiment of the present invention.

According to the present embodiment, when requesting the server 100 to transmit the first image data from the head mount display device (communication unit), the server 100 transmits the first image data 901, 902, 903, and 904 to which the relative positional relationship with the first image data 901 is connected.

When the user's command or the gesture / motion of the user is input while the first image data 901 is being displayed on the head-mounted display device 500, any one of the second image data to the fourth image data Image data is displayed without delay.

When the user moves the position to the right by one half while the user is watching the first image data 901, the gesture or motion of the user is displayed on the head mount display device 500, Through the sensor unit. Then, it is checked whether there is image data photographed from the right side of the first image data 901, and the second image data 902 judged to have been photographed by the right side camera is displayed on the basis of the relative position information .

In detail, the sensor unit of the head-mounted display apparatus 500 not only monitors the head movement of the user but also detects whether the viewing position of the user is moved forward or backward or leftward or rightward, and the control unit of the apparatus 500 And controls the image data to be variable according to the detected viewing position.

According to an embodiment of the present invention, a plurality of related image data photographed at different positions are received from the server 100, and images photographed by cameras at different positions are successively displayed according to a user's gesture or motion , It is possible to view images at various positions without delay.

Claims (7)

A head-mounted display device capable of connecting to a server in which video data is stored and receiving the video data,
A communication unit for receiving the image data from the server;
A display unit for displaying image data transmitted by the communication unit,
A control unit for controlling operations of the communication unit and the display unit,
And a sensor unit for detecting a head movement and a viewing position of a user wearing the head mount display apparatus,
Wherein the image data includes first image data and second image data photographed at different positions, and in the first image data and the second image data, a position of a camera that photographs the first image data, The relative position information of the cameras that have captured the video data is inserted,
Wherein the communication unit receives the first image data and the second image data simultaneously from the server. The method according to claim 1,
The server analyzes the first image data and the second image data and analyzes the relative positions of the cameras that respectively photograph the first image data and the second image data. The server analyzes the first image data and the second image data, Mount display device. The method according to claim 1,
And an image analyzer for analyzing the first image data and the second image data and analyzing the relative positions of the cameras that respectively photograph the first image data and the second image data, Device. The method according to claim 2 or 3,
Wherein the image analyzing unit comprises:
An object analyzer for extracting a fixed object whose position or shape is fixed in the image of the first image data and the second image data and a variable object whose position or shape is changed,
A camera position determination unit that determines a relative position between a first camera that has captured the first image data and a second camera that captured the second image data using the distance or angle between the objects analyzed by the object analyzing unit, Wealth,
And a position information insertion unit for inserting the relative position information between the first camera and the second camera into the first image data and the second image data analyzed by the camera position determination unit, Device. The method according to claim 2 or 3,
Wherein when the server receives the video data transmission request from the communication unit, the server transmits the video data to which the requested video data and the relative position information are coupled together. A server for storing image data; And
And a head mount display device for requesting the server to transmit the image data or receiving the image data transmitted from the server,
The head mounted display device includes a communication unit for receiving the image data from the server, a display unit for displaying image data transmitted by the communication unit, a control unit for controlling operations of the communication unit and the display unit, And a sensor unit for detecting head movement and viewing position fluctuation of a user wearing the apparatus,
The server includes an object analyzing unit for analyzing an object of each of the images captured by the cameras taken at different positions, a camera position determining unit determining the camera position based on the positions of the analyzed objects, And a position information inserting unit for inserting and storing position information of the camera in the photographed image data. The method according to claim 6,
Wherein the server transmits to the communication unit the video data to which the requested video data and the relative position information are connected, when the video data transmission request is received from the communication unit. .
KR1020160030427A 2016-03-14 2016-03-14 Head-mounted display apparatus using a plurality of data and system for transmitting and receiving the plurality of data KR101784095B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020160030427A KR101784095B1 (en) 2016-03-14 2016-03-14 Head-mounted display apparatus using a plurality of data and system for transmitting and receiving the plurality of data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020160030427A KR101784095B1 (en) 2016-03-14 2016-03-14 Head-mounted display apparatus using a plurality of data and system for transmitting and receiving the plurality of data

Publications (2)

Publication Number Publication Date
KR20170107137A true KR20170107137A (en) 2017-09-25
KR101784095B1 KR101784095B1 (en) 2017-10-11

Family

ID=60035096

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020160030427A KR101784095B1 (en) 2016-03-14 2016-03-14 Head-mounted display apparatus using a plurality of data and system for transmitting and receiving the plurality of data

Country Status (1)

Country Link
KR (1) KR101784095B1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20190118939A (en) * 2018-04-11 2019-10-21 이보람 Mr telescope and system and method for operating the same
KR102048354B1 (en) * 2019-07-10 2020-01-23 한화시스템(주) Apparatus and method for displaying images in a wireless helmet
KR102125299B1 (en) * 2020-02-05 2020-07-07 한화시스템 주식회사 System and method for battlefield situation recognition for combat vehicle

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102103404B1 (en) * 2019-05-23 2020-04-23 주식회사 피앤씨솔루션 A operation situation sharing system using head mounted display apparatus

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
IL139995A (en) * 2000-11-29 2007-07-24 Rvc Llc System and method for spherical stereoscopic photographing

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20190118939A (en) * 2018-04-11 2019-10-21 이보람 Mr telescope and system and method for operating the same
KR102048354B1 (en) * 2019-07-10 2020-01-23 한화시스템(주) Apparatus and method for displaying images in a wireless helmet
KR102125299B1 (en) * 2020-02-05 2020-07-07 한화시스템 주식회사 System and method for battlefield situation recognition for combat vehicle

Also Published As

Publication number Publication date
KR101784095B1 (en) 2017-10-11

Similar Documents

Publication Publication Date Title
JP7408678B2 (en) Image processing method and head mounted display device
RU2670784C9 (en) Orientation and visualization of virtual object
US20230141039A1 (en) Immersive displays
US10268276B2 (en) Autonomous computing and telecommunications head-up displays glasses
US9245389B2 (en) Information processing apparatus and recording medium
CN111445583B (en) Augmented reality processing method and device, storage medium and electronic equipment
US9529428B1 (en) Using head movement to adjust focus on content of a display
WO2013166362A2 (en) Collaboration environment using see through displays
WO2013028586A1 (en) Location based skins for mixed reality displays
KR101784095B1 (en) Head-mounted display apparatus using a plurality of data and system for transmitting and receiving the plurality of data
US20150172634A1 (en) Dynamic POV Composite 3D Video System
WO2021147921A1 (en) Image processing method, electronic device and computer-readable storage medium
KR20150044488A (en) Method and system of operating image contents for video see-through head mounted display
CN112835445A (en) Interaction method, device and system in virtual reality scene
CN111103975B (en) Display method, electronic equipment and system
EP3402410B1 (en) Detection system
JP2005303683A (en) Image transceiver
US10902617B2 (en) Data processing for position detection using optically detectable indicators
CN111982293A (en) Body temperature measuring method and device, electronic equipment and storage medium
JP2021125789A (en) Image processing device, image processing system, image processing method, and computer program
WO2016182504A1 (en) A virtual reality headset
KR20160013853A (en) A head mounted display and a method for providing audio signal using the same
US20230217007A1 (en) Hyper-connected and synchronized ar glasses
GB2571286A (en) Virtual reality
US20240078734A1 (en) Information interaction method and apparatus, electronic device and storage medium

Legal Events

Date Code Title Description
E701 Decision to grant or registration of patent right
GRNT Written decision to grant